Automatic Testing of Python Functions Based on Contracts

How to improve the correctness of your programs with little additional effort

Marko Ristin, Phillip Schanely, Lauren De bruyn

Code Analysis Programming Software Design Testing

See in schedule: Fri, Jul 30, 11:10-11:55 CEST (45 min) Download/View Slides

Good programs are correct programs. Testing is an indispensable tool making sure that the software we produce is correct. The bulk of today’s Python programs are tested using unit tests, on a case-by-case basis. While unit tests are important, thorough tests are tedious to write. A lot of edge cases need to be considered and human errors easily slip in.

The edge cases are often simply omitted, e.g., for the lack of time. However, this leaves a lot of untested holes in the program. Instead of hunting for edge cases, what if we specified the behavior of a function and let the development tools test it for us?

In this talk, we first look into how to specify the behavior of functions using contracts, i.e. the properties of the input and the output of a function. The contracts can reduce the amount of testing code, and allow the developer to focus on more tangible pieces of code, namely defining the behavior.

Next, we examine two of our tools for automatic testing based on contracts (crosshair and icontract-hypothesis). We show how they can be used in everyday programming with little additional effort. A corpus of programs (Python-by-contract) is introduced to evaluate where the tools shine and what limitations they hit. Finally, we show how to use the tools in the classroom to help students with debugging, providing hopefully a better educational experience.

The talk is aimed at Pythonistas familiar with lambda functions, Boolean logic (“and”, “or”) and quantifiers (“all”, “any”).

Type: Talk (45 mins); Python level: Intermediate; Domain level: Beginner