Read on Twitter
#Python tip: #hypothesis is good at finding bugs; however, often as not, the bug is in your understanding of what the code is supposed to do.
1/
1/
Initial belief: The JSON module is buggy because #hypothesis finds cases that don't round-trip.
Bugs in understanding:
* The JSON spec doesn't have NaNs
* A JSON module feature is that lists and tuples both serialize into arrays but can't be distinguished when deserialized.
2/
Bugs in understanding:
* The JSON spec doesn't have NaNs
* A JSON module feature is that lists and tuples both serialize into arrays but can't be distinguished when deserialized.
2/
Initial belief:The colorsys module is buggy because #hypothesis finds conversions that don't round-trip
Bugs in understanding:
* color gamuts are limited
* colors in one gamut may not be representable in another
* implementations are constrained by underlying specifications
3/
Bugs in understanding:
* color gamuts are limited
* colors in one gamut may not be representable in another
* implementations are constrained by underlying specifications
3/
Initial belief: A quadratic formula function is buggy because #hypothesis finds cases that don't round-trip.
Bugs in understanding:
* Floating point numbers differ from Real numbers: representation errors, round-off errors, catastrophic cancellation, overflow/underflow, ...
4/
Bugs in understanding:
* Floating point numbers differ from Real numbers: representation errors, round-off errors, catastrophic cancellation, overflow/underflow, ...
4/
So, the reason I like #hypothesis is that it falls in the same category as Alloy and TLA⁺.
They are tools that reveal flaws in your understanding or specifications of problems.
5/
They are tools that reveal flaws in your understanding or specifications of problems.
5/
Particularly for floating point code, by the time you've added enough constraints to satisfy #hypothesis, you've developed a pretty good understanding of what your code actually does.
And that is valuable even if the code under test didn't have any bugs from the outset.
6/
And that is valuable even if the code under test didn't have any bugs from the outset.
6/
