Read on Twitter
Git bisect 
a thread 
.
a thread .
I assume you've used or heard of git bisect.
Recap:
1. `git bisect start BAD GOOD`
2. <RUN TEST>
3 label result with `git bisect good/bad`
4. Goto 2 until git bisect tells you it's done.
A longer intro: https://hackernoon.com/understanding-git-bisect-i-e-use-binary-search-to-find-the-change-that-introduced-a-bug-89489b4c9fa6
Recap:
1. `git bisect start BAD GOOD`
2. <RUN TEST>
3 label result with `git bisect good/bad`
4. Goto 2 until git bisect tells you it's done.
A longer intro: https://hackernoon.com/understanding-git-bisect-i-e-use-binary-search-to-find-the-change-that-introduced-a-bug-89489b4c9fa6
Do yourself the favor and always start git bisect with the good and bad commits!
That way you can "git bisect reset" it later, and start over using your shell history.
`git bisect start BAD GOOD`
`git bisect start main v0.1.0` # Assuming that main is bad and v0.1.0 is good.
That way you can "git bisect reset" it later, and start over using your shell history.
`git bisect start BAD GOOD`
`git bisect start main v0.1.0` # Assuming that main is bad and v0.1.0 is good.
You probably want `--first-parent` (needs newest Git).
In good Git tradition, it is confusingly named. Essentially it only considers changes directly on main. Feature branches are ignored.
`git bisect start --first-parent main v0.1.0`
https://git-scm.com/docs/git-bisect#Documentation/git-bisect.txt---first-parent
In good Git tradition, it is confusingly named. Essentially it only considers changes directly on main. Feature branches are ignored.
`git bisect start --first-parent main v0.1.0`
https://git-scm.com/docs/git-bisect#Documentation/git-bisect.txt---first-parent
I always mess up, so I save the result:
1. `git bisect log > ../bisect.log`
2. Fix your mistakes in ../bisect.log
3. Start again: `git bisect replay ../bisect.log`
(Put the log outside your git dir, otherwise your repo is dirty.)
1. `git bisect log > ../bisect.log`
2. Fix your mistakes in ../bisect.log
3. Start again: `git bisect replay ../bisect.log`
(Put the log outside your git dir, otherwise your repo is dirty.)
git bisect run is awesome and plays well with Unix tools.
`git bisect run make` if you use make.
`git bisect run make -C <build_dir>` if you use CMake and have configured it once before. (do yourself a favor and put
it completely outside your repo)
`git bisect run make` if you use make.
`git bisect run make -C <build_dir>` if you use CMake and have configured it once before. (do yourself a favor and put
it completely outside your repo)
This works for almost anything.
`git bisect run cargo test`
`git bisect run npm test`
`git bisect run yarn test`
`git bisect run cargo run -- ...`
If not write a script: >>
`git bisect run cargo test`
`git bisect run npm test`
`git bisect run yarn test`
`git bisect run cargo run -- ...`
If not write a script: >>
Run scripts like so: `git bisect run ~/custom_script.sh`
1. Put them outside the repo.
2. Test them manually first.
3. "set -e" is good.
4. Know about exit codes. 0: good, 1-124: bad. 125: skip. 126-255: abort.
5. Only one line should return "bad" (1-124).
6. 125 is for skipping
1. Put them outside the repo.
2. Test them manually first.
3. "set -e" is good.
4. Know about exit codes. 0: good, 1-124: bad. 125: skip. 126-255: abort.
5. Only one line should return "bad" (1-124).
6. 125 is for skipping
Use the aborting exit codes to know where the script failed.
cmake -B /tmp/build || exit 128
make -C /tmp/build || exit 129
The exit code will tell you which stepped failed.
(If you want to skip build failures, use exit code 125.)
cmake -B /tmp/build || exit 128
make -C /tmp/build || exit 129
The exit code will tell you which stepped failed.
(If you want to skip build failures, use exit code 125.)
Some scripts take very long. The below command tells you how many commits remain. You can run it in a second terminal.
`git bisect visualize --oneline | wc -l`.
(Num of steps is approx log2 of this.)
`git bisect visualize --oneline | wc -l`.
(Num of steps is approx log2 of this.)
After your script has run, immediately save the run in a log!
`git bisect log > ../bisect_run.log`
Again `git bisect replay` let's you fix any mistakes.
`git bisect log > ../bisect_run.log`
Again `git bisect replay` let's you fix any mistakes.
Check your bisect results! (This literally happened to me while writing this thread.)
Repeat your script and see if the output of the first breaking commit is the same error that you were looking for.
If not, add `grep -q`
Repeat your script and see if the output of the first breaking commit is the same error that you were looking for.
If not, add `grep -q`
Things where git bisect will not work.
- Undefined Behavior.
- Non-deterministic builds. Clean your build. Lock your manifest file!
- In summary, if the result depends on the previous checked out commit in anyway the whole process falls apart.
- Undefined Behavior.
- Non-deterministic builds. Clean your build. Lock your manifest file!
- In summary, if the result depends on the previous checked out commit in anyway the whole process falls apart.
Prefer correctness over speed, especially if you wrote a run
script.
There will be ~10 steps. If it is automatic and runs in the background it can take twice the time. Having to do it three times because the script found the wrong answer because you messed up takes longer.
script.
There will be ~10 steps. If it is automatic and runs in the background it can take twice the time. Having to do it three times because the script found the wrong answer because you messed up takes longer.
Hilarious things that went wrong WHILE WRITING THIS THREAD.
1. Did not use `cargo clean` and regretted it.
2. Did not grep for the exact error and found a different error.
1. Did not use `cargo clean` and regretted it.
2. Did not grep for the exact error and found a different error.
All your labels are bad or good? Your script is probably broken.
Remember, if the bug introduction is uniformly distributed, binary search should have bad/good 50% each. Even if there's a heavy skew because you started with a really early version, 100% one way is unlikely.
Remember, if the bug introduction is uniformly distributed, binary search should have bad/good 50% each. Even if there's a heavy skew because you started with a really early version, 100% one way is unlikely.
git bisect is mostly good for features that you didn't have under test.
Bad reason: Unit tests fail. (Just add 'em to CI.)
Good reason: A user reports a regression on a platform you cannot easily have in CI.
Bad reason: Unit tests fail. (Just add 'em to CI.)
Good reason: A user reports a regression on a platform you cannot easily have in CI.
Want to learn how the algorithm works? And why it is not simply binary search?
@ChristianCouder wrote about it:
https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.html
@ChristianCouder wrote about it:
https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.html
And finally, if you ever find a good use case for probabilistic (Bayesian Search) bisect, please let me know.
I haven't had truly random bugs. Mostly they're just depending on something outside "the system".
https://github.com/siedentop/bbchop
https://en.wikipedia.org/wiki/Bayesian_search_theory
I haven't had truly random bugs. Mostly they're just depending on something outside "the system".
https://github.com/siedentop/bbchop
https://en.wikipedia.org/wiki/Bayesian_search_theory
Happy Bisecting!
Resources:
[1]: https://git-scm.com/docs/git-bisect#Documentation/git-bisect.txt---first-parent
[2]: https://twitter.com/fhinkel/status/1092413239395344386
[3]: https://hackernoon.com/understanding-git-bisect-i-e-use-binary-search-to-find-the-change-that-introduced-a-bug-89489b4c9fa6
[4]: https://stackoverflow.com/questions/17267816/git-bisect-with-merged-commits
[5]: https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.html
[6]: https://github.com/siedentop/bbchop
[1]: https://git-scm.com/docs/git-bisect#Documentation/git-bisect.txt---first-parent
[2]: https://twitter.com/fhinkel/status/1092413239395344386
[3]: https://hackernoon.com/understanding-git-bisect-i-e-use-binary-search-to-find-the-change-that-introduced-a-bug-89489b4c9fa6
[4]: https://stackoverflow.com/questions/17267816/git-bisect-with-merged-commits
[5]: https://mirrors.edge.kernel.org/pub/software/scm/git/docs/git-bisect-lk2009.html
[6]: https://github.com/siedentop/bbchop
