PHPBench is over 5 years old. I started working on it because I wanted to write a new implementation of the PHPCR spec, and I wanted to ensure that it was faster than the existing Jackalope implementation, I needed a framework to write a large number of performance tests. The existing Athletic framework didn't quite meet my needs so I decided to write my own.
I didn't write the next PHPCR implementation (luckily, it would have been terrible) but I did spend the next several months working on PHPBench, a large part of it was written while I was cycling from Vorarlberg (Austria) to Ankara going back through Greece and up to Slovenia. I gave one of my first talks ever at the Istanbul PHP User Group.
Lots of experiments were carried out, some of which really useful (retry threshold), some were awesome in a nerdy way, and some were, in retrospect, bad ideas (implementing a JSON query language to query a database - now fortunately removed).
After a more or less active period of a year, my interest dropped off and I started working on Phpactor which has sucked much of my time and enthusiasm over the past years during which PHPBench has seen increasing adoption:
But I've never quite been happy with it:
- Comparing two result sets (for regressions) was not convenient at all.
- There was no solution for running PHPBench meaningfully in a CI environment.
This year I had a one month holiday, and after cycling to Scotland for two weeks I arrived at my brothers place in the middle of the Galoway forest in Scotland. It was 13 miles to the nearest shop and I had two weeks of spare time.
I started by working on my usual side-project, Phpactor, and it required some performance testing, I used PHPBench and was pained that I couldn't easily compare the performance improvement with some refactored code.
I switched projects half way through the feature (leaving Phpactor's
in a precarious state) and spent over a week working on new PHPBench features.
- You can use a previous suite as a baseline when running your benchmarks and generating reports.
- The Assertion feature has been completely re-implemented with a DSL
- You can assert against the baseline (within a margin of error) - which hopefully will allow regression testing in CI environments.
- Features have been removed.
- The README and documentation has been reviewed and updated.
- The codebase has variously updated, notably real type-hints have been added internally with the help of Rector.
- XDebug profile generation is now included by default.
Showing the % difference to a previous run
So after 5 years and 17 pre-alpha versions
1.0.0-alpha1 has been tagged.
If you use PHPBench, please give it a try - in particular I'd like to hear about success (or not) stories about using the baseline feature in a CI environment.