PHPBench 1.0.0-alpha5
Last modified 2022/09/14 11:14PHPBench Alpha 5 as been tagged.
I tagged PHPBench Alpha 1 in about 5 months ago since that time numerous improvements and features have been introduced, I will talk about three of them.
Local Executor ¶
One problem I had when developing benchmarks is that it’s difficult to debug
when the benchmark launches in a separate process, for this reason there is
now the local
executor:
$ phpbench run --executor=local
Will run the benchmark in the same process as PHPBench, allowing you to do
whatever you need to do (var_dump($foo);die()
) to debug your benchmark.
Better Assertion and Baseline Feedback ¶
The default progress logger has been simplified. If a --ref
baseline
reference is given it also shows the percent difference:
If an assertion fails it is highlighted RED, if it is tolerated it is CYAN and if the assertion passes unambiguously it is GREEN.
There will be more work on the real-time progress output in subsequent
versions reducing the need for using --report=aggregate
allowing faster
feedback.
New Assertion DSL ¶
The original assertion DSL introduced in alpha1
was quite limited. Probably
the biggest issue was that you could only assert against hardcoded and
pre-calculated values, other than that it only allowed simple comparisons.
The assertions now use a custom expression language is much more comprehensive featuring (for example) arithmetic, logical operators, operator precdence, and functions.
/**
* @Assert("mode(variant.time.avg) as ms < mode(baseline.time.avg) as ms +/- 5%")
*/
public function benchFoo(): void
{
}
It provides full access to ANY metrics which were recorded against an
iteration, by default time
and memory
- but extensions
could theoretically provide, for example, the number of function calls.
Assertion results are displayed and partially evaluated to provide better context.
Integrated Documentation ¶
The new examples in the documentation are included from code which is executed in continuous integration.
Next Steps ¶
I think PHPBench 1.0.0 is getting closer. The new expression language can also help with other aspects of PHPBench:
- Reporting: The current “report generator” is inflexible. With the
new expression language it should be far easier to achieve the same thing
with only expressions:
--report='{"name": "benchmark.name", "mode": "mode(variant.time.avg) as ms", "best": "min(variant.time.avg) as ms", "stdev": "stdev(variant.time.avg) as ms"}'
- Progress: The ability to easily customize the progress output, for
example:
format("%s %s vs %s", variant.benchmark.name,mode(variant.time.avg) as ms, mode(baseline.time.avg) as ms