Benchmarking PHP 7.3 vs 7.4 with Symfony 4.4 (+ trouble with OPCache Preloading)

PHP 7.4 version brings in a slew of new features to the language, including a notable feature for performance: OPCache Preloading. It promises significant gains for complex PHP apps where the bootstrap is sizeable. I wanted to find out how realistic these claims were and if the feature is ready for prime time in real world deployments.

I decided to do a round of benchmarks with eZ Platform, a complex Symfony application on Symfony 4.4, which supports OPCache Preloading. If you're not familiar with eZ Platform, it is our Digital Experience Platform (DXP) for creating websites and web applications. Our foundation is the Symfony framework and we just upgraded eZ Platform to Symfony 4.

Unfortunately at the time of writing we did not have a release that is fully certified to work with Symfony 4.4 and PHP 7.4, so I ended up stitching together a version just for these benchmarks: github.com/janit/ezplatform-php74. This version represents a development version of our upcoming eZ Platform 3.0.0 release scheduled to be available in 2020.

The codebase is not optimized and not all features are guaranteed to work, but all the major features are in place. I felt it would be able to give some insight into relative performance improvements between PHP 7.3, 7.4 and 7.4 with OPCache Preloading.

The benchmark target is a simple landing page which loads and renders some content from our content repository (data model), whose SQL calls is effiently cached. TTL was set to zero so I would not be get numbers for caching, but pure processing throughput. Benchmarks were ran three times and the average was used in the report below.

I used dedicated VPSes for the target application and the load generator. The target server had 2 vCPU and 4 GB of RAM assigned. Both were Ubuntu 18.04 LTS with PHP packages (7.3 and 7.4) from Sury repositories with nginx 1.14.0 for the web server with PHP-FPM. Default settings for PHP was used, except for opcache.validate_timestamps which was set to off.

Running into trouble with OPCache Preloading

Since Symfony 4.4 and greater versions support OPCache Preloading out of the box, enabling in theory requires defining two configuration values in your php.ini file:

opcache.preload_user=www-data
opcache.preload=/var/www/ezplatform/var/cache/prod/srcApp_KernelProdContainer.preload.php

Initially I ran into some trouble of not having the generated preload file in place since I was clearing caches using the dev environment, instead of prod. Once this was in place I ran into another issue, with the system reporting a segfault in the FPM log (/var/log/php7.4-fpm.log):

[05-Dec-2019 12:57:53] WARNING: [pool www] child 2194 exited on signal 11 (SIGSEGV - core dumped) after 0.952515 seconds from start

The preloading caused the app to crash immediately, so I decided to dig underneath the hood a little bit and poke around with the generated preload file. I found that by commenting the includes for the Symfony container the constant segfaults were gone:

// require __DIR__.'/ContainerNacE9A2/srcApp_KernelProdContainer.php';

After removing this line PHP-FPM would not crash immediately, but start throwing seemingly random errors regarding missing methods of classes in dependencies:

2019/12/05 13:56:42 [error] 6644#6644: *131 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Uncaught Error: Call to undefined method Hoa\Consistency\Consistency::flexEntity() in /var/www/ezplatform/vendor/hoa/consistency/Consistency.php:361

So obviously the kernel was included for a reason and the system was not finding resources is was expecting. I noticed the compiled Symfony kernel was quite large at 1.3 MB, so I so I tried upping memory limits for OPCache and PHP itself. Generous resources did not help.

After reading through a few bug reports like Preloading segfaults at preload time and at runtime, I decided not to go further down the rabbit hole at this point in time. I already had benchmark results for PHP 7.3 vs. PHP 7.4 and that was already worth reporting.

Benchmark results and conclusion

Here is the chart comparing the throughput of the demo installation with PHP 7.3 and 7.4:

It seems that at higher concurrency PHP 7.4 maintains a significant lead over PHP 7.3 consistently without any code or configuration changes. I was not able to test and verify the claims that OPCache Preloading would bring an additional 30% to 50% improvement in performance, but this is an significant improvement over the previous version of PHP.

Regarding OPCache Preloading in general, I'm sure it just takes time before the whole ecosystem matures to support it. The issues I covered are likely caused by some issues with incompatible dependencies, and probably some parts of eZ Platform code base as well. So over time we look forward to unlocking even more performance improvements with 7.4.

In case you did not know, PHP 7.4 is the end of the line for PHP 7 series. The next major version will be PHP 8.0, which will again introduce new language features, more deprecations as well as potential performance improvements like the JIT (Just In Time) compiler that has been in the works for quite a few years now. Good times.

Oh and if If you've got an idea on why the app causes PHP to segfault, any ideas or suggestions are welcome in the comments. I can poke at it for you, or you can do it yourself by cloning the repository. Happy hacking and remember this is no official release ;)

UPDATE: More success with a simpler app: PHP 7.4 OPCache Preloading benchmark