PHP 8.0 and 7.4 benchmarks on Symfony: JIT boost up to +9% on top of OPCache Preload

PHP 8.0 and 7.4 benchmarks on Symfony: JIT boost up to +9% on top of OPCache Preload

The PHP 8.0 is scheduled to be released on November 26 2020, soon to be followed by the latest version of the Symfony framework 5.2. This is a quick round of benchmarks done on the latest release of both projects. It's a pure hello world scenario, and should be taken as such. Only relative performance counts. In the real world YMMV.

UPDATE 1: This article was originally missing the numbers with the PHP 8.0 JIT due to it not being enabled by default. Thanks to André Rømcke pointing it out!

UPDATE 2: PHP 8.0.0 and Symfony 5.2.0 have now been released. I ran the benchmarks again, but results very similar so the original values are valid.

The benchmarks were done on PHP 8.0-RC5 and Symfony 5.2-RC2. Both are not final versions, but late release candidates that could be very close to the final releases. The execution environment was Hetzner Helsinki datacenter with a Virtual Server CCX11 (2 dedicated vCPU, 8GB RAM) doing the serving, and another to run the benchmarks. The OS was Ubuntu 20.20 with PHP installed from the packages from Ondřej Surý.

The application was a bare-bones Symfony 5.2 application with the lucky number controller action at the root. The action spits out a snippet of HTML with a random number between 100 and 999. I ran the load test with five concurrencies (1, 10, 25, 50, 100) three times for each configuration and used the average number for the report.

A total of five configurations were tested using a PHP-FPM and Nginx setup:

  • PHP 7.4 without OPCache Preloading
  • PHP 7.4 with OPCache Preloading
  • PHP 8.0 without OPCache Preloading
  • PHP 8.0 with OPCache Preloading
  • PHP 8.0 with OPCache Preloading and JIT enabled

The OPCache config the same used for the post benchmarking OPCache Preloading:

opcache.preload_user=www-data
opcache.memory_consumption=1024
opcache.interned_strings_buffer=256
opcache.max_accelerated_files=30000
opcache.validate_timestamps=0

# only when OPCache Preloading was enabled
opcache.preload=/var/www/php80/var/cache/prod/App_KernelProdContainer.preload.php

# only when JIT was enabled
opcache.jit_buffer_size=512mb
opcache.jit=1225

Results and conclusion

The load balancing was ran using hey and I report the numbers for throughput (req/s) and response time (ms). The app does not connect to remote services like databases.

Throughput of Symfony 5.2 benchmarks on PHP 7.4 and PHP 8.0 (with and without OPCache and JIT)

With throughput both PHP 7.4 and 8.0 benefit significantly from enabling OPCache Preloading. On average PHP 7.4 is slightly more performant, but not by a significant margin. For throughput there seems to be no change in concurrency 10 to 100, but PHP 8.0 with JIT is curiously less performant as low concurrency but comes alive and edges past 7.4 with OPCache Preloading from at 25+ concurrency onwards.

Response times of Symfony 5.2 benchmarks on PHP 7.4 and PHP 8.0 (with and without OPCache and JIT)

For response times the story is similar as for throughput. Enabling OPCache Preloading yielding better results than an upgrade from PHP 7.4 to 8.0. For response times (smaller is better) PHP 8.0 is slightly better than 7.4, but not by a meaningful margin. Response time is higher at low currency, but stays consistent from 10 to 100. 8.0 with JIT is similar story: It's worst at low concurrency, but best at high load.

All in all the improvements in baseline performance for an application like Symfony is very similar between PHP 8.0 and 7.4. This is not surprising as it has been known that the JIT in PHP 8.0, that was already around in an early form for PHP 7.0, is more beneficial for calculations and other data processing tasks which are CPU bound.

Regarding performance JIT seems to be a mixed bag here, slowing down at low concurrency and providing a significant improvement to throughput over OPCache Preloading only. More investigation needed, but here the case is not clear as it could hinder performance. JIT seems like something to be considered case by case for PHP.

So for now it is safe to say you're not in a hurry to upgrade from 7.4 to 8.0 for performance reasons. You'll be better off by upgrading from PHP 7.3 to 7.4 and enabling OPCache Preloading. That's not to say that PHP 8.0 is a disappointment by any means. It's packed with new language features and important deprecations. Most importantly is good to see that there have been no performance regressions (like there was with the move from 4.4 to PHP 5.0 way back in 2004). A great start for 8.x.

Comments and Feedback welcome on Discourse: PHP 8.0 and 7.4 benchmarks

Photo by Jonas Thijs on Unsplash

eZ Platform is now Ibexa DXP

Ibexa DXP was announced in October 2020. It replaces the eZ Platform brand name, but behind the scenes it is an evolution of the technology. Read the Ibexa DXP v3.2 announcement blog post to learn all about our new product family: Ibexa Content, Ibexa Experience and Ibexa Commerce

Introducing Ibexa DXP 3.2
Product Launch: Introducing Ibexa DXP 3.2

Insights and News

DEVELOPER INSIGHTS
DEVELOPER INSIGHTS
By Jani Tarvainen
22/12/2020 | 8 Min read
DEVELOPER INSIGHTS
By Jani Tarvainen
18/12/2020 | 1 Min read