Benchmark: Composer Autoloader
In computer programming, just like science more broadly, it is important to occasionally set aside some time to test the Obvious Truths. Sure, the results are usually just that — obvious. But sometimes, sometimes, a Grand Deception is uncovered. As Carl Sagan once said, “The universe is not required to be in perfect harmony with human ambition.”
With that in mind, I decided to take a look at the different autoloader options provided by Composer, a popular PHP dependency manager. At the time of this writing, there are three flavors:
# Default: generic autoloading composer dumpautoload # Optimized: PSR-0/4 classes are loaded from a classmap composer dumpautoload -o # Authoritative Classmap: autoloading happens entirely via classmap composer dumpautoload -a
Three options, (obviously) ordered least to most efficient. In fact, according to the documentation, optimized autoloaders are “recommended especially [when used on] production [environments]”. Which makes sense. An agnostic autoloader (the default) doesn’t really know or care about each and every last class or file; it just uses certain naming patterns to guess where something might be found and, when a class is referenced, checks the filesystem to see if it exists, then load it if it does. The classmaps, by comparison, already know where everything is with certainty, so if a referenced class is under its umbrella, it loads it, if not, it doesn’t.
The unoptimized default was the clear winner, with 5-11% better performance than the “optimized” builds.
To see if the issue might be due to a bug in PHP7’s Opcode caching, I re-ran the tests with it disabled.
Nope. In fact the relative performance of the “optimized” builds ended up even worse, now trailing the default by 10-15%.
So what gives?
It turns out that classmaps have a cost of their own: data. Information about every class, trait, and instance, whether they end up being used or not, is collected and stored in an array which is then copied one or more times before winding up inside the actual autoloader handler. If a site has a lot of dependencies, and if the ratio of used:unused for any given page is low, that winds up being unnecessary overhead.
Ultimately, whether or not classmap optimization will be helpful or hurtful to a project’s performance depends on whether or not the cost of agnosticism (namely filesystem lookups) is greater or lesser than the weight of the pre-mapped data.
Optimization wins if:
- There are few dependencies;
- The ratio of used:unused files is high;
- Filesystem lookups are slow;
Optimization loses if:
- There are a lot of dependencies;
- Few dependencies are actually used;
- Filesystem lookups are speedy;
In the case of the particular site featured in these tests, the classmap data amounted to 31KB, but no single page ever ended up referencing more than 5-10% of the collection. Furthering the divide, the site was running on a dedicated Linux server with a fast SSD, making lookup times negligible.
The obvious truth turned out not to be so obviously true.
A cross-section of seven URLs from a moderately heavy WordPress site were each loaded 40 times per configuration (i.e. 280 individual page loads per batch). The sum of the PHP execution times for each set of seven pages became the figures used for comparison. The first 20 runs for each batch were discarded, the remaining 20 used for the benchmarks.
To calculate the PHP execution times as accurately as possible, the main
index.php file was modified to execute
microtime(true) at the start and end of each request and record the difference.
A dedicated PHP-FPM socket was used to handle requests for the site, and external traffic locked out for the duration. The PHP process was restarted before each batch.