1. Cache the output at the edges: Use Varnish or other reverse proxy cache.
2. Cache byte code: Use APC or XCache PHP opcode cache.
3. Cache and minimize database I/O: reduce database touches using memcached, redis, file caches, and application-level caches (ie. global vars)
4. Do event logging in local files, not to the database: Make all write operations as simple and fast as possible, any data that is not needed in realtime can be written to a plain old file and processed later.
5. Use a CDN, especially for delivering static assets
6. Server tuning: Apache, MySQL, and Linux have lots of settings that affect performance, especially the timeout settings ought to be turned down.
7. Identify bottlenecks: At the system level use tools such strace, top, iostat, vmstat, and query logging to see which layer is using the most time and resources. Also there's an excellent PHP code profiling service called New Relic that drops you right into the function and db query that's eating up most of the time in slow requests.
8. Load testing: DoS yourself. Stress test your stack to find bottlenecks and tune them out
9. Remove unused modules: For each component in the stack unload any default modules that are not needed to deliver your service.
10. Don't use ORMs and other dummy abstractions: Take off the training wheels and write your own queries.
11. Make the entry pages fast, simple, and cacheable. Nobody is reading that silly news feed in bottom corner of your front page and it's killing your database, so take it out.
Most of the time a PHP slows down because each PHP process is blocked waiting for I/O from some other layer, either a slow disk, or overloaded database, or hung memcached process, or slow REST API call to a 3rd party service ... often just strace'ing a live PHP process will show you what its waiting for ... in short, blocking I/O slows down everything. The key to going faster is:
* keep it simple
* cache as much as possible in local memory
* do as few blocking I/O operations as possible per request
"Identify bottlenecks" - I hope this is the first topic covered. There's no point in fiddling about with PHP if the actual bottleneck is the fact that you're sharing a slow connection with dozens of other companies or are hosted on the opposite side of the world to most of your customers.
Having said that, I'm looking forward to the book, as there doesn't seem to have been much on this topic since the O'Reilly books on scaling web applications (2006 I think).
Not sure about 'best', but the way I go about it is to measure the total time required for an application to complete a task, then attempt to measure the individual components (time spent in PHP, data transfer, rending on client side etc.) and keep narrowing it down until you find the exact cause, such as a database query which requires optimisation.
I'm expecting the book to go much further than this. Your list is a fine starting point, but its fairly basic and lacks specifics.
I want to know about the weird things that happen when you push your PHP stack to really ridiculous limits. I think that's what this book will offer. If it ends up being stuff like "use APC! minimize trips to the DB!" I would be very surprised and disappointed.
I agree- good starting point, but the book will be much more in depth. It's going to include a fair amount of high-level design, identifying how and where to scale, weird issues you run into (fun fact: we don't use APC @ Twitpic), solutions to PHP anti-scaling quirks (you can't set sub-1s mySQL timeouts, for instance, although I think this is due to libmysql, not PHP), lots of case studies, scaling design patterns, and way more cool stuff.
Wouldn't you lose one of the advantages of PHP which is that every request is stateless and that when it fails it doesn't fail hard killing the application?
Not any more nowadays. I am the original author of Photon and today PHP is extremely robust and you can catch all the exceptions/errors (outside the ones which are making your code invalid, but hey, do your homework). The current PHP 5.3 is also wonderful at not leaking memory. So you get the speed of bare PHP with a framework (because nothing is reloaded, the framework can really load once, reuse many times).
On a small system, a standard HTTP request through Mongrel2, to the framework and back is about 2ms (including 1ms of network latency between the hosts, data from my production monitoring). This is the "hello world" latency. It makes using PHP very fun and very fast. Also, a single PHP process running in the background uses about 12MB, it means that with 12MBx3 + 10MB Mongrel2, you can serve 1000's of users with nearly no load on your system (if you push the load only when needed at the DB level). Tracing a PHP process, you can run it without system calls at all (only gettimeoftheday for logging).
But, this is cutting edge, so, you need to be open minded and ready to dig in the code (less than 25k SLOC anyway I think).
Current PHP is 5.4. Did you perform any test or benchmark with the new version? Do you gain anything from it? By the way, your project looks very interesting!
Very limited tests, one PECL extension for my projects was updated to support 5.4 only recently. The biggest win will most likely be for the POST payload (multipart) parsing because 5.4 is faster on array/strings. The good point is that with PHP farm, you can run both 5.3 and 5.4 in parallel to compare :)
If it's pushing PHP to ridiculous limits you seek then you want Facebook's HipHop compiler. But web stack design is really an art form, hard to summarize in an HN comment.
I'm curious why you suggest avoiding ORMs. I've been debating this for a while and while I like the accessibility and abstraction that ORMs provide in Django and Rails I'm not familiar enough with Doctrine, DB_DataObject or the other PHP options to really have formed my own opinions yet.
Is this a language specific suggestion or something broader? I would love to hear a deeper debate on the subject.
I think the advice to not use an ORM* is very misguided. Use the ORM for the 80-90% of queries that are "SELECT * FROM table WHERE id=5" and "UPDATE table SET name='jim' WHERE id=5". For the rest of the queries, use your ORM's ability to run custom queries. People who have problems with ORMs are probably not using them correctly.
There was a debate on HN about ORM's a while ago, and one of the most significant comments I read was something along of the lines of "Every project I've ever worked on that 'didn't use an ORM' implemented the same functionality of one in a less-maintainable way."
* Or other object-oriented abstraction layer for database access, ORMs aren't the only option.
I totally agree. ORMs are not a bad thing and save you from writing the same code over and over again. The key is to understand what it's doing behind the scenes and not be afraid to go in and change it if it's generating stupid queries.
Is there some extra overhead? Of course. Probably a lot less than cleaning up the damage from some cowboy coding.
> * Or other object-oriented abstraction layer for database access, ORMs aren't the only option.
What do you mean by this? That you should consider a non-relational database? Clearly any mapping from an OOP language to a relational database is an object-relational mapping.
The key criticism in that article seems to be "ORMs are imperfect abstractions for SQL." Who the hell cares? This is the real world. We don't need perfect abstractions.
It could be as simple as the performance loss incurred by an ORM is greater than the benefit of using it. You have PDO with PHP, which can be used quite adequately as an ORM itself (you can fetch a tailored SQL query into a class as opposed to working with basic arrays).
I'd argue that, in some cases, an ORM is only used as a reaction to the separation of concerns. Writing SQL mixes up your languages, so why not try to build that SQL in the language you're working with? Or maybe SQL is considered too difficult.
I quite like having full control over a 'raw' query without requiring an ORM's opinion on how it should be built.
PDO doesn't handle mapping of nested relationships, though which is where it falls down when compared to ActiveRecord implementations. For example I want to fetch an author and all of the author's books in a single query then loop over and print out author name, and each book. A standard ActiveRecord ORM will give you an array of authors where each author contains an array of books, where as with PDO you're still left to manually detect a change in author by a change in the primary key which is more complex than just writing a nested for loop.
Forgive me if I misunderstand, but if you can't pull this off in a single SQL query, then ActiveRecord can't either. It just hides the fact it's performing several queries to get what you want.
Using PHP for a 30+M registered users website, I'm using a modified Code Igniter for the base framework, and specific Zend Framework's modules on some functionality where they provide good libraries.
As the parent said, ultimately cpu cycle on your web servers don't matter much, you will be limited by IO everywhere. Knowing how to properly use redis' data types and handle multi-level cache invalidation will be a much more valuable asset than using a framework you might not enjoy the most / be the most fluent with just because it's a wee bit faster.
Even in terms of actual php's calculation, your main strong point is figuring out what you can remove from the page processing and put in a daemon instead. "If the user wants to do action A, what is the minimal amount of things I can do to make him believe that it has been done", do that in your controllers, and put all the actual hard work in an event processing queue that doesn't have an user waiting for it to answer back.
1. Cache the output at the edges: Use Varnish or other reverse proxy cache.
2. Cache byte code: Use APC or XCache PHP opcode cache.
3. Cache and minimize database I/O: reduce database touches using memcached, redis, file caches, and application-level caches (ie. global vars)
4. Do event logging in local files, not to the database: Make all write operations as simple and fast as possible, any data that is not needed in realtime can be written to a plain old file and processed later.
5. Use a CDN, especially for delivering static assets
6. Server tuning: Apache, MySQL, and Linux have lots of settings that affect performance, especially the timeout settings ought to be turned down.
7. Identify bottlenecks: At the system level use tools such strace, top, iostat, vmstat, and query logging to see which layer is using the most time and resources. Also there's an excellent PHP code profiling service called New Relic that drops you right into the function and db query that's eating up most of the time in slow requests.
8. Load testing: DoS yourself. Stress test your stack to find bottlenecks and tune them out
9. Remove unused modules: For each component in the stack unload any default modules that are not needed to deliver your service.
10. Don't use ORMs and other dummy abstractions: Take off the training wheels and write your own queries.
11. Make the entry pages fast, simple, and cacheable. Nobody is reading that silly news feed in bottom corner of your front page and it's killing your database, so take it out.
Most of the time a PHP slows down because each PHP process is blocked waiting for I/O from some other layer, either a slow disk, or overloaded database, or hung memcached process, or slow REST API call to a 3rd party service ... often just strace'ing a live PHP process will show you what its waiting for ... in short, blocking I/O slows down everything. The key to going faster is:
* keep it simple
* cache as much as possible in local memory
* do as few blocking I/O operations as possible per request