Is Your Site Hacker News Ready?

« »

Years and years ago, somebody coined the term “slashdotting.” In essence, it’s the experience when your website is linked to by a larger website, overwhelming your servers with a crush of traffic.

Hacker News can have this effect, especially if your website reaches the front page for any length of time.

Two weeks ago, I was featured on the Hacker News front page for nearly six hours. In that time I received thousand of hits to my website. The initial crush took down my web server. This is my story, and how you can avoid a similar fate.

My setup

I use ServerGrove with a 2.5 GB virtual server to host a WordPress blog. The webserver I have installed is Apache. I know that WordPress is a terribly inefficient application; to counter the traffic I receive normally from my readers I have W3 Total Cache standing in front.

This caching application is configured to create HTML caches of the pages. It also uses Amazon S3 to host assets like CSS and images. W3 Total Cache also has a built-in object cache, which I use and store in memory.

The problem

When my post was voted up on Hacker News, it hit the front page. This sent a crushing flood of traffic to my website; I saw traffic in the 200 hits per minute range.

I quickly figured out that first and foremost the cache was misconfigured and object caching was disabled. I enabled this. With object caching disabled, WordPress had to construct many of the objects it needed when WordPress loaded.

I also detected that Apache was experiencing a high number of defunct processes. I checked my Apache configuration and I discovered that my custom configuration had been replaced during a system upgrade with a default configuration. I had 1 server with a maximum of 20 clients and 300 requests per child.

I increased this two two servers, sixty clients and 3,000 requests per client. Immediately I noticed the number of defunct processes drop, the RAM utilization rise, and the site stabilize.

For the next six hours I watched hundreds of hits per minute come in, and the site performed extremely well.

Lessons learned

I learned a number of important lessons through this process.

Brandon Savage is the author of Mastering Object Oriented PHP and Practical Design Patterns in PHP

Posted on 2/7/2013 at 7:00 am
Categories: System Architecture, Technology

psychologia... wrote at 2/7/2013 8:37 am:

Mine is :)
w3tc + big cdn… but I’m curious to what it can hold agains.

Phil (@#twitter) wrote at 2/7/2013 9:48 am:

Nice article, lol. I am interested in more details on this section “Upgrades should be double checked for configuration changes.”

Hikari wrote at 2/14/2013 4:19 pm:

Indeed, WordPress is simple and somewhat powerful, but very inefficient, and most plugins have bad quality. If we don’t be careful, because of plugins, each hit can result in 3 or 4 pageloads, and each pageload requires a full WP startup!

I love it and plugins is what keeps it alive, but we must be careful.

« »

Copyright © 2023 by Brandon Savage. All rights reserved.