How Your Web Server Works

Julian Doherty


What happens when a request arrives?

Pixies and/or gnomes not acceptable explanation

Need to understand how requests are queued, multiplexed, managed

Lets take a look at a few

Lots more, but I’ve worked with these, and won’t sound as clueless talking about them.

(Yes, I know some of those technically aren’t web servers, but humour me…)


Like a city train station with multiple platforms

Each request is a train

Can handle as many trains at once as you have platforms

Conductor controls which trains go onto which platforms

What happens if you have too many trains at once?

If all platforms are full, trains have to queue up and wait

Service slows down

If trains take too long to finish on platform, services get cancelled

Sometimes on platform

Sometimes the trains waiting outside

BUT - A few slow trains on some platforms don’t slow other platforms down though

But, if the queue of trains gets too long

You may as well just give up and go home. You’re not getting in



Web servers are boring unless they’re doing something

Apache doesn’t do much on it’s own

But has lots of open source community activity

Many modules to plug in to do almost anything

Being on the beaten track has it’s advantages

Apache / Passenger / Rails

Passenger is a module that lets you run Rails apps on Apache



Conceptually the same as Apache

Separate platforms, trains come in

Can still hang with too many long requests


But under the hood it’s all happening in the same place. Not separated at all

Ok most of the time


Can cause problems if you’re not careful to keep things separated in memory


Shared state concurrency is easy to screw up though



private static final DateFormat dateFormatter = new SimpleDateFormat();

SimpleDateFormat isn’t threadsafe. Instance shared between all threads accessing code

Just a matter of time till…

Shared state concurrency bugs will bite you in the ass

Well, maybe not as dramaticly. You wish

Will probably just manifest as cryptic to debug NullPointerException, or as corrupted data you don’t notice till months later…

Tomcat / JRuby / Rails

Tomcat / JRuby / Rails

Another concurrency model: Supermarket with lots of checkouts

Simple conceptually

Each checkout has own queue

Works well (mostly…)

Mongrel /Rails

Supermarket checkout problem

Some guy paying for groceries with small change slowly

Everyone else at that checkout has to wait

If you choose the wrong queue, you’re stuffed

Mongrel / Rails

Suffers from “guy paying with small change”

Means more monitoring/restarting required when slow requests block it up

Restart kills all requests in queue for mongrel. Users see errors as a result

Forget about trains and pedestrians for a minute

Think about a waiter at a restaurant


Nginx hides single thread event driven model in black box

Don’t have to care about it day to day. Just use it and it works


Node puts it in your face


In theory, single waiter could serve millions of customers

Would be slow, but requires no more resource usage

Can’t do this with Apache/Tomcat style multi-process. Limit to how many “train platforms” you can build

But what if you COULD build millions of train platforms?

And do it cheaply and quickly?

And pack them into your available space really efficiently?


Lets you build millions of “train platforms”

Actually, you build a new platform for a single train

Then tear it down

All really cheaply



Another Erlang web server, similar to Mochiweb

Apache dies at ~4,000 concurrent reqs. Yaws still ticking up around ~80,000.

Total throughput does not decrease as load increases.

Work done to get 1,000,000 concurrent requests on a single box with Mochiweb



Julian Doherty