LShift Ltd.

EC2 Latency

September 24th, 2006 matthias

We want to use Amazon’s Elastic Compute Cloud to run scalability tests, rather than buying/leasing lots of kit that we only need for a few hours every now and then. There are a few challenges though. The biggest is probably translating the results from the EC2 tests to the envisaged deployment scenarios. There are several factors to consider here: hardware, operating system, network, impact of virtualisation.

Amazon provide details of the virtual hardware:

Each instance predictably provides the equivalent of a system with a 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of network bandwidth.

We also know that the default operating system is Fedora Linux.

One figure that is notable for its absence is network latency, in particular between several EC2 instances. Latency has a major impact on scalability, so is crucial that we know what it is.

A ping between two of my EC2 instances takes approximately 250 microseconds. That is about three times higher than I get between machines on our local network, but it is roughly the same when the machines are separated by a firewalls.

I have tried the ping test a few times, between different instances created at different times. The latency was always the same. However, given the EC2 setup, I suspect that latency may vary quite considerably, depending on where the instances happen to be running - the same location, the same rack or even the same machine. How much variation is there in instance placement?

For example, say I deploy some app on four instances one day, then tear it down and deploy it again some other day. If the performance of my app is inter-instance latency-bound, could I end up with performance differences of several orders of magnitude, i.e. if I was lucky to get all four instances deployed on a single physical machine in one case, and unlucky to get four machines on different continents in the other?

Hopefully Amazon will release some more information on their EC2 setup that may answer some of the above questions. Users of the service can also help by running some tests and publishing the results.

Entry Filed under: Technology