We want to use Amazon’s [Elastic Compute
Cloud](http://aws.amazon.com/ec2) to run scalability tests, rather
than buying/leasing lots of kit that we only need for a few hours
every now and then. There are a few challenges though. The biggest is probably translating the results from the EC2 tests
to the envisaged deployment scenarios. There are several factors to
consider here: hardware, operating system, network,
impact of virtualisation.
Amazon provide details of the *virtual* hardware:
> Each instance predictably provides the equivalent of a system with a
> 1.7Ghz Xeon CPU, 1.75GB of RAM, 160GB of local disk, and 250Mb/s of
> network bandwidth.
We also know that the default operating system is Fedora Linux.
One figure that is notable for its absence is network latency, in
particular between several EC2 instances. Latency has a major impact
on scalability, so is crucial that we know what it is.
A ping between two of my EC2 instances takes approximately 250
microseconds. That is about three times higher than I get between
machines on our local network, but it is roughly the same when the machines are separated by a firewalls.
I have tried the ping test a few times, between different instances
created at different times. The latency was always the same. However,
given the EC2 setup, I suspect that latency may vary quite
considerably, depending on where the instances happen to be running –
the same location, the same rack or even the same machine. How much
variation is there in instance placement?
For example, say I deploy some app on four instances one day, then
tear it down and deploy it again some other day. If the performance of
my app is inter-instance latency-bound, could I end up with
performance differences of several orders of magnitude, i.e. if I was
lucky to get all four instances deployed on a single physical machine
in one case, and unlucky to get four machines on different continents
in the other?
Hopefully Amazon will release some more information on their EC2 setup that may answer some of the above questions. Users of the service can also help by running some tests and publishing the results.