technology from back to front

How fast can Erlang create processes?

Very fast indeed.

1> spawntest:serial_spawn(1).
3.58599e+5

That’s telling me that Erlang can create and tear down processes at a
rate of roughly 350,000 Hz. The numbers change slightly – things slow
down – if I’m running the test in parallel:

2> spawntest:serial_spawn(10).
3.48489e+5
3> spawntest:serial_spawn(10).
3.40288e+5

4> spawntest:serial_spawn(100).
3.35983e+5
5> spawntest:serial_spawn(100).
3.36743e+5

[Update: I forgot to mention earlier that the system seems to spend 50% CPU in user and 50% in system time. Very odd! I wonder what the Erlang runtime is doing to spend so much system time?]

Here’s the code for what I’m doing:

-module(spawntest).
-export([serial_spawn/1]).

serial_spawn(M) ->
N = 1000000,
NpM = N div M,
Start = erlang:now(),
dotimes(M, fun () -> serial_spawn(self(), NpM) end),
dotimes(M, fun () -> receive X -> X end end),
Stop = erlang:now(),
(NpM * M) / time_diff(Start, Stop).

serial_spawn(Who, 0) -> Who ! done;
serial_spawn(Who, Count) ->
spawn(fun () ->
serial_spawn(Who, Count – 1)
end).

dotimes(0, _) -> done;
dotimes(N, F) ->
F(),
dotimes(N – 1, F).

time_diff({A1,A2,A3}, {B1,B2,B3}) ->
(B1 – A1) * 1000000 + (B2 – A2) + (B3 – A3) / 1000000.0 .

This is all on an Intel Pentium 4 running at 2.8GHz, with 1MB cache, on Debian linux, with erlang\_11.b.0-3\_all.deb.

by
tonyg
on
10/09/06
  1. My guess is that the kernel time is syscalls to malloc and free for the allocation and deallocation of stack and heap for each process.

  2. malloc and free aren’t syscalls – malloc calls sbrk or similar if it needs more memory, but otherwise it manages the heap in userspace. It should have a certain amount of hysteresis before it tries to give any memory back to kernel space, so all the malloc and free stuff should be happening entirely in userspace.

    Of course being a garbage-collected language it won’t be calling malloc and free – it’ll be directly calling sbrk or similar (mmap in fact AIUI) and managing the memory returned itself.

  3. SM Smithfield
    on 13/09/06 at 1:22 am

    Just ran your test on a Mac (10.4 G5 2.1Ghz 512kB cache)
    7> c(spawntest).
    {ok,spawntest}
    8> spawntest:serialspawn(1).
    6.48313e+5
    9> spawntest:serial
    spawn(10).
    6.48131e+5
    10> spawntest:serialspawn(10).
    6.36188e+5
    11> spawntest:serial
    spawn(100).
    6.33489e+5
    12> spawntest:serial_spawn(100).
    7.06393e+5

    I think these fellows might be onto something.

  4. ratatask
    on 13/09/06 at 7:00 am

    strace the erlang process if you want to see what it’s doing using
    system time.
    I wouldn’t be surprised if it was time reading, or a select/poll loop :-|

  5. @ratatask: yes, that’d probably give some idea. I don’t just yet fancy the job of trawling through the megabytes of strace output looking for patterns, though :-)

  6. It does make a lot of poll() call, but also much much more times() and gettimeofday() calls (700 times more than poll()).

  7. “I don’t just yet fancy the job of trawling through the megabytes of strace output looking for patterns, ”

    You could maybe use the summary flag on strace to “strace -c” to give you a breakdown.

 
 


5 × five =

2000-14 LShift Ltd, 1st Floor, Hoxton Point, 6 Rufus Street, London, N1 6PE, UK+44 (0)20 7729 7060   Contact us