Distributed Shared Memory over ATM...A REALITY as of 11am today. [if you want

[David Farber <farber () central cis upenn edu>]

Date: Wed, 28 Sep 1994 14:07:28 -0400
From: shaffer () aurora cis upenn edu (John Shaffer)
Posted-Date: Wed, 28 Sep 1994 14:07:28 -0400
To: f-troup () aurora cis upenn edu
Subject: Distributed Shared Memory over ATM...A REALITY as of 11am today.




For any who are interested....MNFS is running over ATM this morning.


I will send two postscript graphs following this message:


Both are of just plain old page-fault times....2000 page-faults
taken from a uniform distributed page range of [0...511].
The 2000 samples for each run were combined by averaging together
each group of 20 to get 100 averages...which are plotted.
Each shows both ATM and ethernet times to satisfy the page fault.  The
page fault time was measured using the TRACE macros available under AIX.
It measures the time the page fault occurred to the time the page is
installed and "pf_end()" is called.


The user program simply touches a randomly selected page in [0..511]
range and then invalidates it.  Just READs....no writes yet.  Writes
WORK mind you, they are just not shown in these graphs.


Two machines, an MNFS (memory) server and a client.  Client does
the page faults, the server satisfies them.  Both machines are
RS/6000s.  Client is a 64MB model 580 and server is 64MB model 360.
ATM is using Brendan Traws Host Interface, driver is Jonathan Smiths
TCP/IP stuff and the connection is via GLINK boards that Drew Moore
worked on.  Speed is 155Mb/s.


The first graph is of "cold" memory on the server and one can see the
gradual "warming" of the server memory as pages start "hanging around"
on the server.  In the beginning, pages must be grabbed from disk.


The second is after the server memory is heated up and is holding
pages not yet swapped out to disk.  The spike on this graph I have
not determined the actual cause.  I figure its either competition
from other ethernet traffic OR some weird "out-o-sync"ness.  Today,
I dont care.  Maybe tomorrow I will but today I just wanna feel
good for a while.


Propagation time for 4096 byte page over 10 Mb/s ==> ~ 3.3 ms
Propagation time for 4096 byte page over 155 Mb/s ==> ~ 0.2 ms


Difference is: ~3.1 ms...now look at the where the lines settle on
the "hot" graph (2nd one)  voila, *about* 3.1 ms apart.  Whoa!
Experimentation supporting the theory. Whoa. dude.


john