From: Justin Voskuhl (jvoskuhl_at_hotmail.com)
Date: Wed Feb 25 2004 - 18:06:13 PST
This paper describes the design of a system that's designed to handle
e-mail. It accepts messages via SMTP and allows users to retreive messages
using POP. The interesting thing about the system is how hands off it tries
to be in terms of administration. It avoids forcing the administrator to
tune the machines, which is something many admins spend their time doing
manually to get good utilization out of the systems they run.
The systems that take part in the cluster do not need to be uniform - the
performance and storage space can vary. The system is divided into Nodes,
with the idea that a Node can perform any task. You can deliver a message
to any node and The Right Thing happens - it decides whether to store the
message into a node that already has messages for you or it may find a less
loaded node to store the message on.
The system also handles fail-over. When a Node stops responding the other
nodes in the system notice, and take on the work for that node. Similarly,
new nodes are automatically recognized and made part of the system without
the administrator needing to do anything special (with the possible
exception of configuring external routing tables.) The great thing about
this system is that it can dynamically distribute load even though the
machines are heterogenous.
The best examples of our systems in production for high scale services can
do much of what this paper describes, but also handle a few additional
wrinkles. When raw hardware is racked up in our data centers, in some cases
our systems can image the machines automatically and determine what server
type they need to be and bring them into the service without manual
intervention. This "setup" problem (installing the OS, installing the
database(s), installing the software) often becomes massively time consuming
for the administrators as the system scales out.
_________________________________________________________________
Find and compare great deals on Broadband access at the MSN High-Speed
Marketplace. http://click.atdmt.com/AVE/go/onm00200360ave/direct/01/
This archive was generated by hypermail 2.1.6 : Wed Feb 25 2004 - 19:12:28 PST