From: Ian King (iking_at_killthewabbit.org)
Date: Wed Feb 25 2004 - 17:14:43 PST
The authors describe a set of programs and protocols to provide distributed
computing for the servicing of information requests via Internet protocols such
as HTTP. The system's goals include fault-tolerance, scalability and ease of
administration, and is premised on the use of 'commodity' computers. The paper
describes two implementations, one being a modem pool system structured upon the
authors' architecture, the other being the HotBot search engine, which predates
the authors' work but relies on many of the same principles.
The system described decomposes the functionality into three tiers, then further
decomposes the member services within those tiers. One or more of the
fundamental functional units may exist on a given computer, and certain types of
functional units are duplicated to achieve scalability and fault-tolerance.
Another goal for the system was to separate the implementation of services from
the supporting platform; arguably, this system provides that abstraction for a
class of services involving access to information.
A critical element of this design is the concept of "soft state", i.e. state
that can be lost and dynamically reconstructed. Equally important is the
commitment to "good, but maybe not perfect" results; consistency may be
compromised in the interests of availability. Taken together, these concepts
contribute to the feasibility of the system. The authors carefully delineate
areas where strict consistency is required - such as the personalization
database - from those where "good enough" information are sufficient to meet the
service requirements.
Ease of administration is addressed in several ways. An interesting compromise
in this regard is the centralization of load balancing; while empirically
derived, the authors claim this was not a bottleneck in the described
implementations, and contributed to simplicity. The ability of a node to
dynamically garner soft state, and cause no harm to the entire cluster while
inconsistent, contributes to ease of extension or replacement of failing nodes.
This distribution of work lives considerably above the level of "distributed
computing" as represented by a system such as Emerald or a distributed VM
implementation, and this seems both appropriate and sufficient for the goals of
the system.
This archive was generated by hypermail 2.1.6 : Wed Feb 25 2004 - 17:26:56 PST