From: Ian King (iking_at_killthewabbit.org)
Date: Wed Mar 03 2004 - 17:42:36 PST
The authors describe a widely-distributed file storage system that distributes
data at the block level, rather than the file level as in PAST. Goals include
robustness in a network where server nodes are not centrally administered and
may join and leave the network often, together with a certain amount of security
against malicious insertion of data. The system is structured to be read-only
to clients, and those who insert data into it are "publishers".
As in the PAST paper, this is really about placement and routing algorithms.
The paper describes a method of storing file *blocks*, not files; the
integration of blocks into files is a matter for the client above the layer
described. Discovery of blocks that have been placed in the system is based on
the reproducibility of the algorithm that placed them there in the first place;
this algorithm is cleverly designed to (theoretically) avoid clustering of
blocks or their replicas among geographically or administratively related
servers. Using a content hash as the content's ID is quite clever, as it
creates a stronger correlation between the label and its target than other
approaches, and provides for a certain amount of implicit verification of
content.
Load balancing is significantly easier because the mechanism works in uniformly
sized blocks, rather than non-uniform files. In order to address the
differences in storage capacity among servers, the mechanism also "blocks"
server capacity through "virtual servers" that partition a physical server's
capacity. Caching is similar to PAST, and should therefore be as effective.
Because the Chord routing/mapping algorithm creates location diversity in its
mapping mechanism, the replication algorithm (and corresponding replica location
algorithm) can be naively simple and, therefore, quick. The "circular list"
structure of a Chord table also facilitates nodes joining and leaving, through
simplifying how other nodes assume the work of a departing node and how new
nodes are given a share of the load.
There were a couple of points where the realities of writing software came
through; the server selection algorithm used in testing was acknowledged as
"flawed", and a server could not retrieve from its own cache!
While the Chord algorithm is quite clever, I just can't get excited about this
paper, or about PAST; these seem to me to be solutions in search of problems.
These papers are about routing/mapping algorithms more than about filesystems
and, while obviously smart people did very smart things to make these work, I
can't help but wonder why. For instance, I sincerely doubt that a financial
institution would use this mechanism to ensure it would not lose its data.
This archive was generated by hypermail 2.1.6 : Wed Mar 03 2004 - 18:04:29 PST