_____________ SECTION8 Irene Zhang _____________ Table of Contents _________________ 1 Administrivia 2 Lab Architecture 3 Shardmaster .. 3.1 Interface .. 3.2 Replication .. 3.3 Balance 1 Administrivia =============== - Jeff Dean in EEB 105 at 3:30 = architect of Spanner 2 Lab Architecture ================== - Shards that each hold a subset of the keys - replica groups of storage servers - Each replica group gets assigned a few shards - Centralized shardmaster that holds the assignment between shards and groups (Part A) - Replica groups serves keys for their assigned shards (Part B) 3 Shardmaster ============= - system moves through sequence of configurations of mapping between shards and groups. first config is 0, all shards = gid 0 - shardmaster dictates what configuration the system is in 3.1 Interface ~~~~~~~~~~~~~ - JOIN(gid) - LEAVE(gid) - MOVE(shard id, gid) - QUERY(number) if -1 or bigger than known config, return biggest 3.2 Replication ~~~~~~~~~~~~~~~ - use your kv store from last lab - one instance per config or one instance per change will work - so either replace your put/get calls with shardmaster calls - or use instances to agree to next config (but remember if you lose, you might have to change your next config membership) 3.3 Balance ~~~~~~~~~~~ - Want to ensure that group sizes are balanced. tests will check! - And ensure you move as few shards as possible with each change - I just took one shard from the largest group each time, but there are other ways to do it - small number of shards (~10), so no need to do anything fancy like consistent hashing