CSE 599c Hot Topics in Data Management Systems

Lecture 17: Peer-to-Peer Systems.

Large-Scale Distribution.

Magdalena Balazinska

Winter 2006

Today's Paper

R. Huebsch, J. M. Hellerstein, N. Lanham, B. T. Loo, S. Shenker, I. Stoica. Querying the Internet with PIER. VLDB 2003.

Content of today's lecture also from:

I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan, Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications. SIGCOMM 2001.

Outline

Motivation and goals

Architecture

Performance

Goals

PIER: P2P Info. Exchange and Retrieval

Massively distributed query engine

Bring database query processing facilities to new, widely distributed environments

Example: overlay-based peer-to-peer system

peer

overlay

link

Motivation

Example scenarios

File sharing: simple selection queries

Network monitoring: fancy join queries

If IP is potential scanner, check if other sites also believe it is a scanner

Why distributed query processing?

Queries over live data (rather than warehouse)

Cheaper to administer than centralized solution

Shared control and responsibility

Revisiting Database Principles

Problems

DBMSs do not scale

Need to relax some fundamental principles

Revisited principles

P1: Relaxed consistency

P2: Organic scaling

P3: Natural habitats for data

P4: Std schemas via grassroots software

Architecture

Three-tier system

Application

PIER	engine

DHT

Figure 1

From [Huebsch2003]

Overlay Routing

Based on a distributed lookup services

(also called the content routing layer)

API: lookup(key)  node (or group of nodes)

Main properties of distributed lookup service

Natural load balance among all N nodes

Lookup requires O(log N) hops

When a node joins or leaves, only few items (1/N items) need to change location

Overlay Routing: Chord

Figure 2

From [Stoica2001]

Principle:

consistent hasing

Overlay Routing: CAN

Figure 2

From [Huebsch2003]

PIER Architecture: DHT Layer

Figure 1

From [Huebsch2003]

Based on Chord,

CAN, or other

protocol

Local storage

Heart of PIER's DHT layer

Storage Manager

Data lives outside PIER, in its “natural habitat”

Data is temporarily copied into the query system storage space

Enables data to be indexed (assigned to a node)

Enables efficient queries over the indexed data

Provider

Naming scheme

Each relation corresponds to a namespace

Subset of tuple attributes  resourceID

Random number  unique instanceID

API

Table 3

From [Huebsch2003]

Key used for

routing layer

PIER Layer

Figure 1

From [Huebsch2003]

Push-based dataflow engine

Equi-join algorithms

Variant of pipelined symmetric hash join

Fetch matches

Symmetric semi-join

Bloom join

Join Performance

Table 4 from

[Huebsch2003]

Figure 4 from

[Huebsch2003]

Other Performance Results

Scalability results

Latency of first result increases slowly with size

Bw utilization increases with workload but gets spread among different groups of nodes

Query-site inbound capacity is quickly bottleneck

What happens when nodes join and leave?

Figure 6

Trade-off between recall and refresh rate

Replication would increase recall but also overhead

Conclusion

PIER: Internet-scale query processor

Consistency

Based on soft-state

Dilated reachable snapshot semantics

Focuses on query execution techniques

Relies on parallel processing for performance

Future work

Query optimization, how about updates?

Catalogs, range predicates, agg, streams

2006-01-01T23:16:26