CSE 599c Hot Topics in Data Management Systems

Lecture 5: Stream Data Models and Operators

Magdalena Balazinska

Winter 2006

Today's Papers

D. Abadi, D. Carney, U. Çetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, and S. Zdonik. Aurora: A New Model and Architecture for Data Stream Management. In VLDB Journal (12)2, 2003.

A. Arasu, S. Babu, and J. Widom. The CQL Continuous Query Language: Semantic Foundations and Query Execution. VLDB Journal (to appear).

Outline

The Aurora approach (SQuAl)

Stream and query models

Operators

Query examples

Comparing SQual and CQL

Stream

header

data

(timestamp, v1,...,vn)

Stream: append-only sequence of tuples

All tuples on stream have same schema

Timestamp is used for QoS

Tuple:

Query

Aurora node

Input

streams

Output

streams

Sum

Map

Union

Avg

Join

Max

Sum

Min

Stream

Operator

Quality of service graphs

Connection points

Aurora Operators

Order-agnostic

Filter

Map

Union

Order-sensitive

Aggregate

Join

BSort, Resample

Filter Example

Filter

S

temp > 105

temp > 100

otherwise

(A,107)

(B,101)

(C,98)

Input tuples

Output tuples

Input schema:

(location, temp)

Filter Example

Filter

S

temp > 105

temp > 100

otherwise

(A,107)

(B,101)

(C,98)

(A,107)

(B,101)

(C,98)

Input tuples

Output tuples

Input schema:

(location, temp)

Map Example

Map

S

new.location = old.location

new.temp_celcius = 5/9*(old.temp - 32)

(A,107)

(B,101)

(C,98)

Map Example

Map

S

(A,107)

(B,101)

(C,98)

(A,42)

(B,38)

(C,37)

new.location = old.location

new.temp_celcius = 5/9*(old.temp - 32)

Union Example

Union

S1

S2

S3

(A,107)

(A,108)

(B,101)

(C,98)

(C,95)

Union Example

Union

S1

S2

S3

(A,107)

(B,101)

(C,98)

(C,95)

(A,108)

(A,107)

(A,108)

(B,101)

(C,98)

(C,95)

Aggregate Example

Agg

Operator parameters:

group by location

average temp,

order on time, window size 1 h, advance 1h

(1:00,A,74)

(1:00,B,65)

(1:35,A,76)

(1:34,B,65)

(2:05,A,72)

(2:03,B,63)

Output tuples

Input schema: (time, location, temp)

S

Input tuples

Aggregate Example

Agg

Operator parameters:

group by location

average temp,

order on time, window size 1 h, advance 1h

(1:00,A,75)

(1:00,B,65)

(1:00,A,74)

(1:00,B,65)

(1:35,A,76)

(1:34,B,65)

(2:05,A,72)

(2:03,B,63)

Input schema: (time, location, temp)

S

Output tuples

Input tuples

Join Example

Join

S

R

Operator parameters:

S order on time, R order on time, window size 1h

predicate S.location = R.location,

(1:00,C,74)

(1:00,A,0.3)

(1:35,A,76)

(1:34,C,0.4)

(2:05,B,72)

(2:03,A,0.2)

Input tuples

Output tuples

Join Example

Join

S

R

(1:00,C,74)

(1:00,A,0.3)

(1:35,A,76)

(1:34,C,0.4)

(2:05,B,72)

(2:03,A,0.2)

1:00,A,0.3)

1:34,C,0.4)

(1:35,A,76,

(1:00,C,74,

2:03,A,0.2)

(1:35,A,76,

Input tuples

Output tuples

Operator parameters:

S order on time, R order on time, window size 1h

predicate S.location = R.location,

2006-01-01T23:16:26