CSE 599c Hot Topics in Data Management Systems

Lecture 15: Federated Systems.

Autonomy and Heterogeneity

Magdalena Balazinska

Winter 2006

Today's Paper

U. Srivastava, J. Widom, K. Munagala, and R. Motwani. Query Optimization over Web Services. Technical Report, Stanford University, Oct 2005.

Content of today's lecture also based on

R. Hull and J. Su. Tools for Composite Web Services: A Short Overview. SIGMOD Record. 34(2). June 2005.

Outline

Web services introduction

Definition, motivation, and overview

Challenge 1: query optimization

Optimization goal

Optimization approach

Challenge 2: data integration

Challenge 3: fault-tolerance (next lecture)

Web Services Introduction

Web service is a programmatic interface

Enables app-to-app communication

Main goals

Build large-scale E2E services over the web

(Semi)-automatic composition!

Requires discovery, analysis, data integration

Also requires coordination and monitoring

Web Service Stack

From [Hull05]

Web Service Stack

SOAP: Protocol for exchanging XML-based messages. Enables RPC functionality

WSDL: Language for defining Web service interfaces

BPEL4WS: Language for manually specifying composite Web services (workflow)

UDDI: Registry service (directory) to register and query Web services

Web Services Properties

Complete autonomy

Pre-defined data locations

Requires bottom-up integration

Loose coupling

Each site exports a specific interface

Heterogeneity

Independent schema definitions

Different data semantics

Data integration is a challenge

Web Service Management System (WSMS)

From

[Srivastava05]

Query over Web Services

Web services

WS1: name(n)  credit rating(cr)

WS2: name(n)  credit card numbers (ccn)

WS3: card number(ccn)  payment hist. (ph)

Precedence constraint: WS2  WS3

Query

Query Optimization

Optimization goal: maximize throughput

Cost metric

bottleneck cost: equation 1

minimize the maximum time that a Web service spends per original input tuple

Only data shipping

Without precedence constraint

Could send requests in parallel and join locally

But: selective vs proliferative services

No Precedence Constraints

Optimal order when selectivity  1.0

Chain of services with increasing cost

Intuition: most costly processes fewest tuples

Optimal order when selectivity  1.0

Send all requests in parallel

Intuition: all services process fewest tuples

Approach does not minimize resources used

Approach exploits pipelined parallelism

With Precedence Constraints

Basic idea: recursive algorithm

Step 1) Obtain optimal plan for each child

Step 2) Merge children plans

Step 3) Add self to front of plan

At each step, plan is in canonical form

Figure 5

Also, need to use optimal request batch size

Challenge 2: Data Integration

Need to map schemas of different services

Figure out that “salary” is the same as “pay”

But, salary hourly or weekly + lunch allowance!

More than syntactic pb, must map semantics!

Want to provide integrated view to user

Similarly, to describe and search for services

need taxonomies or ontologies

need standards

Conclusion

Web services

Potential for building complex services

Characterized by autonomy and heterogeneity

Query optimization

Maximize throughput not minimize resources

Focus on parallelism or pipelined parallelism

Data integration is a hard problem

Next question: what if different pieces break?

2006-01-01T23:16:26