Chapter 8 For Galax Developers

8.1 Galax Source Code Architecture

The Galax source-code directories roughly correspond to each phase of the query processor. (Put link to Jerome's tutorial presentation here)

The processing phases are:

Document processing

  Document Parsing =>  
 [Schema Normalization (below) =>]
    Validation => 
      Loading => 
        Evaluation (below)

Schema processing

  Schema Parsing => 
    Schema Normalization =>
      Validation (above)
      Static Typing (below)

Query processing

  Query Parsing => 
    Normalization => 
   [Schema Normalization (above) =>]
      Static Typing (optional phase) => 
        Rewriting => 
          Compilation => 
         [Loading (above) =>]
            Evaluation =>
              Serialization

8.1.1 General

Makefile

Main Makefile

base/

Command-line argument parsing
Global variables (conf.mlp)
XQuery Errors
String pools
XML Whitespace handling

ast/

All ASTs: XQuery User & Core, XQuery Type User & Core
Pretty printers for all ASTs

config/

monitor/

CPU &/or memory monitoring of each processing phase

toplevel/

Main programs for command-line tools (see 'Generated executables' below)

website/

Local copy of Galax web site

8.1.2 Datamodel

datatypes/ (*** Doug)

XML Schema simple datatypes -- Lexers and basic operations
datatypes_lexer.mll To learn about O'Caml lex, read: http://caml.inria.fr/ocaml/htmlman/manual026.html Sections 12.1 and 12.2 Other examples of lexers in lexing/*.mll

We are going to extend this module to include lexer for: xsd:date, xsd:time, xsd:dateTime, xdt:yearMonthDuration, xdt:dayTimeDuration (Skip Gregorian types for now, xsd:gDay, xsd:gMonth, etc)
dateTime.ml,mli This module will implement the datatypes and basic operations

namespace/

XML Qualified Names (prefix:localname) -- Lexer and basic operations -- QName resolution prefix => URI
Names of builtin functions & operators

dm/ (*** Doug)

Abstract data model interface for Nodes
Concrete data model implementation for AtomicValues

datamodel/

Main-memory implementation of abstract data model for Nodes ( Document-Object Model or DOM)

jungledm/

Secondary storage implementation of Galax datamodel (Jungle)

physicaldm/

Physical data model

streaming/

XML parser to untyped and typed SAX streams
export datamodel to SAX stream

8.1.3 Processing Model

procctxt/

Processing context contains all query-processor state:
- Parse context
- Normalization context
- Static context
- Rewrite context
- Dynamic context

procmod/

Processing model dynamically "glues" together phases (controlled by command-line arguments or API)

8.1.4 Query Parsing

lexing/

Lexers for XQuery (excludes all simple datatypes)

parsing/

Parsing context
Parsing phase

8.1.5 Normalization

normalization/

Normalization context
Normalization phase (XQuery AST => XQuery Core AST)
Overloaded functions

8.1.6 Static Typing

fsa/

Finite-state Automata for checking sub-typing relation

typing/

Static-typing context
Static-typing phase

8.1.7 Schema/Validation

schema/

Schema-validation context
Schema normalization phase (XML Schema => XQuery Core Types)
Document validation phase
Judgments(functions) for comparing XQuery types

8.1.8 Rewriting

cleaning/

Logical optimization/rewriting phase
Sort-by-document order (DDO) optimization

rewriting/

Generic AST rewriter

8.1.9 Compilation

compile/

Compilation phase

algebra/

AST for compiled algebra
Dynamic context
Implementations (dynamic) of most built-in functions & operators

8.1.10 Evaluation

evaluation/

Evaluation phase

stdlib/

Static typing of built-in functions & operators
Implementations (dynamic) of built-in functions fn:doc, fn:error
Signatures of built-in functions & operators (pervasive.xqp) Corresponds to sections in http://www.w3c.org/TR/xpath-functions/

8.1.11 Serialization

serialization/

Serialize SAX stream to XML document (in O'Caml formatter)

8.1.12 Testing

usecases/

Tests of XQuery Usecases Implements examples in http://www.w3.org/TR/xquery-use-cases/

examples/

Tests of O'Caml, C & Java APIs

regress/

Regression tests (needs separate xqueryunit/ CVS package)

8.1.13 APIs

galapi/

O'Caml, C & Java APIs to Galax processor

8.1.14 External libraries & tools

tools/

Required tools:

http
pcre
pxp-engine
netstring

Optional supported tools:

Jungle

Optional unsupported tools:

glx_curl
jabber

8.1.15 Extensions

extensions/

apache
jabber

8.1.16 Experimental Galax extensions

projection/

Document projection

wsdl/

wsdl_usecases/

Web-service interfaces

8.1.17 Documentation

Changes Change log!! Protocol: always document your changes in Changes file; use log entry as input message to 'cvs commit'
BUGS Out of date
LICENSE
README
STATUS
TODO

8.1.18 Generated executables

ocaml-galax: O'Caml top-level interpretor that loads Galax library. Usage: ocaml-galax -I $(HOME)/Galax/lib/caml-devel
galax-run: Complete XQuery engine For Usage: galax-run --help See also: all: rule in usecases/Makefile
galax.a: Library versions of Galax
galax.cma: byte code
galax.cmxa: machine code
galax-parse: Syntax checking on query, validation on a document
galax-compile: Parsing, normalization, optimization, and prints resulting expression
galax-mapschema: Takes XML Schema and prints out internal XQuery type

Auxiliary research tools:

galax-mapwsdl: Imports/exports Galax queries as WSDL Web Services
xquery2soap
galax-project: Takes XQuery query and figures out what fragments of documents are necessary to evaluate the query