Semi-automatic wrapper generation - Ashish & Knoblock

• People generally didn't like the paper. The consensus was that the system doesn't do much - it's just a bunch of LEX and YACC rules
• However, dissenters claimed it gives a great demo
• Another complaint was that some would like to see it applied to another domain
• There should be a canonical test set for testing web agents
• Need to hardcode query
• made bad assumption that the unit of interest is the page

Grammatical inference - Freitag

• General agreement that this paper had good discussion, experiments and data
• Abstraction of text to more general features is a great idea
• Independence of features is important, separation is good
• Suggestions were made to improve the system by preprocessing with recognizers a la Nick's wrapper induction and to use hierarchical information to improve results

Learning Symbolic Knowledge - Craven et al

• Did a good job on a simplified version of the problem
• Lots of good ideas - good challenge paper
• Nothing more than a homepage finder?
• Need to distinguish web objects and real objects in ontology
• Also made page-unit assumption

The great wrapper debate (to wrap or not to wrap)

Dan's slides

Softbot Perception

Examples:

• an html page is labeled as "Tom Mitchell's home page"
• a set of digits is labeled as a "UW phone number"
• a line of text is labeled as price(Encarta) = $69.95.

Decompose "perception" into

Token translation:

Assigning semantic labels to individual tokens.

Identifying units:

Putting together tokens into higher level descriptions.

Information extraction:

Using partial description of units and tokens to draw a conclusion about the perceived information.

Theme: REGULAR FORMATING:

• Units often organized in a regular fashion, For example: separated by standard delimiters (e.g. line breaks, commas, or spaces).
• Font and size information
• Capitalization

Theme: SEMANTIC EXPECTATIONS:

Semantic expectations are useful when they help to make perceptual decisions;

E.g. we can conclude that the number we find, which begins with the area code 415, is unlikely to be the office phone number of a person who works at the University of Washington.

Theme: HIERARCHY

• Home pages contain addresses,
  • addresses contain names, and
    • names are decomposed into first and last name.

• Catalog descriptions contain list of items,
  • each item contain a price, and
    • prices are typically preceded by $.

Theme: CONTEXT:

For example,

• When an extension is given instead of the whole phone number, often there is a standard prefix associated with an institution
• One can infer the extension's area code by knowing the city where the information is located.

Theme: MULTIPLE VIEWS:

Examples

• analyzing horizontal and vertical spacing,
• searching for patterns in the HTML source for a page,
• trying to identify standard delimiters, and
• looking at the type description of tokens, such as numbers versus letters, etc.
• Bayesian + grammatical inference

Theme: MULTIPLE EXAMPLES

• ILA uses multiple exs to resolve semantic ambiguities,
• WIL uses multiple exs to identify different syntactic variations,
• ShopBot uses multiple examples to identify product description headers and item descriptions.

Theme: SEEDS

For example,

We may start with a simple description of what a phone number looks like and update it as we encounter syntactic variations.

Theme: PRE-PROCESSING OF ITEMS

• items may appear much more similar if described only in terms of whether they contain letters or digits;
• likewise an item that is described in terms of higher level tokens such asprice, product name, vendor, etc. may have some structure that is much easier to spot.

Theme: PARTIAL SUCCESS

Type of partial success
Correct but incomplete, vs Unsound

Theme: EVIDENTIAL RELATIONS

For example,

• A zip code may suggest an address, and
• Address may suggest that we are looking at a home page,
• Which may in term suggest that a phone number ought to be present.

Alan Borning has been looking at using constraints to solve the problem of laying out web pages. The main motivation is that html itself is a crude way to do layout. Html was originally intended to specify the semantics of a document. However, Author's want more control over the appearance, but cannot control the how a page is viewed. Therefore pages look great on the view settings they were designed for, but may not look so good as the settings change. What is needed is a way to control how the layout of a document changes when the viewing parameters are changed. Another problem is layout of automatically created web pages.

Constraints are one possible method for solving the aforementioned problem. Intuitively, a constraint is a relation that we want to hold. An example of a constraint is the statement: x < 7. The author specifies what s/he wants via constraints. A constraint solver determines how.

Types of constraints that could be used to layout web pages

1. label placement relative to figure
2. range on column width
3. requirements on browser window size
4. fonts

Alan talked about 2 different system implementations of the negotiation model

1. Both author and viewer do runtime constraint solving
2. Author's side does runtime constraint solving. Viewer has a fixed set of constraints. An example would be an authoring tool that compiles a Java program that takes the user's constraints as parameters.

There are some HCI issues with this approach. One major question with constraint authoring is how easy would it be for people to author constraint pages. One solution is for some provider to develop a set of constraint templates. The author would choose one and then maybe modify it somewhat. Also, someone suggested that programming by demonstration may be another avenue to explore.

We next dealt with the Solvers of Constraints. Solvers that we looked at derive from the simplex algorithm for linear programming and are listed below.

1. Cassowary - incremental solver that finds locally error better solutions.
2. QOCA - incremental solver that finds least squares solutions. (slower but better that #1)
3. Fourier - batch compiler that produces code for finding locally error better solutions.

• a1x1 + ... anxn = x,
• a1x1 + anxn <= x etc.

Cassowary is an incremental version which is optimized for incremental addition and deletion of constraints such as those in continuous moving figures.

The prototype implementation of the web-constraint system consists of a viewing tool and an authoring tool. The only difference between the two tools is that the viewing tool only has a subset of the functionality of the authoring tool. Cassowary is the solver used.

Some more issues.

1. Issue #1 Sudden changes of layouts are disconcerting for the user. A solution in a template based constraint system is to have overlapping preconditions for templates, and a penalty function for switching.
2. Issue #2 It may be hard to get unique web pages with a template based approach. Uniformity is helpful betwen web pages from the same organization. However, different organizations would like to have a unique style for their own pages. This seems to call for a template authoring tool instead of predefined templates.
3. Issue#3 Combining constraints with strudel.... Strudel and the constraints work address orthogonal issues in web design, can we integrate them?

The paper presents WebSQL, an SQL-like language for querying the Web that allows both textual and structure- or topology-based queries. The design emphasizes "query locality," a theory of query cost based on how much of the Web must be visited to answ er a query. WebSQL uses multiple, existing index servers when appropriate, so it leverages off available resources. The system is envisioned by the authors not as an end-user application but as a development tool.

• W3QL [KS95], our paper 1.4, and
• WebLog [LSS96], Lakshmanan, Sadri, and Subramanian. 1996. "A Declarative Language for Querying and Restructuring the Web." Proc. of 6th International Workshop on Research Issues in Data Engineering, RIDE '96, New Orleans, February.

The main topics taken up in the discussion were:

The Features and Capabilities of WebSQL

	Document(url, title, text, type, length, modif)

	Anchor(base, href, label)

	SELECT	d.url, d.title, d.length, d.modif
	FROM	Document d
		SUCH THAT d MENTIONS "hypertext"
	WHERE	d.type = "text/html" ;

	SELECT	d.url, d.title
	FROM	Document d SUCH THAT
		"http://www.cs.toronto.edu" = |->|->-> d
	WHERE	d.title CONTAINS "database" ;

MENTIONS and CONTAINS

A key point made in the discussion of the BNF specification of WebSQL was the distinction between the seemingly similar keywords MENTIONS and CONTAINS. MENTIONS is used, optionally, in specifying a DomainCond in the FROM clause of a query (Example 2.1 ); CONTAINS is used, also optionally, in the WHERE clause to specify a BoolTerm (Example 2.2). MENTIONS is used with the first argument free and the second bound; it generates bindings for the first argument. CONTAINS is used with both arguments bound; it evaluates bindings. (The distinction gives rise to query optimization opportunities, but the subject was not pursued in the paper or the discussion.)

Query Completeness, Safety, and Computational Complexity

There was considerable discussion of query completeness, safety, and computational complexity in the context of querying the structure of the Web. The sense of the discussion was that there is a tension between completeness of query answers and comput ational tractability. On the one hand, it is highly desirable that the results of any query be complete, or, at the least, that you know whether the query answer is complete or not. On the other hand, an arbitrary query of the Web that gives a complete answer may be computationally intractable either because of the size of the domain searched -- the Web is large -- or because of the type of query made. This tension leads to the need for careful consideration of bounds and range variables to specify the search space, and, of the types of queries permitted. That is, the query language used. Query locality, discussed next, is one strategy within WebSQL that attempts to address this problem. (It was also noted that, as a practical matter, WebSQL's answe rs are incomplete (over the entire Web) because it relies on search engines that do not cover the entire Web.) Alon noted that from a database perspective, it is always necessary to "think big", that is, in terms of large amounts of data. While NP compl exity may be acceptable in small-knowledge-base applications, it is highly unlikely ever to be acceptable in a database context.

There was no clear resolution of this discussion, but Dan noted that the issues raised are similar to those addressed by Doyle and Patil in "Two Theses of Knowledge Representation." The paper is available at:

	http://www.medg.lcs.mit.edu/ftp/doyle/td91.ps

(For a different -- theoretical -- perspective on computational complexity and the Web see Abiteboul and Vianu, "Queries and Computation on the Web," cited as [AV97] in the WebSQL paper and available at

	http://www-rocq.inria.fr/~abitebou/pub/INDEX.html

under Database Theory. [AV97] models the Web as an infinite, semistructured set of objects to capture the intuition that exhaustive exploration of the Web is -- or will soon become -- prohibitively expensive. They introduce a Web machine, anal ogous to a Turing machine, and browser machine and browse/search machine variants. Their goal is to consider the computational characteristics of first-order logic, Datalog, and Datalog with negation in the context of the infinite Web model and the Web m achines. Interesting stuff.)

Other Primitives

Working from one of Dan's slides, there was a brief discussion of other primitives that might be useful to include in WebSQL. Dan suggested:

• Page classification, for example, JPEG isA Image
• HTML parsetree relationships
• Filename regexpr matching in paths
• Sorting

Query Locality

There was a more extended discussion of query locality along two lines: usefulness of the concept and efficiency issues. Query locality is an attempt to characterize the search space or domain of a query. The idea is as follows. Much of the informat ion contained in the structure of the Web is expressible in terms of path regular expressions between objects. One recurring example we have used: the graph structure of links between departmental homepages, personnel, and publications provides indicativ e information about which is which -- especially when used with probabilistic heuristics. So it would be quite useful to characterize the search space in terms of these path regular expressions. Using this strategy, we would have a mapping from the stru ctural queries posed, to the subnetwork searched, and so to the computational complexity of the query -- for a given query language. WebSQL attempts to develop such a mapping from path regular expressions to search space by distinguishing among different types of links: internal links (intra-document), local links (intra-server), and global links (inter-server). (Compare this issue to the preceding discussion of query completeness and computational complexity.)

There was mixed sentiment about the usefulness of the concept, but general agreement that the interior-local-global distinction was not quite the right parse of the idea. (The authors do note that extending the expressiveness of link types, possibly t o arbitrary link types, is one area of possible future work.) There was some discussion of: How do you do better? One idea was to parse on the URL string, for example, javasoft.sun.com versus sun.com as an indicator of locality.

In terms of query locality in the design of WebSQL, Alon made the point that query locality was built into the language too integrally. It might be more useful if query locality were an attachable, application-specific module that could be plugged int o the system. Another opinion along the same lines was that structural queries and textual queries should be orthogonal capabilities rather than tightly integrated.

There followed a general discussion of the merits of the paper cast, loosely speaking, in terms of the theorem-to-definition ratio of the paper. The paper had too many definitions -- maybe -- and not enough interesting theorems. The counterargument w as that, in the assessment of query languages, theorems do not get you much insight. The query languages are all subsets of Datalog -- which is well understood. (This discussion, too, echoed the earlier completeness-tractability theme.) But if the defi nitions are not interesting and the theorems are not interesting, what is interesting about the work? One answer: our next topic.

What is interesting about query language proposals is often the implementation and optimization issues. The big question is: How do you do this (Figure 2 from the paper, in our case) efficiently in a way that satisfies the semantics? The paper gives us the architecture of the system (Figure 2), but unfortunately, very little on the implementation and nothing on efficiency, including any real experimental results. (This counts as a major limitation of the paper.) It was noted that Stanford's Lore project (Lightweight Object Repository) and its query language, Lorel, indicate that the problem of query optimization for semistructured data is wide open. Among the Lore papers available at:

	http://www-db.stanford.edu/lore/

• McHugh, et al. "Lore: A Database Management System for Semistructured Data"
• Goldman and Widom. "DataGuides: Enabling Query Formulation and Optimization in Semistructured Databases."

Utility of WebSQL, Related Work, and Successor Systems

The three possible uses of the system, presented in the paper's discussion section, were raised:

• Selective Indexing
• View Definition
• Link Maintenance

Most of the discussion was about selective indexing and link maintenance. For selective indexing the strongest point in favor of the usefulness of WebSQL was the question: What is the alternative? Answer: write a program that does the same thing. Bu t with a declarative language, you only have to specify what you want, not how to do it, and, you only have to do it once. While a declarative language is the right tool, WebSQL may have the wrong primitives for the task. In particular, the internal-loc al-global link classification may not be useful. This led to a discussion of the usefulness of an "about" primitive, as in "What is this page about?" The consensus seemed to be that an about primitive was a potentially useful idea, but that it was also sema ntically tricky and subtle. Can "aboutness" of a page be determined from the path regular expression used to reach it? Maybe. But it is hard to determine exactly how complex the path regular expression parse would have to be to reliably determine the a boutness of a class of indexed objects. There is also the possibility of alot of false positives.

WebSQL was thought to be useful for link maintenance through a pretty straightforward application. But, Alon made the telling point that link maintenance is just one task in the more general application of website maintenance. This being the case, wh y not do it the right way from the beginning, for example, with Strudel?

Two other possible applications were considered: recommender systems and homepage finders. It was felt that WebSQL showed considerable promise as a recommender system. The advantage of the system is that it can make recommendations based on structura l relationships in addition to keyword vector similarity. It also permits rapid prototyping. One simple example, it could recommend all pages that point to a given page. Though WebSQL seemed promising as a homepage finder based on the combination of he uristics and the structural relationships between pages that match on the heuristics, it was generally felt that a serious shortcoming for this application domain was that WebSQL had no built-in probabilistic capability.

The final topic of discussion was possible successor systems. Here the most interesting possibility raised centered on communication between Strudel-like unmaterialized websites and search engines. Such sites could provide information about paths and structural relationships, among other things. This could be quite useful for structural queries such as those generated by WebSQL. It was noted, however, that if such sites are not materialized, there is no guarantee that the information they supply is true. They could, essentially, say yes to everything with the goal of generating traffic. The meeting broke up with a food-fight over Spam Strudel.

The two papers we discussed on this Thursday, the GroupLens and the Hidden Web papers, deal with social filtering in the realm of recommender systems.

GroupLens

What is it?

GroupLens is a collaborative filtering system designed with netnews in mind. The idea in GroupLens is to find other users whose interests in news articles are correlated with your own. The assumption is that, if your and other users' interests in news articles have been correlated in the past, then your interests are likely to be correlated in the future. What is the useful result of this assumption? That a user can use other users' ratings of news articles to determine how likely that user will be interested in a particular article.

I'm tough. Show me some math

The ratings on news articles were on a scale from 1 to 5, where 5 indicated most liking the article. Below is a sample correlation matrix for 4 users across 6 news articles. The question at hand is, what is Ken's likelihood to like article 6?

Article	Ken	Lee	Meg	Nan
1	1	4	2	2
2	5	2	4	4
3			3
4	2	5		5
5	4	1		1
6		2	5

We need to compute the correlation coefficients r_x,y for each pair of people x,y. The formula for r_x,y is:

Using this formula, we can compute as r_K,L = -0.8 and as r_K,M = 1.0. Because we can use only other users for whom we have a score for article 6, only r_K,L and r_K,M are of interest. Combining these coefficients gives us:

for an ultimate answer of K₆ = 3.3.

Yeah, but will it work for more than 4 users?

All of the tests that the authors reported in the GroupLens paper were on user bases of size 4 - 6ish. While the correlation information that they found was impressive (that correlations in the past really did do a reasonable job of finding correlations in the future), it is not clear that such a system will scale to a user base of hundreds of thousands of netizens.

Some issues about scalability include quality of prediction and performance, both in computational and bandwidth required. On the quality of prediction front, the issue seems to be that there is a startup cost associated with using this system. That is, before any of the past users' ratings can be useful to you, you have to have rated some news articles yourself. How many? Does this number depend upon the number of users?

As for the performance issues, the worry is that the algorithm for computing correlation between users is quadratic in the number of users. For P (the number of people in the user base) being small, O(P²) isn't too bad. However, as P approaches the 6, 7, or 8 digit mark, O(P²) is far more daunting. In addition, even if the cost of computing the correlations is neglected, the cost of transporting this huge database of (user,article) ratings is excessive as the number of users (and even the number of articles) grows tremendously.

One possible solution to this performance problem would be to cluster users into archetypes -- canonical user profiles. Each user would belong to an archetype within the realm of one newsgroup (GroupLens splits the domain of interest at newsgroup boundaries). As users rate articles, their ratings would contribute to their archetype. To determine their predicted ratings, the correlation computation would be made on the AxA matrix of archetypes, not users. Thus, with a limited number of archetypes, the computational cost is fixed. Of course, not all is rosy with archetypes. The main problem is that archetypes are never fine grained enough to suit all users just right. Perhaps a weighted average of several archetypes would solve this problem?

Will somebody think of the children?

Even with issues of scalability aside, there are other concerns of this system, in particular, social implications. One of the foundations for the GroupLens system is that there are other users out there who have rated articles in the past and who have rated articles that you are about to read. What motivates those other users? Why would a user want to submit ratings for an article, or submit an accurate rating? If it's just as easy to click the Severely Dislike button all the time, what reason does a user have to not do so?

Another circumstance is new material. When a new piece of news is posted, it initially has no rating (except, perhaps, the author's rating of her own work). In order for there to be a rating on this article, someone will have to eventually read and rate it. But, what is the incentive to even consider reading an article that has no rating yet? What is a user's motivation not to simply sit on their thumbs waiting for someone else to rate the article first?

Perhaps one solution to the incentive problem would be to transact money somehow. One could imagine that submitting ratings would earn a user credit. Perhaps this credit could be spent by receiving ratings on articles that the user has yet to see. The credit could be higher for newer articles, and lower for articles for which many ratings already exist.

Hidden Web

It's not the kind of web you're thinking of...

The Hidden Web paper is all about data mining for the web of personal contacts that exist between people. The thought is that, if you have a question about some topic, you'd like to find an expert in that field. Instead of conducting a length search "by hand" (sending email to colleagues, then contacting their acquaintances, etc.), it would be nice to narrow that search down to just the right people.

How does it work?

The idea is as follows: starting with a single person (the one who has the question), a graph is built. Nodes in the graph represent people; edges represent contact between the adjacent nodes. The graph is built until a node is added that is considered to be an expert in the topic. Because the graph is always being grown by adding adjacent nodes, there is a path from the seed node to this expert node.

There are two immediate questions that come to mind with this algorithm: (1) how are the nodes found/edges established? (2) how is a node determined to be an expert?

To answer the first question, the algorithm uses a simple iterative graph growth algorithm. To grow the graph one level from node N, N's name is searched for at, say, Alta Vista. Then, all the names from the pages returned by the search engine are extracted. Using the ratio of (number of pages that share the names) : (number of pages that don't share the names), edges between N and these extracted names are established.

To determine if an individual N is an expert in a particular field, a similar technique to that above is used. Pages on which N's name appears are fetched. Then, all capitalized words are considered. A TFIDF metric is finally used to determine the correlation between N's name and words in the topic in question.