CSE 599 D1: Advanced Natural Language Processing- Spring 2019 Wed, Fri 1:30-2:50 pm in Gates 271

 

Instructor: Hanna Hajishirzi (Paul Allen Center 470) (hannaneh at washington dot edu) Office hours: by appointment

TA: Dae Lee (dhlee4 at uw dot edu) Office hours: by appointment

Course Objectives

This advanced course deeply explores important topics in NLP. The objective of this class is to enhance students' knowledge about current techniques, challenges, and developments in different areas of NLP; to encourage discussions and collaborations among students; to improve students' analytical thinking and creativity; and to improve presentation and writing skills. In this class, students are required to read, think, present, and write intensively.

 

Theme: Representation and Reasoning in NLP

This course will be concentrated on knowledge representation and reasoning in NLP, including semantic representations, knowledge acquisition, question answering, and reasoning. The goal is to acquire comprehensive understanding and insights into the emerging developments and challenges of these advanced topics, and to develop future research directions and original research ideas.

 

Class Activities and Evaluation

Students will be evaluated based on these four activities (% of final grade):

(i) leading class discussions (30%),

(ii) participation in the discussions (25%),

(iii) paper review (15%),

(iv) research proposal, in which each student proposes an original research project (30%).

(Optional) students can optionally pursue a research project which will contribute toward additional 25% grade.

 

Prerequisite and Background Material

This advanced course deeply explores important topics in NLP. It is assumed that participants have taken CSE 517, are familiar with the fundamental ideas in NLP, and pursue research in NLP or related fields. Here are some pointers for background materials.

●      UW-NLP CSE517 [here]

●      NLP textbooks:

○      D. Jurafsky & James H. Martin, Speech and Language Processing: n Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition, Prentice Hall, Second Edition, 2009

○      C.D. Manning & H. Schuetze, Foundations of Statistical Natural Language Processing, Cambridge: MIT Press, 1999 (available online, free if accessed from UW computers)

Click here for potential papers for class discussions.

Schedule

Date	Topic	Topics	Note
3-Apr	Introduction
5-Apr	Transformer, GPT, GPT1, BERT, Illustrated Transformer, The Annotated Transformer; led by Antoine and Ari	Contextual text encoding
10-Apr	GCNs, GraphIE, GATs (Optional, will be covered briefly) ,R-GCN (Optional, will be covered briefly); led by James and Colin	Graph encoding
12-Apr	Making the V in VQA Matter, Referring Expression; led by Rowan and Kiana	Language/Vision
17-Apr	Knowledgeable Reader, Using Wiki as an external knowledge; led by Samantha and Kaitlyn	QA and Knowledge
19-Apr	Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text, Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks; led by Shobhit and Sewon	QA and Knowledge
24-Apr	Dynamic Integration of Background Knowledge in Neural NLU Systems, Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehension, Reasoning about Actions and State Changes by Injecting Commonsense Knowledge ;led by Kaidi and Deric	Reasoning in Narratives
26-Apr	CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge, , Ranking and Selecting Multi-Hop Knowledge Paths to Better Predict Human Needs, Modeling Naive Psychology of Characters in Simple Commonsense Stories (Optional), Commonsense Reasoning for Natural Language Understanding: A Survey of Benchmarks, Resources, and Approaches (Optional);led by Maarten and Hannah	Commonsense Reasoning
1-May	Guest speaker; ConvE, DisMult(Optional), ComplEx(Optional); led by Tim	Relational Graph Encoding
3-May	Learning a Natural Language Interface with Neural Programmer,Neural Semantic Parsing with Type Constraints for Semi-Structured Tables, Compositional Semantic Parsing on Semi-Structured Tables (Optional), Neural Symbolic Machines: Learning Semantic Parsers on Freebase with Weak Supervision (Optional) ;led by Derek and Nick	Neural Symbolic Methods	Paper review deadline(Before the class)
8-May	Rationalizing Neural Predictions ,Training Classifiers with Natural Language Explanations(Optional) ,How Much Attention Do You Need? ,The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation ,Using the Output Embedding to Improve Language Models;led by Aida and Ofir
10-May	Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing, Adversarial Decomposition of Text Representation;led by Sean and Phoebe
15-May	Unsupervised Data Augmentation , Semi-Supervised Sequence Modeling with Cross-View Training , Data Programming: Creating Large Training Sets, Quickly (Optional)led by Elizabeth and David(W)
17-May	Spider , TypeSQL, pair2vec;led by Ethan and David(P)
22-May	Graph Convolutional Networks Based Word Embeddings , A Structural Probe for Finding Syntax in Word Representations, Colorless Green Recurrent Networks Dream Hierarchically;led by Naomi and Chih-Chan		First proposal draft submission
24-May	Visual Dialog, Wizard of Wikipedia, Conversing by Reading: Contentful Neural Conversation with On-demand Machine Reading;led by Karen and Akshat
29-May	Neural Belief Tracker, Interpretation of Natural Language Rules in Conversational Machine Reading , Global-Locally Self-Attentive Dialogue State Tracker (Optional);led by Divye and Victor
31-May	Linguistically-Informed Self-Attention for Semantic Role Labeling, GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations, THE LOTTERY TICKET HYPOTHESIS: FINDING SPARSE, TRAINABLE NEURAL NETWORKS;led by Zhaofeng and Ellen
5-Jun			Second proposal draft submission(Noon 5-Jun)
7-Jun	A General Framework for Information Extraction using Dynamic Span Graphs, Giving Attention to the Unexpected; led by Vicky and Yi		Review Period Ends(11:59pm 7-Jun)
12-Jun			Last proposal draft submission(Noon 12-Jun)

 

Paper Review

We will provide the list of papers for the paper report. Check here for instructions for paper review. The paper review should not be longer than 2 pages. Deposit the paper review to the dropbox. Also, to see some more samples for paper reviews you can check the open review website https://openreview.net. For example, these are sample reviews from the most recent ICLR conference: https://openreview.net/group?id=ICLR.cc/2019/Conference

 

Research Proposal

Write a 4-page research proposal paper describing a line of research in NLP. The research proposal should be about a new project that would extend a clearly identified past research contribution. The research proposal should:

●      Build upon or extend what was done in the past work;

●      Address challenges or weaknesses in the past research;

●      Propose logical extensions or next steps to the focus research; and

●      Describe a possible evaluation methodology, experimental design, and required evaluation resources.

You can find some guidelines in the link, or schedule a meeting with the instructor or the TA to learn more.

 

The 4-page limit is a hard constraint and will be enforced seriously. References don't count toward the page limit. Please use the ACL 2019 style files without modification.

Submit the research proposal to the Dropbox.

 

Course Administration and Policies

●      Assignments should be done individually unless otherwise specified. You are expected to maintain the utmost level of academic integrity in the course.

●      Late Policy: Each student has SEVEN PENALTY-FREE DAY FOR THE WHOLE QUARTER(**UPDATED 25 APR 2019**), other than that any late submission will be penalized at a penalty of 10% of the maximum grade per day.

Communication

●      If you have comments of general interest, please use the discussion forum. Please consider posting your questions there; everyone will benefit. We also encourage you to try to answer questions, which will count as class participation. We will monitor daily and contribute as long as the boards are being used.

●      We appreciate feedback throughout the quarter -- you can submit feedback through the Allen School's anonymous feedback tool

●      Submit assignments to the dropbox