Course Description

Compositionality---the fact that the meaning of complex expressions is built up from the meanings of the parts and the structure of the complex---has been argued to be a unique feature of human language, explaining our ability to systematically and productively produce and understand novel utterances. At the same time, this property has played a pivotal role in debates about cognitive architecture and artificial intelligence, with many arguing that standard neural models are fundamentally incapable of the requisite kind of systematic behavior. In this seminar, we will digest a wide range of literature starting with philosophical, linguistic, and cognitive foundations around compositionality, before looking at case studies in nonhuman animals and finally a wide range of recent literature on compositionality in modern language models. Topics include how to measure and detect compositionality, whether natural language and animal communication systems are compositional, how to build neural architectures that exhibit compositionality, compositionality versus generalization more generally, and data augmentation and meta-learning approaches. This will be an active research seminar, where students will work in small groups to execute a prototype research project on a topic of their choosing in consultation with the instructor as well as guide discussions of chosen readings.

Days	Time	Location
Mondays	3:30 - 5:50 PM	DEN 258 https://washington.zoom.us/j/92812102564

Note: while lectures will be delivered live at the above time and location, they will also be recorded and posted to the course Canvas page.

Teaching Staff

Role	Name	Office	Office Hours
Instructor	Shane Steinert-Threlkeld	GUG 415K and https://washington.zoom.us/my/shanest	Wednesday 3-5PM

Preferred Prerequisites

LING 571
LING 578
LING 574 helpful

Course Resources

Canvas: lecture recordings, discussion board, homework submission / grading
Patas Account Request
Computing Resources Orientation
Condor Wiki Pages

Policies

As a project-oriented, student-driven, seminar-style class, active participation---in the classroom, or in Zoom, as well as on Canvas---is expected. Primary course-time will be conducted as a "role-playing reading seminar", where each person reads a paper with a particular role / perspective and uses that to kick off discussion (see Course Structure for more information).

A final project will be carried out in small groups. Groups are free to divide up work as they see fit, but will be required to explain the division of labor with their final project. Except under rare circumstances, every member of a group will receive the same grades.

We understand that you may face hard times as we navigate an ever-changing world due to the COVID-19 pandemic and many other world events. If you find yourself struggling with a difficult concept; stressed over politics or health; slowed by monopolistic internet providers; or annoyed at a classmate, please remember that they might feel similar. Maybe not in your very moment, but certainly recently or soon. Some of you may find the return to hybrid teaching conducive to your style of learning and personality. Others may find it stressful or difficult. These are all normal reactions. Please have compassion and empathy, and assume that everyone is doing their best.

If you find yourself having trouble learning in class, please do not hesitate to let me know. My goal is to make this class a bright spot in these unprecedented times, and to do whatever we can to promote a healthy learning environment for all.

A note on time zones

All deadlines and meeting times for this class are in "Pacific Time". Now that we are in Daylight Savings Time, this is UTC-7.

Grading

60% weekly reading assignments
40% final project

Communication

Any questions concerning course content and logistics should be posted on the Canvas discussion board. If a more personal issue arises, you can email me personally; include "LING575" in the subject line. You can expect responses from teaching staff within 24 hours, but only during normal business hours, and excluding weekends.

Religious Accommodation

Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW's policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy (https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/). Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form (https://registrar.washington.edu/students/religious-accommodations-request/).

Access and Accommodations

Your experience in this class is important to me. If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course.

If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DRS at 206-543-8924 or uwdrs@uw.edu or disability.uw.edu. DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions. Reasonable accommodations are established through an interactive process between you, your instructor(s) and DRS. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law.

Safety

Call SafeCampus at 206-685-7233 anytime---no matter where you work or study---to anonymously discuss safety and well-being concerns for yourself or others. SafeCampus's team of caring professionals will provide individualized support, while discussing short- and long-term solutions and connecting you with additional resources when requested.

Course Structure

This course will function as a role-playing seminar: each week, we will read and discuss two papers as a group. Each student will sign up for a role for one of the papers, and will be expected to contribute to the discussion from that perspective. Every role must have at least one student, but some roles can have more than one. (More information is on Canvas.) The roles:

🕵 Archaeologist: Determine where this paper sits in the wider landscape of previous and subsequent work. More concretely: find and report on one prior paper that substantially influenced the current paper and one newer paper that cites this current paper. (You can use Google Scholar or SemanticScholar to find papers citing the current paper.)
🔬 Reproducer: You are attempting to reproduce the main results of the paper, either to confirm its conclusions, or to carry out similar experiments. If you are unable to reproduce the results, try to understand why. If you are able to, explain what helped the most. (Note: you are not expected to actually reproduce papers for this class. Rather, assume that you are trying to, and report on what you find on whether you would be able to.)
🧑‍🔬 Researcher: Based on the current paper, propose an experiment or analysis that would extend the work in a new direction. This could be a new dataset, a new model, a new evaluation metric, etc. In particular, try to think of something that is only possible thanks to the current paper, not just a simple extension. (Note: these can serve as inspiration for your final project.)
⚖ Reviewer: Write a referee report for the current paper, following the form for ACL Rolling Review (up through the "Overall Assessment"). Aim to approach papers charitably, but critically, thinking of how the paper can be made into the best version of itself. Their reviewer tutorial may also be helpful.

Each week, a shared Google Slides deck will be created, and each role-player will be expected to contribute a couple of slides, summarizing their view of the paper from their role. The deck will be shared with the class, and will be used to guide the discussion of the paper.

Non-presenter assignment: for the other paper that week, for which you have not signed up for one of the roles, you will be asked to (i) contribute an alternative title for the paper and (ii) at least one question about the paper (e.g. something that was confusing, or someothing that you want to hear more about).

Because of the length of the seminar meetings, we will take a 10-minute break in the middle of each class, in between the discussion of each paper.

One final note: in the interest of this being a student-led seminar, the reading list below can change based on the interests of the participants and everyone's projects. If you have a paper that you would like to discuss, please let me know.

Date	Topics	Primary Readings	Secondary Readings	Additional info
Mar 25	Introduction Definitions + Arguments	Compositionality	Compositionality and Cognitive Architecture Compositionality, I. Definitions and Variants (Sections 1 and 2) Compositionality, II. Arguments and Problems A taxonomy and review of generalization research in NLP	Group formation (out)
Apr 1	Measuring compositionality	Measuring Compositionality in Representation Learning Measuring Compositional Generalization: A Comprehensive Method on Realistic Data	Compositionality and Generalization In Emergent Languages (Sections 1, 2.1, 3) Language as an evolutionary system (esp. Appendix A)	Group formation due
Apr 8	Benchmarks, I	Generalization without systematicity: On the compositional skills of sequence-to-sequence recurrent networks Compositionality Decomposed: How Do Neural Networks Generalise?	mSCAN: A Dataset for Multilingual Compositional Generalisation Evaluation Human few-shot learning of compositional instructions	Proposal guidelines out
Apr 15	Benchmarks, II	COGS: A Compositional Generalization Challenge Based on Semantic Interpretation A Benchmark for Systematic Generalization in Grounded Language Understanding	SLOG: A Structural Generalization Benchmark for Semantic Parsing
Apr 22	Architectural approaches	Span-based Semantic Parsing for Compositional Generalization Compositional generalization by learning analytical expressions	Permutation Equivariant Models for Compositional Generalization in Language Compositional Generalization via Neural-Symbolic Stack Machines	Proposal due
Apr 29	Other approaches	Good-Enough Compositional Data Augmentation Human-like systematic generalization through a meta-learning neural network	The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers Learning Compositional Rules via Neural Program Synthesis
May 6	Prompting approaches	Compositional Semantic Parsing with Large Language Models Faith and Fate: Limits of Transformers on Compositionality	Least-to-Most Prompting Enables Complex Reasoning in Large Language Models On the Compositional Generalization Gap of In-Context Learning How Do In-Context Examples Affect Compositional Generalization? Measuring and Narrowing the Compositionality Gap in Language Models
May 13	Emergence of compositionality	Compression and communication in the cultural evolution of linguistic structure Compositionality and Generalization In Emergent Languages	Emergence of Grounded Compositional Language in Multi-Agent Populations Toward the Emergence of Nontrivial Compositionality
May 20	Animal communication	Formal Monkey Linguistics (through Section 3) Syntax and compositionality in animal communication	Compositionality and competition in monkey alert calls Honey Bee Recruitment: The Dance-Language Controversy	Final paper guidelines out
May 27	No Class (Memorial Day)