Multilingual Natural Language Processing

Learning Goals

Students who take this course should be able to develop linguistically motivated solutions to core and applied NLP tasks for any language. This includes understanding and mitigating the difficulties posed by lack of data in low-resourced languages or language varieties, and the necessity to model particular properties of the language of interest such as complex morphology or syntax. The course will introduce modeling solutions to these issues such as multilingual or cross-lingual methods, linguistically informed NLP models, and methods for effectively bootstrapping systems with limited data or human intervention. The project work will involve building an end-to-end NLP pipeline in a language you don’t know.

Announcements

• [Sept 24] Assignment 2 Released Here
• [Sept 14] Assignment 1 Recitation slides have been posted.
• [Sept 10] Assignment 1 Released [Description, Code+Data]
• [Sept 3] “Language in 10” sign up sheet has been released on Piazza.
• [Aug 23] Piazza, Canvas links updated
• [Aug 23] Class schedule updated
• [Jul 16] Website created

Pre-requisites

You must have taken an NLP class previously. Some examples include:

• Intro to Natural Language Processing (11-411/11-611)
• Algorithms for NLP (11-711)
• Neural Networks for NLP (11-747)

The assignments for the class will be done by creating neural network models, and examples will be provided using PyTorch. If you are not familiar with PyTorch, we suggest you attempt to familiarize yourself using online tutorials (for example “Deep Learning for NLP with PyTorch”) before starting the class.

Syllabus

_{The lecture plan and homework/project dates are tentative and subject to change.}

Class Format

Following safety guidelines due to the ongoing pandemic, all the lectures will be held on Zoom. To make the online lectures a good experience for the students, we will aim to make the lectures as interactive as possible. On most days, there will be a short ~30 minute lecture, with an optional reading assignment given beforehand. In every class, a group of students will be asked to introduce a new language in short 10 minute presentation (more details coming soon). At the end, we will have a 30 min discussion session using breakout rooms (these may be graded for participation) followed by a short 10 min summary.

Grading

This is a project based course and grading will be done based on class participation, three homework assignments and one final project with each of them contributing to your final grade as follows:

• Class Participation (15%)
• Assignment 1 (15%)
• Assignment 2 (20%)
• Assignment 3 (20%)
• Final Project (30%)

Final Project

The final project work will be expected to be a novel contribution to knowledge on multilingual language processing. In general, we will accept contributions in several categories, including the following:

1. Proposal of a novel method for multilingual or low-resource language processing that is better (more accurate, computationally efficient, or data efficient) than other methods in the literature. It may be easier to do this as a follow-up on one of the assignments, but you are free to tackle other tasks as well.
2. An extensive comparison of existing methods in the literature for tackling a particular multilingual or low-resource language processing task, analyzing their strengths and weaknesses and when you may expect them to succeed or fail (on multiple datasets).
3. Building a state-of-the-art language tool for a language for which such technology does not currently exist. You should apply methods that you learned in class in a way that is specifically tailored to the language based on its unique linguistic properties, related languages, or data availability.

For all submissions, they should include a report of up to 8 pages describing:
a. Background of the task and problems involved therein.
b. The choice of methodology, demonstrating internalization of the knowledge obtained in class.
c. Experimental setting and results, including strong baselines and proposed improvements for (1) and (3).
d. Analysis of the results demonstrating the characteristics of the implemented methods (especially important for (2)).
e. Which members of the group contributed in which ways to the implementation.

Grading:

• A+: A respectable research contribution that is novel and effective, and could be submitted largely as-is as a paper to an academic conference. All elements above are high quality.
• A: A respectable contribution that is largely complete and promising, but the description, sophistication of methodology, experiments, or analysis are not as fleshed out and complete as A+ assignments.
• A-: All required elements are present, but one or more are lacking to some extent.
• B+ or B: The project is complete, but one or more of the required elements is seriously lacking.
• B- or below: The project or description thereof is seriously incomplete.

Negative Results: Sometimes experiments don’t work as planned. If you try hard to get positive results but are not successful, you may still get a good grade by clearly describing why you thought your methods would work, and then performing an analysis of why your initial assumptions were incorrect, leading to results that did not match your initial expectations. The bar for paper writing, experimentation, and analysis will be a bit higher in these cases, as we want to make sure that you really made a serious effort.

Policies

Academic honesty. Homework assignments are to be completed individually. Verbal collaboration on homework assignments is acceptable, as well as re-implementation of relevant algorithms from research papers, but everything you turn in must be your own work, and you must note the names of anyone you collaborated with on each problem and cite resources that you used to learn about the problem. Suspected violations of academic integrity rules will be handled in accordance with the CMU guidelines on collaboration and cheating.

Note to Students

Take care of yourself! As a student, you may experience a range of challenges that can interfere with learning, such as strained relationships, increased anxiety, substance use, feeling down, difficulty concentrating and/or lack of motivation. All of us benefit from support during times of struggle. There are many helpful resources available on campus and an important part of having a healthy life is learning how to ask for help. Asking for support sooner rather than later is almost always helpful. CMU services are available, and treatment does work. You can learn more about confidential mental health services available on campus at: http://www.cmu.edu/counseling/. Support is always available (24/7) from Counseling and Psychological Services: 412-268-2922.

Accommodations for Students with Disabilities:

If you have a disability and have an accommodations letter from the Disability Resources office, I encourage you to discuss your accommodations and needs with us as early in the semester as possible. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, I encourage you to contact them at access@andrew.cmu.edu.