This course covers several important machine learning algorithms for natural language processing including decision tree, kNN, Naive Bayes, support vector machine, maximum entropy / multinomial logistic regression, conditional random fields, and neural networks. Students implement many of the algorithms and apply these algorithms to some NLP tasks.
|Tuesday and Thursday||1:00 - 2:20 PM||ECE 045|
|Instructor||Shane Steinert-Threlkeld||Guggenheim 418-D (and Zoom)||Tuesday, 2:30 - 4:30 PM|
|Teaching Assistant||Yuanhe Tian||Guggenheim 417 (the Treehouse)||Wednesday, 3 - 4 PM
Friday 10 - 11 AM
There is no required textbook. Instead, the course readings will be drawn from contemporary articles and tutorials available online. Some material will be pulled from the following books, which will be referenced by their parenthecized names:
N.B.: All homework grading will take place on the patas cluster using Condor, so your code must run there. I strongly encourage you to ensure you have an account set up by the time of the first course meeting.
Unless explicitly mentioned below, the shared policies of the LING 57x course series apply to this course. Please read those policies for more information.
As per the policy above, all communication outside of the classroom should take place on Canvas. You can expect responses from teaching staff within 48 hours, but only during normal business hours, and excluding weekends.
N.B.: while CLMS students have a private Slack channel, I strongly encourage questions concerning course content and assignments to be posted to the Canvas discussion board, for two reasons. (i) Teaching staff will not look at Canvas, so misinformation can spread. (ii) Not every student in the course is in the CLMS program, but they deserve to be included in course discussions and likely have many of the same questions.
Washington state law requires that UW develop a policy for accommodation of student absences or significant hardship due to reasons of faith or conscience, or for organized religious activities. The UW’s policy, including more information about how to request an accommodation, is available at Religious Accommodations Policy (https://registrar.washington.edu/staffandfaculty/religious-accommodations-policy/). Accommodations must be requested within the first two weeks of this course using the Religious Accommodations Request form (https://registrar.washington.edu/students/religious-accommodations-request/).
Your experience in this class is important to me. If you have already established accommodations with Disability Resources for Students (DRS), please communicate your approved accommodations to me at your earliest convenience so we can discuss your needs in this course.
If you have not yet established services through DRS, but have a temporary health condition or permanent disability that requires accommodations (conditions include but not limited to; mental health, attention-related, learning, vision, hearing, physical or health impacts), you are welcome to contact DRS at 206-543-8924 or email@example.com or disability.uw.edu. DRS offers resources and coordinates reasonable accommodations for students with disabilities and/or temporary health conditions. Reasonable accommodations are established through an interactive process between you, your instructor(s) and DRS. It is the policy and practice of the University of Washington to create inclusive and accessible learning environments consistent with federal and state law.
Call SafeCampus at 206-685-7233 anytime – no matter where you work or study – to anonymously discuss safety and well-being concerns for yourself or others. SafeCampus’s team of caring professionals will provide individualized support, while discussing short- and long-term solutions and connecting you with additional resources when requested.
|Date||Topics||Suggested Readings||Assignment out||Assignment due|
[classification] [probability] [MALLET]
Cover and Thomas, ch 2
|Jan 9||Decision Trees||ML ch 9|
|Jan 14||Naive Bayes||
JM ch 4
McCallum and Nigam 1998
[pdf, tex, slides]
|Jan 16||k-Nearest Neighbors (kNN)||IR 14.3 -14.4||HW1 due|
|Jan 21||Feature Selection||
Unit 1 recap
Maximum entropy (MaxEnt) I: main concept
linear programming intro
Berger et al 1996, "A maximum entropy approach to NLP"
|HW4 out||Reading #1 in|
|Jan 30||MaxEnt II: modeling and decoding||Ratnaparkhi 1997||HW3 in|
|Feb 4||MaxEnt II (cont)||HW5 out|
MaxEnt III: training
MaxEnt IV: beam search
Klein and Manning 2003
|Feb 11||Conditional Random Fields (CRF)||Sutton and McCallum 2006||
|Feb 13||Support Vector Machines (SVM) I: linear||IR, Ch 15||HW5 in|
|Feb 18||SVM I (cont)||IR, Ch 15||Reading #3||Reading #2 in|
SVM II: non-linear
|IR Ch 15||HW6 in|
SVM III: tree kernel
SVM IV: transductive SVM
Collins and Duffy 2001
Joachims 1999 (ICML)
|HW7 out||Reading #3 in|
|Feb 27||Neural Networks: Introduction||
DL book ch 6.1-6.4
3blue1brown NN videos
DL book ch 6.5
CS 231n notes 1
CS 231n notes 2 (vector/tensor derivatives)
Yes, you should understand backprop
|Mar 5||Recurrent Neural Networks||
DL book ch 10
The Unreasonable Effectiveness of Recurrent Neural Networks
Sequence to Sequence Learning with Neural Networks (original seq2seq paper)
Neural Machine Translation by Jointly Learning to Align and Translate (original seq2seq + attention paper)
Pre-training / transfer learning
Attention is All You Need (original Transformer paper)
The Annotated Transformer
The Illustrated Transformer
NLP's ImageNet Moment Has Arrived
The Illustrated BERT, ELMo, and co. (How NLP Cracked Transfer Learning)
|Mar 12||Summary||HW8 in|