Technical Tutorials, Notes, and Suggested Reading

These notes or slides were written for various occasions. Many are high-level explanations aimed at conveying key intuitions before the student reads a more detailed treatment.

If any of these are inaccessible to you, please let me know!

• The difference between natural language processing (NLP) and computational linguistics (CL)
• The difference between AI, ML, and NLP
• The difference between frequentists and Bayesians: A dialogue
• The three cultures of machine learning
• Probability crash course (video+slides) — how to build simple probabilistic models
• Interactive lessons in log-linear modeling — nifty visualization toy with sliders
  • Latent-variable log-linear modeling
• Interactive visualization of kernel SVMs (by Guillaume Caron: I'm only hosting it)
• Lagrange multipliers — high-level explanation
• Dynamics of optimizing Gaussian mixture models (Jupyter R notebook)
• Variational inference — high-level explanation
• Belief propagation (slides) — high-level explanation; ACL 2014/2015 tutorial
• Hidden Markov Models (video+more) — a fun detailed example
• Back-propagation (animated slides+video) — high-level explanation; also see suggested readings at top of this page
• Understanding the inside-outside and forward-backward algorithms -- they're just backprop
• An annotated drawing of an LSTM unit (based on Graves 2012)
• Semiring-weighted finite-state machines (draft tutorial) — email me for this
• Bayesian generative modeling (video+slides) — works up to topic models and Bayesian HMMs
• Minimum spanning tree (tutorial paper) — deep and clear coverage of how 7 algorithms were designed
• Convert a formula from SAT to CNF-SAT (pseudocode and discussion)
• Competitive grammar writing exercise, with software

Many additional topics are explained nicely on slides for my courses (or conference talks). Here are some papers of mine related to teaching.

I've also explained many things by email over the years. Sadly, I haven't collected the emails, but I will post them here if I run across them....

If you like the videos, my other recorded tech talks are here.

See also my advice page.