AS.050.375, AS.050.675

Probabilistic Models of the Visual Cortex:
 
Tues/Thurs: 9:00-10:15am, Krieger 205, Fall 2021.
 
 

---

Course Description

The course gives an introduction to computational models of the mammalian visual cortex. It covers topics in low-, mid-, and high-level vision. It briefly discusses the relevant evidence from anatomy, electrophysiology, imaging (e.g., fMRI), and psychophysics. It concentrates on mathematical modelling of these phenomena taking into account recent progress in probabilistic models of computer vision and developments in machine learning.

Important information

• Instructor: Prof. Alan Yuille, ayuille1@jhu.edu, office hours: by email appointment. TA: Fengze Liu, fliu23@jhu.edu, office hours Friday 4-5pm, CA: Yihong Sun, ysun86@jhu.edu, office hours Monday 3-4pm
• Piazza discussion site [Piazza Course Site] We encourage the students to post questions about the course and homeworks on Piazza.
  

Reading and Background Material

• Course notes and course readings (posted online).
  
• A.L. Yuille and D.K. Kersten. EarlyVision. Invited Book Chapter in From Neuron to Cognition. 2016. This covers much of the first part of the course.
  
• Understanding Vision: Theories, Models, and Data. Zhaoping Li. Oxford University Press. 2014. This gives more detailed coverage of some topics in the first part of the course.
  
• Note: the second part of the course is based on recent work and so there is no book reference (just class notes and readings).
• Seeing: The Computational Approach to Biological Vision.  J.P. Frisby and J.V. Stone. (2nd Ed) MIT Press. 2010. Introduces the computational approach to vision but with limited mathematics.
• Stanford tutorials, the engineering website, http://deeplearning.stanford.edu/wiki/index.php/UFLDL_Tutorial, is a good resource for students with limited statistics background, as it introduces very basic regression, dimension reduction (PCA, ICA)  and vectors.
• Stanford tutorials,  http://vision.stanford.edu/teaching/cs231n/2017/ is a good supplement for the later part of the course, it describes how convolutional network are done and has a lot of codes and examples.
• Syllabus.
  

Homeworks

Grading Plan: 5 homework assignments will be posted here(roughly biweekly). The assignments need to be submitted via Gradescope (Entry Code: N8JYDJ).

Homework 1 Due on Oct 5 before class.
Homework 2 Due on Oct 26 before class.
Homework 3 Due on Nov 16 before class.
Homework 4 Due on Dec 14 11:59pm.
Homework 5 Optional to submit.

Preliminary Schedule (based on the last year's schedule and will be updated as the semester goes)

Lecture	Date	Topics	Handouts	Required Reading	Optional Reading
1	Aug-31	Introduction	Slides	YuilleKersten (Section 1.1, 1.2)	An extended version of the lecture:Slides
2	Sept-2	Introduction to Retina and Primary Visual Cortex (V1)	Retina V1	YuilleKersten (Section 1.3)	ReadMe Lecture by Clay Reid V1_Mike_May BialekPhotoReceptors GollischRetina VisualCrowding
3	Sept-7	Linear Filtering	Slides	YuilleKersten (Section 2.1)	KokkinosLinearFiltering CarandiniEarlyVisualSystem
4	Sept-9	Sparsity	Sparsity	YuilleKersten (Section 2.2)	SparsityPowerpoints SparsityYuilleKersten Mini-Epitomes Background redeading:BarlowSparsity1972
5	Sept-14	Filters for Binocular Stereo and Motion	Stereo Motion Figures	YuilleKersten (Section 2.4)
6	Sept-16	Hebbian Learning and Regression	Hebbian Regression	YuilleKersten (Section 2.3)	TalebiV1RF GallantNaturalStimulus ZhangCNN ZhangCNNV1Patterns
7	Sept-21	Vision as Bayesian Inference: Edges	VisionAsBayesianInference VisionAsProbabilisticInference	YuilleKersten (Section 3.1, 3.2)	chater2006probabilistic yuille2006vision
8	Sept-23	Bayes Decision Theory	BayesDecisionTheoryYuilleKersten	YuilleKersten (Section 3.1, 3.2)	YuilleLecture2UCLA
9	Sept-28	Cue Coupling Weak	Slides		Yuille&Buelthoff(1993)
10	Sept-30	Cue Coupling II	CueCouplingStrong DivisiveNormalization		ProbModelsOnGraphs
11	Oct-5	Context and Markov Random Fields	Slides	YuilleKersten (Section 4)	BeliefPropagationMFT
12	Oct-7	Context Examples	Slides	YuilleKersten (Section 4)	Same as Lecture 11
13	Oct-12	Motion and Kalman Filter	Slides BarlowTripathy Burgi et al.		A198
14	Oct-14	Boltzmann Machines & More Context Examples	Slides		TS Lee (2014)
15	Oct-19	Intro to Deep Nets	Slides FerusVittal MathDetails HintonAlexNet		YaminsNature2016 Yuille2020
16	Oct-21	Adversarial Machine Learning	Jason_Yosinski Slides PatchAttack	ZhouFirestone
17	Oct-26	Interpretable Deep Networks	UnsupervisedNAS QiaoFewShot		Bolei Zhou UnsupervisedDeepNets SmirnakisYuille1994 UnsupervisedFlow
18	Oct-28	The 3D world and unsupervised learning	Lambertian model of reflectance1 Lambertian model of reflectance2		EveryPixelCountsChenxuLuo2018 LambertianLighting
19	Nov-2	The 3D world and unsupervised learning(continue)	GeometryAndMotion Motion_Geometry2020
20	Nov-4	Human/Animal Parsing	BootstrappingDeepNetCueCoupling HumanAnimalParsing ParsingHumansCourse
21	Nov-9	Learning by Immagination	YouOnlyAnnotateOnce PhysicalSceneUnderstanding SimulationEngine
22	Nov-11	Compositional Models	CompositionalTheory ComplexityFundamentalProblem CompositionalModelsLearning		Generative Vision Model
23	Nov-16	Compositional Networks	AdversarialPatches CompositionalModelsOcclusion CompNetsAdamK		GeorgeCAPCHAS Detection_with_CompositionalNets Occluder_Localization_with_CompNets HongruZhuCogSci2019
24	Nov-18	Analysis by Synthesis	AnalysisBySynthesisIntro AnalysisBySynthesisDDMCMC		Image Parsing
	Nov-23	Thanksgiving
	Nov-25	Thanksgiving
25	Nov-30	Vision, Language, and Turing Tests	JunhuaMaoTextCaptioning CLEVR-Ref+
26	Dec-2	Model Robustness and Generalizability	AdversarialExaminerIntro CompositionalOpenSetActivity