New York University Tandon School of Engineering
Department of Electrical and Computer Engineering
Introduction to Machine Learning
Fall 2018
Instructor: Anna Choromanska
Course Prerequisites
1) Undergraduate Probability
2) Mathematical maturity:
https://en.wikipedia.org/wiki/Mathematical_maturity
Office hours
Prof. Anna Choromanska: Friday 13.30-15.00, Room LC266D, 5 Metrotech Center
TAs:
Jason Cramer: Thursday 11.00-13.00, Prague Conference Room, 370 Jay on the 13th
floor (where CUSP is located)
Shihao Ju: Monday 9.30 – 11.30, Room 120.I, 2 MetroTech, 9th floor
Bo Pang: Tuesday 10:00-12:00, Room LC025, 5 Metrotech Center
Course Description
Machine Learning is nowadays one of the most rapidly developing technical
fields both in the academia and industry. It is also a fundamental tool used in
a wide range of different data science fields. This course presents the basic
concepts, techniques, and algorithms in machine learning from both
theoretical and practical perspective. The program of the course includes
empirical risk minimization, support vector machines, kernels, clustering,
principal component analysis, Expectation-Maximization, graphical models,
and neural networks.
Textbook
There is no textbook required. The list of recommended texts:
Pattern recognition and machine learning, C.M. Bishop
Pattern classification, R. O. Duda, P. E. Hart, and D.G. Stork
T. Jebara. Course notes, Machine Learning
S. Dasgupta. Course notes, CSE 291: Topics in unsupervised learning
Homeworks
For coding, preferred environments is Matlab. Homeworks are due at 3pm on the
given day.
Course Work and Grading
Your final grade will be determined roughly as follows:
1
Homework 30%
Midterm 30%
Final 40%
Tentative Schedule
• Week 1 (09.07.2018): (Topic 2) Regression, Empirical Risk Minimization, Least
Squares, Higher Order Polynomials, Under-fitting / Over-fitting, Cross-Validation
and (Topic 3) Additive Models and Linear Regression, Sinusoids and Radial Basis
Functions, Classification, Logistic Regression, Gradient Descent
Homework 1 is released and due 09.21.2018.
• Week 2 (09.14.2018): (Topic 4) Perceptron, Online & Stochastic Gradient
Descent, Convergence Guarantee, Perceptron vs. Linear Regression, Multi-Layer
Neural Networks, Back-Propagation, Demo: LeNet, Deep Learning
• Week 3 (09.21.2018): (Topic 5) Generalization Guarantees, VC-Dimension,
Nearest Neighbor Classification (infinite VC dimension), Structural Risk
Minimization, Support Vector Machines
Due date for Homework 1.
Homework 2 is released and due 10.05.2018.
• Week 4 (09.28.2018): (Topic 6) Kernels and Mappings and (Topic 7) Introduction
to Probability Models
• Week 5 (10.05.2018): (Topic 8) Discrete Probability Models, Independence,
Bernoulli Distribution, Text : N a ï v e , Bayes, Categorical / Multinomial
Distribution, Text: Bag of Words and (Topic 9) Continuous Probability Models,
Gaussian Distribution, Maximum Likelihood Gaussian, Sampling from a Gaussian
Due date for Homework 2.
Homework 3 is released and due 10.19.2018.
• Week 6 (10.12.2018): (Topic 10) Classification with Gaussians, Regression with
Gaussians, Principal Components Analysis and (Topic 11) Maximum Likelihood as
Bayesian Inference, Maximum A Posteriori, Bayesian Gaussian Estimation
• Week 7 (10.19.2018): (Topic 12) Mixture Models and Hidden Variables,
2
Clustering, K-Means, Expectation Maximization and (Topic 13) Expectation
Maximization
Due date for Homework 3.
Homework 4 is released and due 11.02.2018.
• Week 8 (10.26.2018): MIDTERM
• Week 9 (11.02.2018): (Topic 14) Structuring Probability Functions for Storage,
Structuring Probability Functions for Inference, Basic Graphical Models, Graphical
Models, Parameters as Nodes
Due date for Homework 4.
Homework 5 is released and due 12.14.2018.
• Week 10 (11.09.2018): (Topic 15) Bayes Ball Algorithm and (Topic 16 Part 1)
Junction Tree Algorithm
• Week 11(11.16.2018) (Topic 16 Part 2 and Topic 17) Junction Tree Algorithm
• Week 12 (11.30.2018) (Topic 18) JTA and (Topic 19) HMM
• Week 13 (12.07.2018) (Topic 20) HMM
• Week 14 (12.14.2018) Introduction to Deep Learning and CNNs
Due date for Homework 5.
• Week 15 (…): FINAL EXAM
