arXiv:1709.02840v3 [cs.
LG] 17 May 2018
                                              A Brief Introduction to Machine
                                                        Learning for Engineers
                                          (2018), “A Brief Introduction to Machine Learning for Engineers”, :
                                         Vol. XX, No. XX, pp 1–231. DOI: XXX.
                                                                                       Osvaldo Simeone
                                                                                Department of Informatics
                                                                                    King’s College London
                                                                                osvaldo.simeone@kcl.ac.uk
                              Contents
I   Basics                                                                          5
1 Introduction                                                                       6
  1.1 What is Machine Learning? . . . . . . . . . . . . . . . . .                    6
  1.2 When to Use Machine Learning? . . . . . . . . . . . . . .                      8
  1.3 Goals and Outline . . . . . . . . . . . . . . . . . . . . . .                 11
2 A Gentle Introduction through Linear Regression                                   15
  2.1 Supervised Learning . . . . . . . . . . . . . . .     .   .   .   .   .   .   15
  2.2 Inference . . . . . . . . . . . . . . . . . . . . .   .   .   .   .   .   .   17
  2.3 Frequentist Approach . . . . . . . . . . . . . .      .   .   .   .   .   .   19
  2.4 Bayesian Approach . . . . . . . . . . . . . . .       .   .   .   .   .   .   36
  2.5 Minimum Description Length (MDL)∗ . . . . .           .   .   .   .   .   .   42
  2.6 Information-Theoretic Metrics . . . . . . . . . .     .   .   .   .   .   .   44
  2.7 Interpretation and Causality∗ . . . . . . . . . .     .   .   .   .   .   .   47
  2.8 Summary . . . . . . . . . . . . . . . . . . . . .     .   .   .   .   .   .   49
3 Probabilistic Models for Learning                                                 51
  3.1 Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . .               52
  3.2 The Exponential Family . . . . . . . . . . . . . . . . . . .                  53
  3.3 Frequentist Learning . . . . . . . . . . . . . . . . . . . . .                59