Machine Learning - Appunti completi

MACHINE LEARNING 2018/2019DETAILED PROGRAMProf. F. VandinLast Update: January 20^th, 2019

This document describes, for each topic, the level of detail required for the material presented during the lectures (see the slides). The level of detail relates to what has been presented during the lectures, so all details means that all details presented during the lectures (see the slides) is required while main idea means that only the understanding of the concepts presented is required, while the details (e.g., details of propositions, proofs, formulas) are not required. Note that the background material (probability, linear algebra) is assumed to be known at the level of detail used during the presentation of the topics below. For some propositions the corresponding proposition in the book is used for clarity.

1. Learning Model
   • All details presented in class required, including: definitions, propositions' statements, proof of Corollary 2.3 [UML], definition of agnostic PAC learnability for general loss functions
2. Uniform Convergence
   • Lemma 4.2 [UML]: statement and proof with all details
   • Definition of uniform convergence property: main idea (details of definition not required)
   • Corollary 4.6 [UML]: only main idea and bound on the number of samples
3. Basics of Statistics
   • definition confidence interval, definition rejection region: details
   • hypothesis testing: main idea; hypothesis testing rejection rule in detail
   • everything else: main idea
4. Linear Models
   • linear predictors/models: definitions with all details
   • linear classification, perceptron: definitions and algorithm in detail
   • proposition on perceptron convergence: only main idea (as in slide "Perceptron: Notes")
   • linear regression: definitions, matrix form, derivation best predictor, use of generalized inverse in detail (derivation generalized inverse: not required)
   • logistic regression: definition, loss function, equivalence MLE solution and ERM solution in detail
5. Bias-Complexity
   • No Free Lunch (NFL) theorem, NFL and priori knowledge: only main idea
   • approximation error + estimation error, complexity and error decomposition: all details

MACHINE LEARNING 2018/2019

DETAILED PROGRAM

Prof. F. VandinLast Update: January 20^th, 2019

This document describes, for each topic, the level of detail required for the material presented during the lectures (see the slides). The level of detail relates to what has been presented during the lectures, so all details means that all details presented during the lectures (see the slides) is required while main idea means that only the understanding of the concepts presented is required, while the details (e.g., details of propositions, proofs, formulas) are not required. Note that the background material (probability, linear algebra) is assumed to be known at the level of detail used during the presentation of the topics below. For some propositions the corresponding proposition in the book is used for clarity.

1. Learning Model
   • All details presented in class required, including: definitions, propositions' statements, proof of Corollary 2.3 [UML], definition of agnostic PAC learnability for general loss functions.
2. Uniform Convergence
   • Lemma 4.2 [UML]: statement and proof with all details.
   • Definition of uniform convergence property: main idea (details of definition not required).
   • Corollary 4.6 [UML]: only main idea and bound on the number of samples.
3. Basics of Statistics
   • definition confidence interval, definition rejection region: details.
   • hypothesis testing: main idea; hypothesis testing rejection rule in detail.
   • everything else: main idea.
4. Linear Models
   • linear predictors/models: definitions with all details.
   • linear classification, perceptron: definitions and algorithm in detail.
   • proposition on perceptron convergence: only main idea (as in slide “Perceptron: Notes”).
   • linear regression: definitions, matrix form, derivation best predictor, use of generalized inverse in detail (derivation generalized inverse: not required).
   • logistic regression: definition, loss function, equivalence MLE solution and ERM solution in detail.
5. Bias-Complexity
   • No Free Lunch (NFL) theorem, NFL and priori knowledge: only main idea.
   • approximation error + estimation error, complexity and error decomposition: all details.

6. VC-dimension

• Restrictions, shattering, VC-dimension: definitions in detail
• Fundamental Theorems