Machine Learning
Maestría en Ingeniería - Ingeniería de Sistemas y Computación

Departamento de Ingeniería de Sistemas e Industrial
Universidad Nacional de Colombia
Ing. Fabio A. González O., Ph.D.
Of. 114, Edif. Nuevo de Ingeniería


Course Description


The main goal of Machine Learning (ML) is the development of systems that are able to autonomously change their behavior based on experience. ML offers some of the more effective techniques for knowledge discovery in large data sets. ML has played a fundamental role in areas such as bioinformatics, information retrieval, business intelligence and autonomous vehicle development.

The main goal of this course is to study the computational, mathematical and statistical foundations of ML, which are essential for the theoretical analysis of existing learning algorithms, the development of new algorithms and the well-founded application of ML to solve real-world problems.



Topic Material Assignments Presentations
1. Introduction
Brief Introduction to ML (notes)
[Mit97] Cap 1
[Alp04] Cap 1,2
[DHS00] A.1, A.2
Assignment 1
The great robot race

Introduction to ML (
2. Bayesian decision theory
2.1 A review of probability theory
2.2 Classification
2.3 Lost and risk
2.4 Naive Bayes classifier
2.5 Bayesian Networks
2.6 Maximum likelihood estimation
2.7 Bayesian estimation
2.8 Parametric Classification
2.9 Expectation Maximization
Chap 3 (notes)
Chap 4 (notes)
Chap 7 (Sect. 7.4) (EM notes)
(Parametric notes)
[DHS00] Chap 3
Assignment 2
Assignment 3
Graphical models (notes)
Paper presentation:
[Friedman99] Alejandro Riveros 
[Goldenberg05] Camilo López 
[Dietterich02] Jeison Gutierres 
3. Kernel methods
3.1 The kernel trick
3.2 Kernel ridge regression
3.3 Kernel functions
3.4 Other kernel Algorithms
3.5 Kernels in complex structured data
[SC04] Chap 2
Introd. to kernel methods
Matlab examples (.pdf,.m)
Support Vector Classification (part 4)
Paper presentation
[Grauman05] María E. Rojas
[Lodhi02] David Becerra
[Leslie02] Daniel Restrepo
4. Regularization and model complexity
4.1 Bias vs variance tradeoff
4.2 Risk and empirical risk
4.3 Complexity measures
Chap 4 (Sect. 4.3, 4.7, 4.8)
(bias/var notes)
Assignment 4 Video:
Iterative Regularization Scheme and Early Stopping in Learning from Examples
Paper presentation:
[Roberts00] José Luis Morales
[Lawrence96] Javier Vargas
[Mehta95] Alexander Cerón
5. Performance evaluation
5.1 Performance evaluation in supervised learning
5.2 Performance evaluation in unsupervised learning
5.3 Hypthesis testing

[Alp04] Cap 14
[TSK05] Chap 8 (Sect. 8.5)
Assignment 5 Video:
An Empirical Comparison of Learning Methods and Metrics
Paper presentation:
[Hand01] Carlos Garzón
[Domingos99] Jimmy Cifuentes
[Japkowicz02] Omar Erazo
6. Combining multiple classifiers
6.1 Voting
6.2 Error correcting codes
6.3 Bagging
6.4 Boosting
[Alp04] Cap 15 [Dietterich00]  Sandra Tocarruncho
[Oza05] Emir Cortés
[Mason00] Edwin Niño
7. Clustering and density estimation
Miguel Dussan: Web Clustering
Rolando Beltrán: Bi-Clustering
Wilson Soto: Sequence Clustering
Final Exam (19/6/08)
Project (08/7/08)



Additional references