Class:	TTh 2-3:45, Baskin Engineering ???
Office hours:	Mo,We 10-11, 331 Baskin Engineering
Prerequisite:	CMPS 242 - Machine Learning

Inportant web pages

The web page for the Computation Learning Theory conference COLT
Web page on Kernal Machines

Summary of lectures

1:	General introduction
	Tricks of the trade
	- Mixing HMMs
	- Vicinity lemma for HMMs
	- Exponential mixing
	- Combining HMMs in a better way
	- Bush-derby

2:	How to sum infinitely many paths in an HMM	
	- Its all matrix algebra
	- Its the all-pairs shortest path alg.
	- Its like the conversion of an NFA to a reg. expr.
	- Open problem: Covering algorithm beats boosting

3:   	Boosting as entropy projection
	- Edge wrt updated distr. is constraint to be 0
	Application of boosting to microarray data (Jun Liao)
	InfoBoost: constraining the conditional edges to be 0
	- InfoBoost as good as covering (Kohei Hatano)

4:      Efficient Margin Maximizing with Boosting
	Using multiplicative algs for in-vitro selection

5:	Finish in-vitro selection 
	A paging algorithm based on the Randomized Weighted Majority Alg. 
	(Robert Gramacy)

6:	How to use expert-style algs for paging (Robert Gramacy)
	Relative loss bounds for a priori and a posteriori filtering
	Generalalization to Kalman filters: Introductory article
	Web site with lots of material on Kalman filters

7:	How engineers use Kalman Filters (Dirk Robinson)
	Bregman divergences - explicit and implicit updates
	Proof of filtering bounds Paper with proofs

8: 	Principal Component Analysis
	Proof of what it optimizes
	Multilayer auto-associative neural nets

9:      Application of PCA to face recognition summary paper (Robert Gramacy) 
	The exponential family and Bregman divergences

10:	A generalizaton of PCA to the exponential family paper 
	Mutual information (Kohei Hatano) summary
	Three apps:	InfoBoost (Kohei Hatano)
			Feature Selection (Jun Liao)

11:	The information bottleneck method (Kohei Hatano) paper

12:  	More about the information bottle neck method
	Designing a caching strategy using the muliplicative kernels

13:	An introduction of MCMC sampling for Machine Learning paper

14:   	More on MCMC - simulated annealing - Gibbs sampling
	Classification with Bayesian methods

15:     Bush-derby for speech 
	- How, when and why does it work?

16:   	Deterministic annealing (Graham Grindley) paper notes

17:	The Minimum Relative Entropy Principle - Connection to Boosting 
	and how it is used to motivate most natural distributions
        Using the Information Bottleneck Method for clustering paper (Dima Kuzmin)

18:	Jordan Lecture on 
	A boosting method based on Linear Programming paper (Jun Liao)


19:	Finish boosting method based on LP notes (Jun Liao)
	Adapting Codes and Embeddings for Polychotomies paper poster (Gunnar Raetsch)
	Overview of class and discussion progress we made