Machine Learning and Optimization
JeanFrancoisPuget 2700028FGP Visits (4451)
The view that Machine Learning is tightly linked to optimization is becoming common knowledge. There are training courses ( Stephen Boyd's and Steve Wright's ) mixing the two. The Data Smart book by John Foreman describes some key machine learning techniques as optimization problems using Microsoft Excel solver. This book is a great read by the way. But what made me write this blog entry is that I was quite delighted when I read the following from John Mount
My opinion is the best machine learning work is an attempt to re-phrase prediction as an optimization problem (see for example: Bennett, K. P., & Parrado-Hernandez, E. (2006). The Interplay of Optimization and Machine Learning Research. Journal of Machine Learning Research, 7, 1265–1281). Good machine learning papers use good optimization techniques and bad machine learning papers (most of them in fact) use bad out of date ad-hoc optimization techniques.
I could not have said it better.
One thing we did in the past was to use CPLEX quadratic solver to compute supp
Back to the general topic, the convergence of machine learning and optimization means a lot to me. Indeed I first got a PhD in Machine Learning, then switched to working on optimization. I might be back at doing some machine learning work soon as a matter of fact.
Update on Sept 23. Added a link to Steve Wright tutorial.