DAMSL-201 Optimization

Syllabus

• First and second derivatives of a function of several real variables. 
• Extrema of functions: Lagrange multipliers.
• Extrema of functions: consideration of the second derivatives 
• Extrema of functions: convex functions (general definition without differentiation, characterisation of convex functions, local and global minima of convex functions, quadratic functions and positive definite matrices).
• Newton’s method 
• General results on nonlinear optimisation problems 
• Relaxation and gradient methods for unconstrained problems
• Conjugate gradient methods for unconstrained problems
• Relaxation, gradient and penalty-function methods for constrained problems 
• The Kuhn-Tucker conditions  
• Lagrangians and saddle points. Introduction to duality  

Learning Outcomes

The course aims to provide students with the theoretical knowledge and practical skills necessary to formulate and solve optimisation problems in various computer science contexts. The objectives of the course are 

• familiarity with the formulation of optimisation problems 
• the necessary background in mathematical analysis techniques 
• an understanding of the practical application of the techniques to real systems.

Students’ familiarity with these topics will be reinforced through the application of established methods in the context of exercises, as well as the preparation of a project aimed at familiarising and presenting textual and experimental analysis techniques.