MS-E2122: Nonlinear optimization
| Course name | Nonlinear optimization |
|---|---|
| Course code | MS-E2122 |
| Abbreviation | “NL, Nonlinear, Nonlinis” |
| Period | I-II |
| Lecturer | Fernando Dias |
Description
Goes through easy cases of nonlinear optimization. Topics start from general optimization ideas and proofs of convexity. These ideas are utilized to provide different conditions and algorithms on specific kinds of nonlinear problems with a priority on feasibility in the sense of whether a problem can be solved in a reasonable / straightforward manner. The course is completed significantly with assignments which are a mixture of optimization theory and algorithm implementation through programming.
Course material
Official material
Lecture notes and compilations of notes. Lecture slides. Notes/slides are of good quality but often lack examples. A notable issue with multiple sources is the different and conflicting numbering systems between the sources.
Extra material
Nonlinear Programming, Theory and Algorithms by Bazaraa, Sherali and Shetty is given as an official additional material.
Contents and workload
Workload depends on coding knowledge and prerequisites. Overall workload quite average for a Master’s level course.
Previous course iterations have included a notable amount of coding implementations which can be hard to approach as these kinds of algorithms have not been considered fully in prior courses or in the exercises within the course. Sometimes the implementations feel like hocus pocus but they just work in the end.
Workload is relatively low until dual problems are introduced. A lot of the methods in the first half are discussed in other courses (MS-C2105, MS-C2107, Calculus/Linear algebra courses etc.). Dual problems are usually kinda hard to approach in initial courses and thus they are likely the first notably tougher topics during the course. The latter half is not too bad but requires notably more work / thought put into the course topics than during the first half.
Overall workload
Weekly contents
| Week | Topic | Good to know / description |
|---|---|---|
| 1 | Problem definitions, convexity, hyperplanes | Basic building blocks, included in other courses |
| 2 | Convexity and theorems around it | Convexity is important = We spend two weeks on it |
| 3 | Optimality conditions | Considered in MS-C2105 or similar |
| 4 | Gradient methods | Considered quickly in many courses like MS-C2107 |
| 5 | Newton methods | You know the idea, algorithms are implemented |
| 6 | KKT Conditions | Likely considered in courses like MS-C2105 |
| 7 | Lagrangian Dual Problems | Methods which utilize dual problems |
| 8 | Penalty Methods | Switching inequalities to penalties in objective |
| 9 | Barrier Methods | Builds on the ideas from week 8 |
| 10 | Feasible direction methods | Improvements on gradient/Newton methods |
| 11 | Spatial Branching-and-Bound | Breaking nonconvex sets to convex cases |
| 12 | Conclusion, may be a guest lecture | - |
Practicalities
Course has 4 assignment sheets and a course exam. Nothing is necessarily mandatory for completion but the above description is the only way to complete the course currently.
Related courses
Official prerequisites
Recommended prerequisites:
- MS-A00XX Matrix Algebra
- MS-A01XX Differential and integral calculus 1
- MS-A02XX Differential and integral calculus 2
- MS-C2105 Introduction to Optimization
Additional prerequisites
- Some course which has extensive matrix equation solving can be useful (there can be couple miserable algebra exercises).
- Some course which utilizes Python or Julia (/MATLAB) to ease the burden on coding. It is unsure which language the course is going to use in the future.
- MS-C2107 has some similar algorithm implementations with MATLAB (Gradient methods on one week, MATLAB is close enough to Julia).
More like this
- MS-E2121: Linear Optimization - Has included similar algorithm design but the topics are different
- MS-E2145: Combinatorial Optimization - Another topic under nonlinear optimization but a very different direction from this course as it considers optimization in graph problems and complexity theory
- MS-E2177: Seminar on Case Studies in Operations Research - Can include case studies which have nonlinear optimization principles but often are a lot more surface-level than what this course is
Trivia
- Has utilized grading TA:s which are unknown to the course attendees.
- Fernando is a great character when lecturing, definitely would recommend attending lectures at least once if you are not usually attending them.