Nonlinear Programming Notes
A consolidated set of notes on nonlinear programming, including convex sets, convex functions, unconstrained optimization, constrained optimization, Lagrangian formulation, KKT conditions, and convex duality.
Academic Notes · Optimization · Nonlinear Methods
Nonlinear Programming
Notes on convexity, unconstrained and constrained optimization, Lagrangian duality, KKT theory, optimality conditions, and algorithms for nonlinear optimization.
Nonlinear Programming
Notes and drafts.
This section collects notes on the mathematical and algorithmic foundations of nonlinear programming. The focus is on convex sets, convex functions, optimality conditions, Lagrangian methods, KKT theory, duality, and basic iterative methods for nonlinear optimization.
Convex Sets and Convex Functions
Notes on affine sets, convex sets, cones, epigraphs, level sets, Jensen's inequality, composition rules, and Hessian-based convexity tests.
Unconstrained Optimization
Notes on first-order and second-order necessary and sufficient conditions, stationary points, Hessian tests, and local versus global optimality.
KKT Conditions and Constraint Qualifications
Notes on stationarity, primal feasibility, dual feasibility, complementary slackness, active constraints, and constraint qualification assumptions.
Lagrangian Duality
Notes on the Lagrangian function, dual function, weak duality, strong duality, saddle points, and duality gaps in nonlinear programming.
Algorithms for Nonlinear Optimization
Notes on gradient descent, Newton's method, line search, projected gradient methods, and basic convergence arguments.