Lecture Outlines (Optimization Theory and its Applications)

Bahman Gharesifard
Fall 2016


  • Lecture 1: (From Heron of Alexandria, Dido, and Tartaglia's problems to Max-flow)
  • Lecture 2: (Existence of an optimizer: Weierstrass theorem, and relaxing compactness ala coercivity)
  • Lecture 3: (First-order necessary conditions of optimality: unconstrained case)
  • Lecture 4: (Second-order necessary conditions of optimality, and spectral decomposition of symmetric matrices)
  • Lecture 5: (Constrained optimization: scenarios with infeasible directions)
  • Lecture 6: (Optimization constrained on lower dimensional surfaces: variations using curves)
  • Lecture 7: (Affine sets, hyperplanes, and convex sets)
  • Lecture 8: (Geometry of convex set and cones, Jensen's inequality, operations on convex sets and Minkowski sums)
  • Lecture 9: (Convex functions, quasiconvexity, strict convexity, and epigraphs)
  • Lecture 10: (First and second-order conditions for convexity, and global feature of minimizers of convex functions)
  • Lecture 11: (Examples: quadratic functions and log-determinant function on the space of positive definite symmetric matrices)
  • Lecture 12: (Variational inequality I)
  • Lecture 13: (Variational inequality II, examples)
  • Lecture 14: (Projection maps)
  • Lecture 15: (Separation: Support hyperplane theorem)
  • Lecture 16: (Separation of convex sets, strict and proper separation)
  • Lecture 17: (A glimpse at duality using a separation theorem: Dubovitskii-Milytuin theorem)
  • Lecture 18: (Euler-Lagrange equation, Lagrangian mechanics, and Legendre transformation to Hamiltonian mechanics)
  • Lecture 19: (Fenchel duality theorem)
  • Lecture 20: (Applications of Fenchel duality theorem to resource allocation)
  • Lecture 21: (von Neumann's minmax theorem through Fenchel duality)
  • Lecture 22: (Saddle-point theorem, Lagrangian duality, and primal-dual problems)
  • Lecture 23: (Nonlinear programming: Karush-Kuhn-Tcuker necessary conditions of optimality in the presence of inequalities)
  • Lecture 24: (Nonlinear programming through duality)
  • Lecture 25: (Linear programming and duality)
  • Lecture 26: (Overview of some numerical methods for optimization)
  • Lecture 27: (Convergence of discrete-time gradient flow dynamics with constant step size)
  • Lecture 28: (Convergence of discrete-time gradient flow dynamics with diminishing step size)
  • Lecture 29: (Convergence rates: linear, sublinear, superlinear, and quadratic)
  • Lecture 30: (Newton's method, and its convergence properties: applications to least squares problem)
  • Lecture 31: (Saddle-point dynamics)
  • Lecture 32: (Project Presentations)
    • (Alternating Direction Method of Multipliers and regularization)
    • (Linear Matrix Inequalities in systems and control)
  • Lecture 33: (Project Presentations)
    • (Fictitious play learning in zero-sum games)
    • (Potential games)
  • Lecture 34: (Project Presentations)
    • (Sum Of Squares convexity))

For questions, contact me with bahman at mast.queensu.ca or with 613-533-2441 (I prefer emails)