Programming, Numerics and Optimization
Spring 2021 (course flyer [pdf])
Course schedule
Tuesday 9:30 a.m., Room S4 (4th floor), currently online, IPPT PAN
 (A1) Preliminaries, programming basics I (Mar., 2nd)
 (A2) Programming basics II (Mar., 9th)
 (A3) Programming basics III (Mar., 16th)
 (B1) Basics of numerical computations (Mar., 23rd)
 (B2) Numerical integration of ODEs (Mar., 30th)
 (A4) Objectoriented programming (Apr., 6th)
 (B3) Linear systems I (Apr., 13th)
 (B4) Linear systems II (Apr., 20th)
 (C1) Basics of optimization (Apr., 27th)
 (C2) Unconstrained optimization I (May, 4th)
 (C3) Unconstrained optimization I (May, 11th)
 (C4) Constrained optimization (May, 18th)
 (C6) Structural reanalysis in statics (Jun., 1st)
 (C5) Heuristic methods (Jun., 8th)
 (B5) Linear integral equations (Jun., 15th)
Grading and homeworks
Grading [pdf]
Please feel free to contact me, if you have any questions.
 HW1 [pdf]: First steps
 HW2 [pdf]: Loops and functions
 HW3 [pdf]: Arrays and pointers
 HW4 [pdf]: Basics of numerics
 HW5 [zip], readme [pdf]: Numerical integration of ODEs
 HW6 [zip], readme [pdf]: Objects
 HW7 [zip], readme [pdf]: LU decomposition
 HW8 [pdf]: Regularization and iterative linear solvers
 HW9 [pdf]: Linear programming
 HW10 [pdf]: Optimization
Lecture notes
A) Programming
 Basics I [pdf]
 Course preliminaries
 Programming paradigms
 C/C++ basics
 Basics II [pdf]
 Types and variables
 (Few important) operators
 Control flow statements
 Functions
 Basics III [pdf]
 Pointers
 Arrays
 Data structures
 Command line arguments
 Objectoriented programming [pdf]
 The idea
 Objects and classes
 Creating and destroying objects
 Overloaded operators
 STL vector class
B) Numerics
 Basics of numerical computations [pdf]
 Number representations
 Arithmetic errors
 Problems and algorithms
 Conditioning
 Algorithm stability
 Numerical integration of ordinary differential equations [pdf]
 Explicit onestep methods (Euler, RungeKutta, convergence, order and stability of RK, Richardson extrapolation, adaptive stepsize control)
 Implicit onestep methods (modified midpoint, predictorcorrector, AdamsBashforthMoulton)
 Multistep methods
 Methods for 2nd order equations
 Linear systems I: Direct and iterative methods [pdf]
 Basics (basic notions, types of problems, methods of solution)
 Existence and uniqueness of solution
 Direct methods (special matrices, factorizations and decompositions, Gaussian elimination)
 Iterative methods (stationary methods, Krylov subspace methods)
 Linear systems II [pdf]
 Leastsquares problems (normal equations, GaussMarkov theorem, SVDbased and iterative methods)
 Conditioning (vector and matrix norms, conditioning and estimation of accuracy, deconvolution as a common source of illconditioning)
 Regularization (for direct methods, for iterative methods, regularization parameter)
 Large Toeplitz systems
additional files (.avi)
 Linear integral equations [pdf]
 Classification
 Integral operators
 Integral equations of the first kind
 Integral equations of the second kind
 Selected solution methods
C) Optimization
 Basics of optimization (in structural engineering), sensitivity analysis [pdf]
 Optimization problems
 Objective function, variables, constraints
 Sensitivity analysis (Finite Difference Approximations, Direct Differentiation Method, Adjoint Method, Automatic Differentiation)
 Unconstrained optimization I [pdf]
 Basics (stop conditions, rate/order of convergence)
 Line search vs. trust region methods
 Line search  search directions (steepest descent, conjugate direction, Newton direction)
 Line search  step size (Wolfe, strong Wolfe, Goldstein and Price conditions, Zoutendijk condition)
 Univariate optimization
 Unconstrained optimization II [pdf]
 Zero order methods (coordinate descent, Powell's direction set, Rosenbrock method)
 Steepest descent (simplified version: number of iterations and attraction basins of the minima)
 Conjugate gradient methods (conjugate directions, linear and nonlinear conjugate gradient)
 Newton methods (inexact and modified Newton methods)
 QuasiNewton methods (Hessian approximations, DFP and BFGS, Broyden class and SR1)
 Leastsquares problems (GaussNewton and LevenbergMarquardt methods)
additional files (.avi)
 No of iterations in dependence on step length: BW (12.6 MB), CMYK (13.6 MB)
 No of iterations, zoom: BW (19 MB), CMYK (18.6 MB)
 Attraction basins of minima: blueyellow (8.6 MB)
 Constrained optimization [pdf]
 Basics
 Handling constraints (Lagrangian and KKT conditions, penalty function)
 Types of problems
 Linear programming (simplex method, interior point methods)
 Heuristic methods [pdf]
 Coupled local minimizers
 NelderMead method
 Simulated annealing
 Evolutionary (genetic) algorithms
 Swarm intelligence (particle swarm, ant colony)
 Artificial neural networks
 Structural reanalysis in statics [pdf]
 Basics (sensitivity analysis vs. reanalysis, reanalysis and SHM, formulation)
 Combined approximations (CA)
 Virtual distortion method (VDM)
 VDM  sensitivity analysis

