Programming, Numerics and
Optimization
Spring 2018, 2010;
Autumn 20132016,
2011, 20052008 (course
flyer [pdf])
Course schedule
Tuesday 10:00 a.m., Room S4 (4th floor), IPPT
PAN
 (A1) Preliminaries, programming basics I (Mar., 6th)
 (A2) Programming basics II (Mar., 13th)
 (A3) Programming basics III (Mar., 20th)
 (B1) Basics of numerical computations (Mar., 27th)
 (B2) Numerical integration of ordinary differential
equations (Apr., 3rd)
 (B3) Linear systems I: Direct and iterative
methods (Apr., 10th)
 (A4) Objectoriented programming (Apr., 17th)
 (B4) Linear systems II (Apr., 24th)
 (B5) Linear integral equations (May, 8th)
 (C1) Basics of optimization, sensitivity analysis
(May,
15th)
 (C2) Unconstrained optimization I (May, 22nd)
 (C3) Unconstrained optimization II (Jun, 5th)
 (C4) Constrained optimization (Jun, 5th)
 (C5) Heuristic methods (Jun, 12th)
 (C6) Structural reanalysis in statics (Jun, 19th, 9:15 am)
Lecture notes (Spring 2018)
A) Programming
 Basics
I [pdf]
 Course preliminaries
 Programming paradigms
 C/C++ basics
 Basics
II [pdf]
 Types and variables
 (Few important) operators
 Control flow statements
 Functions
 Basics
III [pdf]
 Pointers
 Arrays
 Data structures
 Command line arguments
 Objectoriented
programming [pdf]
 The idea
 Objects and classes
 Creating and destroying objects
 Overloaded operators
 STL vector class
B)
Numerics
 Basics
of numerical computations [pdf]
 Number representations
 Arithmetic errors
 Problems and algorithms
 Conditioning
 Algorithm stability
 Numerical
integration of ordinary differential equations [pdf]
 Explicit onestep methods (Euler, RungeKutta,
convergence, order and stability of RK, Richardson extrapolation,
adaptive stepsize control)
 Implicit onestep methods (modified midpoint,
predictorcorrector, AdamsBashforthMoulton)
 Multistep methods
 Methods for 2nd order equations
 Linear
systems I: Direct and iterative methods [pdf]
 Basics (basic notions, types of problems,
methods of solution)
 Existence and uniqueness of solution
 Direct methods (special matrices,
factorizations
and decompositions, Gaussian elimination)
 Iterative methods (stationary methods, Krylov
subspace methods)
 Linear
systems II [pdf]
 Leastsquares problems (normal equations,
GaussMarkov theorem, SVDbased and iterative methods)
 Conditioning (vector and matrix norms,
conditioning and estimation
of accuracy, deconvolution as a common source of illconditioning)
 Regularization (for direct
methods, for
iterative methods, regularization parameter)
 Large Toeplitz
systems
additional
files (.avi)
 Linear
integral equations [pdf]
 Classification
 Integral operators
 Integral equations of the first kind
 Integral equations of the second kind
 Selected solution methods
C) Optimization
 Basics
of
optimization (in structural
engineering), sensitivity analysis [pdf]
 Optimization problems
 Objective function, variables, constraints
 Sensitivity analysis (Finite Difference
Approximations, Direct
Differentiation Method, Adjoint Method, Automatic Differentiation)
 Unconstrained
optimization I [pdf]
 Basics (stop conditions, rate/order of
convergence)
 Line search vs. trust region methods
 Line search  search directions (steepest
descent, conjugate direction, Newton direction)
 Line search  step size (Wolfe, strong Wolfe,
Goldstein and Price conditions, Zoutendijk condition)
 Univariate optimization
 Unconstrained
optimization II [pdf]
 Zero order methods (coordinate descent,
Powell's
direction set, Rosenbrock method)
 Steepest descent (simplified
version: number of iterations and attraction basins of the
minima)
 Conjugate gradient methods (conjugate
directions, linear and nonlinear conjugate gradient)
 Newton methods (inexact and modified Newton
methods)
 QuasiNewton methods (Hessian approximations,
DFP
and BFGS, Broyden class and SR1)
 Leastsquares problems (GaussNewton and
LevenbergMarquardt methods)
additional
files (.avi)
 No of iterations in dependence on step length: BW
(12.6 MB), CMYK
(13.6 MB)
 No of iterations, zoom: BW
(19 MB), CMYK
(18.6 MB)
 Attraction basins of minima: blueyellow
(8.6 MB)
 Constrained
optimization [pdf]
 Basics
 Handling constraints (Lagrangian and KKT
conditions, penalty function)
 Types of problems
 Linear programming (simplex method, interior
point methods)
 Heuristic
methods [pdf]
 Coupled local minimizers
 NelderMead method
 Simulated annealing
 Evolutionary (genetic) algorithms
 Swarm intelligence (particle swarm, ant colony)
 Artificial neural networks
Grading and homeworks
Grading
[pdf]
Please feel free to contact me, if you have any questions.
 HW1 [pdf]:
First steps
 HW2 [pdf]:
Loops and functions
 HW3
[pdf]:
Arrays and pointers
 HW4
[pdf]:
Basics of numerics
 HW5 [zip], readme
[pdf]:
Numerical integration of ODEs
 HW6
[zip], readme [pdf]:
Objects
 HW7
[zip], readme
[pdf]: LU decomposition
 HW8
[pdf]: Regularization and iterative linear solvers
 HW9
[pdf]: Linear programming
 ...
Lecture notes (Autumn 2016)
C) Optimization
 Structural
reanalysis in statics [pdf]
 Basics (sensitivity analysis vs. reanalysis,
reanalysis and SHM, formulation)
 Combined approximations (CA)
 Virtual distortion method (VDM)
 VDM  sensitivity analysis
Grading and homeworks (Autumn 2016)
 HW10
[pdf]: Optimization
Earlier files

