This third book in a suite of four practical guides is an engineer's companion to using numerical methods for the solution of complex mathematical problems. The required software is provided by way of the freeware mathematical library BzzMath that is developed and maintained by the authors. The present volume focuses on optimization and nonlinear systems solution. The book describes numerical methods, innovative techniques and strategies that are all implemented in a well-established, freeware library. Each of these handy guides enables the reader to use and implement standard numerical tools for their work, explaining the theory behind the various functions and problem solvers, and showcasing applications in diverse scientific and engineering fields. Numerous examples, sample codes, programs and applications are proposed and discussed. The book teaches engineers and scientists how to use the latest and most powerful numerical methods for their daily work.

Table of Contents:
Preface

FUNCTION ROOT-FINDING
Introduction
Substitution Algorithms
Bolzano's Algorithm
Function Approximation
Use of a Multiprocessor Machine with a Known Interval of Uncertainty
Search for an Interval of Uncertainty
Stop Criteria
Classes for Function Root-Finding
Case Studies
Tests for BzzFunctionRoot and BzzFunctionRootMP Classes
Some Caveats

ONE-DIMENSIONAL OPTIMIZATION
Introduction
Measuring the Efficiency of the Search for the Minimum
Comparison Methods
Parabolic Interpolation
Cubic Interpolation
Gradient-Based Methods
Combination of Algorithms in a General Program
Parallel Computations
Search for the Interval of Uncertainty
Stop Criteria
Classes for One-Dimensional Minimization
Case Studies
Tests

UNCONSTRAINED OPTIMIZATION
Introduction
Heuristic Methods
Gradient-Based Methods
Conjugate Direction Methods
Newton's Method
Modified Newton Methods
Quasi-Newton Methods
Narrow Valley Effect
Stop Criteria
BzzMath Classes for Unconstrained Multidimensional Minimization
Case Study
Tests

LARGE-SCALE UNCONSTRAINED OPTIMIZATION
Introduction
Collecting a Sparse Symmetric Matrix
Ordering the Hessian Rows and Columns
Quadratic Functions
Hessian Evaluation
Newton's Method
Inexact Newton Methods
Practical Preconditioners
openMP Parallelization
Class for Large-Scale Unconstrained Minimization

ROBUST UNCONSTRAINED MINIMIZATION
Introduction
One-Dimensional Minimization
Classes for One-Dimensional Robust Minimization
Examples in One-Dimensional Space
Examples in Multidimensional Space
Two-Dimensional Space
Classes for Robust Two-Dimensional Minimization
Examples for BzzMinimizationTwoVeryRobust Class
Multidimensional Robust Minimization
Class for Robust Multidimensional Minimization

ROBUST FUNCTION ROOT-FINDING
Introduction
Class and Examples

NONLINEAR SYSTEMS
Introduction
Comparing Nonlinear Systems to Other Iterative Problems
Convergence Test
Substitution Methods
Minimization Methods
Jacobian Evaluation
Newton's Method
Gauss-Newton Method
Modified Newton Methods
Newton's Method and Parallel Computations
Quasi-Newton Methods
Quasi-Newton Methods and Parallel Computing
Stop Criteria
Classes for Nonlinear System Solution with Dense Matrices
Tests for the BzzNonLinearSystem Class
Sparse and Large-Scale Systems
Large Linear System Solution with Iterative Methods
Classes for Nonlinear System Solution with Sparse Matrices
Continuation Methods
Solution of Certain Equations with Respect to Certain Variables
Case Studies
Special Cases
Some Caveats

UNDERDIMENSIONED NONLINEAR SYSTEMS
Introduction
Underdimensioned Linear Systems
Class for Underdimensioned Nonlinear System Solution

CONSTRAINED MINIMIZATION
Introduction
Equality Constraints
Equality and Inequality Constraints
Lagrangian Dual Problem

LINEAR PROGRAMMING
Introduction
Basic Attic Method Concepts
Attic Method
Differences between the Attic Method and Traditional Approaches
Explosion in the Number of Iterations
Degeneracy
Duality
General Considerations

QUADRATIC PROGRAMMING
Introduction
KKT Conditions for a QP Problem
Equality-Constrained QP
Equality- and Inequality-Constrained Problems
Class for QP
Projection or Reduced Direction Search Methods for Bound-Constrained Problems
Equality, Inequality, and Bound Constraints
Tests

CONSTRAINED MINIMIZATION: PENALTY AND BARRIER FUNCTIONS
Introduction
Penalty Function Methods
Barrier Function Methods
Mixed Penalty-Barrier Function Methods

CONSTRAINED MINIMIZATION: ACTIVE SET METHODS
Introduction
Class for Constrained Minimization
Successive Linear Programming
Projection Methods
Reduced Direction Search Methods
Projection or Reduced Direction Search Methods for Bound-Constrained Problems
Successive Quadratic Programming or Projected Lagrangian Method
Narrow Valley Effect
The Nonlinear Constraints Effect
Tests

PARAMETRIC CONTINUATION IN OPTIMIZATION AND PROCESS CONTROL
Introduction
Algebraic Constraints