Shortly after the end of World War II high-speed digital computing machines were being developed. It was clear that the mathematical aspects of com­ putation needed to be reexamined in order to make efficient use of high-speed digital computers for mathematical computations. Accordingly, under the leadership of Min a Rees, John Curtiss, and others, an Institute for Numerical Analysis was set up at the University of California at Los Angeles under the sponsorship of the National Bureau of Standards. A similar institute was formed at the National Bureau of Standards in Washington, D. C. In 1949 J. Barkeley Rosser became Director of the group at UCLA for a period of two years. During this period we organized a seminar on the study of solu­ tions of simultaneous linear equations and on the determination of eigen­ values. G. Forsythe, W. Karush, C. Lanczos, T. Motzkin, L. J. Paige, and others attended this seminar. We discovered, for example, that even Gaus­ sian elimination was not well understood from a machine point of view and that no effective machine oriented elimination algorithm had been developed. During this period Lanczos developed his three-term relationship and I had the good fortune of suggesting the method of conjugate gradients. We dis­ covered afterward that the basic ideas underlying the two procedures are essentially the same. The concept of conjugacy was not new to me. In a joint paper with G. D.

Table of Contents:
I Newton’s Method and the Gradient Method.- 1 Introduction.- 2 Fundamental Concepts.- 3 Iterative Methods for Solving g(x) = 0.- 4 Convergence Theorems.- 5 Minimization of Functions by Newton’s Method.- 6 Gradient Methods—The Quadratic Case.- 7 General Descent Methods.- 8 Iterative Methods for Solving Linear Equations.- 9 Constrained Minima.- II Conjugate Direction Methods.- 1 Introduction.- 2 Quadratic Functions on En.- 3 Basic Properties of Quadratic Functions.- 4 Minimization of a Quadratic Function F on k-Planes.- 5 Method of Conjugate Directions (CD-Method).- 6 Method of Conjugate Gradients (CG-Algorithm).- 7 Gradient PARTAN.- 8 CG-Algorithms for Nonquadratic Functions.- 9 Numerical Examples.- 10 Least Square Solutions.- III Conjugate Gram-Schmidt Processes.- 1 Introduction.- 2 A Conjugate Gram-Schmidt Process.- 3 CGS-CG-Algorithms.- 4 A Connection of CGS-Algorithms with Gaussian Elimination.- 5 Method of Parallel Displacements.- 6 Methods of Parallel Planes (PARP).- 7 Modifications of Parallel Displacements Algorithms.- 8 CGS-Algorithms for Nonquadratic Functions.- 9 CGS-CG-Routines for Nonquadratic Functions.- 10 Gauss-Seidel CGS-Routines.- 11 The Case of Nonnegative Components.- 12 General Linear Inequality Constraints.- IV Conjugate Gradient Algorithms.- 1 Introduction.- 2 Conjugate Gradient Algorithms.- 3 The Normalized CG-Algorithm.- 4 Termination.- 5 Clustered Eigenvalues.- 6 Nonnegative Hessians.- 7 A Planar CG-Algorithm.- 8 Justification of the Planar CG-Algorithm.- 9 Modifications of the CG-Algorithm.- 10 Two Examples.- 11 Connections between Generalized CG-Algorithms and Stadard CG- and CD-Algorithm.- 12 Least Square Solutions.- 13 Variable Metric Algorithms.- 14 A Planar CG-Algorithm for Nonquadratic Functions.- References.