The quasi-Newton least squares method: A new and fast secant method analyzed for linear systems

We present a new quasi-Newton method that can solve systems of equations of which no information is known explicitly and which requires no special structure of the system matrix, like positive definiteness or sparseness. The method builds an approximate Jacobian based on input-output combinations of a black box system, uses a rank-one update of this Jacobian after each iteration, and satisfies the secant equation. While it has originally been developed for nonlinear equations we analyze its properties and performance when applied to linear systems. Analytically, the method is shown to be convergent in n + 1 iterations (n being the number of unknowns), irrespective of the nature of the system matrix. The performance of this method is greatly superior to other quasi-Newton methods and comparable with GMRes when tested on a number of standardized test-cases.