This is octave.info, produced by makeinfo version 4.13 from octave.texi.

START-INFO-DIR-ENTRY
* Octave: (octave).           Interactive language for numerical computations.

END-INFO-DIR-ENTRY

   Copyright (C) 1996, 1997, 1999, 2000, 2001, 2002, 2005, 2006, 2007,
2011 John W. Eaton.

   Permission is granted to make and distribute verbatim copies of this
manual provided the copyright notice and this permission notice are
preserved on all copies.

   Permission is granted to copy and distribute modified versions of
this manual under the conditions for verbatim copying, provided that
the entire resulting derived work is distributed under the terms of a
permission notice identical to this one.

   Permission is granted to copy and distribute translations of this
manual into another language, under the above conditions for modified
versions.


File: octave.info,  Node: Functions of a Matrix,  Next: Specialized Solvers,  Prev: Matrix Factorizations,  Up: Linear Algebra

18.4 Functions of a Matrix
==========================

 -- Function File:  expm (A)
     Return the exponential of a matrix, defined as the infinite Taylor
     series

          expm(A) = I + A + A^2/2! + A^3/3! + ...

     The Taylor series is _not_ the way to compute the matrix
     exponential; see Moler and Van Loan, `Nineteen Dubious Ways to
     Compute the Exponential of a Matrix', SIAM Review, 1978.  This
     routine uses Ward's diagonal Pade' approximation method with three
     step preconditioning (SIAM Journal on Numerical Analysis, 1977).
     Diagonal Pade' approximations are rational polynomials of matrices

               -1
          D (A)   N (A)

     whose Taylor series matches the first `2q+1' terms of the Taylor
     series above; direct evaluation of the Taylor series (with the
     same preconditioning steps) may be desirable in lieu of the Pade'
     approximation when `Dq(A)' is ill-conditioned.

     *See also:* *note logm: doc-logm, *note sqrtm: doc-sqrtm.

 -- Function File: S = logm (A)
 -- Function File: S = logm (A, OPT_ITERS)
 -- Function File: [S, ITERS] = logm (...)
     Compute the matrix logarithm of the square matrix A.  The
     implementation utilizes a Pade' approximant and the identity

          logm(A) = 2^k * logm(A^(1 / 2^k))

     The optional argument OPT_ITERS is the maximum number of square
     roots to compute and defaults to 100.  The optional output ITERS
     is the number of square roots actually computed.

     *See also:* *note expm: doc-expm, *note sqrtm: doc-sqrtm.

 -- Loadable Function: S = sqrtm (A)
 -- Loadable Function: [S, ERROR_ESTIMATE] = sqrtm (A)
     Compute the matrix square root of the square matrix A.

     Ref: N.J. Higham.  `A New sqrtm for MATLAB'.  Numerical Analysis
     Report No. 336, Manchester Centre for Computational Mathematics,
     Manchester, England, January 1999.

     *See also:* *note expm: doc-expm, *note logm: doc-logm.

 -- Loadable Function:  kron (A, B)
 -- Loadable Function:  kron (A1, A2, ...)
     Form the Kronecker product of two or more matrices, defined block
     by block as

          x = [a(i, j) b]

     For example:

          kron (1:4, ones (3, 1))
                =>  1  2  3  4
                    1  2  3  4
                    1  2  3  4

     If there are more than two input arguments A1, A2, ..., AN the
     Kronecker product is computed as

          kron (kron (A1, A2), ..., AN)

     Since the Kronecker product is associative, this is well-defined.

 -- Loadable Function:  blkmm (A, B)
     Compute products of matrix blocks.  The blocks are given as
     2-dimensional subarrays of the arrays A, B.  The size of A must
     have the form `[m,k,...]' and size of B must be `[k,n,...]'.  The
     result is then of size `[m,n,...]' and is computed as follows:

            for i = 1:prod (size (A)(3:end))
              C(:,:,i) = A(:,:,i) * B(:,:,i)
            endfor

 -- Loadable Function: X = syl (A, B, C)
     Solve the Sylvester equation

          A X + X B + C = 0

     using standard LAPACK subroutines.  For example:

          syl ([1, 2; 3, 4], [5, 6; 7, 8], [9, 10; 11, 12])
               => [ -0.50000, -0.66667; -0.66667, -0.50000 ]


File: octave.info,  Node: Specialized Solvers,  Prev: Functions of a Matrix,  Up: Linear Algebra

18.5 Specialized Solvers
========================

 -- Function File: X = bicg (A, B, RTOL, MAXIT, M1, M2, X0)
 -- Function File: X = bicg (A, B, RTOL, MAXIT, P)
 -- Function File: [X, FLAG, RELRES, ITER, RESVEC] = bicg (A, B, ...)
     Solve `A x = b' using the Bi-conjugate gradient iterative method.

        - RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        - MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value `min (20, numel (b))' is used.

        - X0 the initial guess, if not given or set to [] the default
          value `zeros (size (b))' is used.

     A can be passed as a matrix or as a function handle or inline
     function `f' such that `f(x, "notransp") = A*x' and `f(x,
     "transp") = A'*x'.

     The preconditioner P is given as `P = M1 * M2'.  Both M1 and M2
     can be passed as a matrix or as a function handle or inline
     function `g' such that `g(x, 'notransp') = M1 \ x' or `g(x,
     'notransp') = M2 \ x' and `g(x, 'transp') = M1' \ x' or `g(x,
     'transp') = M2' \ x'.

     If called with more than one output parameter

        - FLAG indicates the exit status:
             - 0: iteration converged to the within the chosen tolerance

             - 1: the maximum number of iterations was reached before
               convergence

             - 3: the algorithm reached stagnation
          (the value 2 is unused but skipped for compatibility).

        - RELRES is the final value of the relative residual.

        - ITER is the number of iterations performed.

        - RESVEC is a vector containing the relative residual at each
          iteration.

     *See also:* *note bicgstab: doc-bicgstab, *note cgs: doc-cgs,
     *note gmres: doc-gmres, *note pcg: doc-pcg.


 -- Function File: X = bicgstab (A, B, RTOL, MAXIT, M1, M2, X0)
 -- Function File: X = bicgstab (A, B, RTOL, MAXIT, P)
 -- Function File: [X, FLAG, RELRES, ITER, RESVEC] = bicgstab (A, B,
          ...)
     Solve `A x = b' using the stabilizied Bi-conjugate gradient
     iterative method.

        - RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        - MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value `min (20, numel (b))' is used.

        - X0 the initial guess, if not given or set to [] the default
          value `zeros (size (b))' is used.

     A can be passed as a matrix or as a function handle or inline
     function `f' such that `f(x) = A*x'.

     The preconditioner P is given as `P = M1 * M2'.  Both M1 and M2
     can be passed as a matrix or as a function handle or inline
     function `g' such that `g(x) = M1 \ x' or `g(x) = M2 \ x'.

     If called with more than one output parameter

        - FLAG indicates the exit status:
             - 0: iteration converged to the within the chosen tolerance

             - 1: the maximum number of iterations was reached before
               convergence

             - 3: the algorithm reached stagnation
          (the value 2 is unused but skipped for compatibility).

        - RELRES is the final value of the relative residual.

        - ITER is the number of iterations performed.

        - RESVEC is a vector containing the relative residual at each
          iteration.

     *See also:* *note bicg: doc-bicg, *note cgs: doc-cgs, *note gmres:
     doc-gmres, *note pcg: doc-pcg.


 -- Function File: X = cgs (A, B, RTOL, MAXIT, M1, M2, X0)
 -- Function File: X = cgs (A, B, RTOL, MAXIT, P)
 -- Function File: [X, FLAG, RELRES, ITER, RESVEC] = cgs (A, B, ...)
     Solve `A x = b', where A is a square matrix, using the Conjugate
     Gradients Squared method.

        - RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        - MAXIT the maximum number of outer iterations, if not given or
          set to [] the default value `min (20, numel (b))' is used.

        - X0 the initial guess, if not given or set to [] the default
          value `zeros (size (b))' is used.

     A can be passed as a matrix or as a function handle or inline
     function `f' such that `f(x) = A*x'.

     The preconditioner P is given as `P = M1 * M2'.  Both M1 and M2
     can be passed as a matrix or as a function handle or inline
     function `g' such that `g(x) = M1 \ x' or `g(x) = M2 \ x'.

     If called with more than one output parameter

        - FLAG indicates the exit status:
             - 0: iteration converged to the within the chosen tolerance

             - 1: the maximum number of iterations was reached before
               convergence

             - 3: the algorithm reached stagnation
          (the value 2 is unused but skipped for compatibility).

        - RELRES is the final value of the relative residual.

        - ITER is the number of iterations performed.

        - RESVEC is a vector containing the relative residual at each
          iteration.

     *See also:* *note pcg: doc-pcg, *note bicgstab: doc-bicgstab,
     *note bicg: doc-bicg, *note gmres: doc-gmres.

 -- Function File: X = gmres (A, B, M, RTOL, MAXIT, M1, M2, X0)
 -- Function File: X = gmres (A, B, M, RTOL, MAXIT, P)
 -- Function File: [X, FLAG, RELRES, ITER, RESVEC] = gmres (...)
     Solve `A x = b' using the Preconditioned GMRES iterative method
     with restart, a.k.a. PGMRES(m).

        - RTOL is the relative tolerance, if not given or set to [] the
          default value 1e-6 is used.

        - MAXIT is the maximum number of outer iterations, if not given
          or set to [] the default value `min (10, numel (b) /
          restart)' is used.

        - X0 is the initial guess, if not given or set to [] the
          default value `zeros(size (b))' is used.

        - M is the restart parameter, if not given or set to [] the
          default value `numel (b)' is used.

     Argument A can be passed as a matrix, function handle, or inline
     function `f' such that `f(x) = A*x'.

     The preconditioner P is given as `P = M1 * M2'.  Both M1 and M2
     can be passed as a matrix, function handle, or inline function `g'
     such that `g(x) = M1\x' or `g(x) = M2\x'.

     Besides the vector X, additional outputs are:

        - FLAG indicates the exit status:
         0 : iteration converged to within the specified tolerance

         1 : maximum number of iterations exceeded

         2 : unused, but skipped for compatibility

         3 : algorithm reached stagnation

        - RELRES is the final value of the relative residual.

        - ITER is a vector containing the number of outer iterations and
          total iterations performed.

        - RESVEC is a vector containing the relative residual at each
          iteration.

     *See also:* *note bicg: doc-bicg, *note bicgstab: doc-bicgstab,
     *note cgs: doc-cgs, *note pcg: doc-pcg.


File: octave.info,  Node: Vectorization and Faster Code Execution,  Next: Nonlinear Equations,  Prev: Linear Algebra,  Up: Top

19 Vectorization and Faster Code Execution
******************************************

Vectorization is a programming technique that uses vector operations
instead of element-by-element loop-based operations.  Besides frequently
producing more succinct Octave code, vectorization also allows for
better optimization in the subsequent implementation.  The
optimizations may occur either in Octave's own Fortran, C, or C++
internal implementation, or even at a lower level depending on the
compiler and external numerical libraries used to build Octave.  The
ultimate goal is to make use of your hardware's vector instructions if
possible or to perform other optimizations in software.

   Vectorization is not a concept unique to Octave, but it is
particularly important because Octave is a matrix-oriented language.
Vectorized Octave code will see a dramatic speed up (10X-100X) in most
cases.

   This chapter discusses vectorization and other techniques for
writing faster code.

* Menu:

* Basic Vectorization::        Basic techniques for code optimization
* Broadcasting::               Broadcasting operations
* Function Application::       Applying functions to arrays, cells, and structs
* Accumulation::               Accumulation functions
* Miscellaneous Techniques::   Other techniques for speeding up code
* Examples::


File: octave.info,  Node: Basic Vectorization,  Next: Broadcasting,  Up: Vectorization and Faster Code Execution

19.1 Basic Vectorization
========================

To a very good first approximation, the goal in vectorization is to
write code that avoids loops and uses whole-array operations.  As a
trivial example, consider

     for i = 1:n
       for j = 1:m
         c(i,j) = a(i,j) + b(i,j);
       endfor
     endfor

compared to the much simpler

     c = a + b;

This isn't merely easier to write; it is also internally much easier to
optimize.  Octave delegates this operation to an underlying
implementation which, among other optimizations, may use special vector
hardware instructions or could conceivably even perform the additions in
parallel.  In general, if the code is vectorized, the underlying
implementation has more freedom about the assumptions it can make in
order to achieve faster execution.

   This is especially important for loops with "cheap" bodies.  Often it
suffices to vectorize just the innermost loop to get acceptable
performance.  A general rule of thumb is that the "order" of the
vectorized body should be greater or equal to the "order" of the
enclosing loop.

   As a less trivial example, instead of

     for i = 1:n-1
       a(i) = b(i+1) - b(i);
     endfor

write

     a = b(2:n) - b(1:n-1);

   This shows an important general concept about using arrays for
indexing instead of looping over an index variable.  *Note Index
Expressions::.  Also use boolean indexing generously.  If a condition
needs to be tested, this condition can also be written as a boolean
index.  For instance, instead of

     for i = 1:n
       if a(i) > 5
         a(i) -= 20
       endif
     endfor

write

     a(a>5) -= 20;

which exploits the fact that `a > 5' produces a boolean index.

   Use elementwise vector operators whenever possible to avoid looping
(operators like `.*' and `.^').  *Note Arithmetic Ops::.  For simple
inline functions, the `vectorize' function can do this automatically.

 -- Built-in Function:  vectorize (FUN)
     Create a vectorized version of the inline function FUN by
     replacing all occurrences of `*', `/', etc., with `.*', `./', etc.

     This may be useful, for example, when using inline functions with
     numerical integration or optimization where a vector-valued
     function is expected.

          fcn = vectorize (inline ("x^2 - 1"))
             => fcn = f(x) = x.^2 - 1
          quadv (fcn, 0, 3)
             => 6

     *See also:* *note inline: doc-inline, *note formula: doc-formula,
     *note argnames: doc-argnames.

   Also exploit broadcasting in these elementwise operators both to
avoid looping and unnecessary intermediate memory allocations.  *Note
Broadcasting::.

   Use built-in and library functions if possible.  Built-in and
compiled functions are very fast.  Even with an m-file library
function, chances are good that it is already optimized, or will be
optimized more in a future release.

   For instance, even better than

     a = b(2:n) - b(1:n-1);

is

     a = diff (b);

   Most Octave functions are written with vector and array arguments in
mind.  If you find yourself writing a loop with a very simple operation,
chances are that such a function already exists.  The following
functions occur frequently in vectorized code:

   * Index manipulation

        * find

        * sub2ind

        * ind2sub

        * sort

        * unique

        * lookup

        * ifelse / merge

   * Repetition
        * repmat

        * repelems

   * Vectorized arithmetic
        * sum

        * prod

        * cumsum

        * cumprod

        * sumsq

        * diff

        * dot

        * cummax

        * cummin

   * Shape of higher dimensional arrays
        * reshape

        * resize

        * permute

        * squeeze

        * deal



File: octave.info,  Node: Broadcasting,  Next: Function Application,  Prev: Basic Vectorization,  Up: Vectorization and Faster Code Execution

19.2 Broadcasting
=================

Broadcasting refers to how Octave binary operators and functions behave
when their matrix or array operands or arguments differ in size.  Since
version 3.6.0, Octave now automatically broadcasts vectors, matrices,
and arrays when using elementwise binary operators and functions.
Broadly speaking, smaller arrays are "broadcast" across the larger one,
until they have a compatible shape.  The rule is that corresponding
array dimensions must either

  1. be equal, or

  2. one of them must be 1.

In case all dimensions are equal, no broadcasting occurs and ordinary
element-by-element arithmetic takes place.  For arrays of higher
dimensions, if the number of dimensions isn't the same, then missing
trailing dimensions are treated as 1.  When one of the dimensions is 1,
the array with that singleton dimension gets copied along that dimension
until it matches the dimension of the other array.  For example,
consider

     x = [1 2 3;
          4 5 6;
          7 8 9];

     y = [10 20 30];

     x + y

Without broadcasting, `x + y' would be an error because the dimensions
do not agree.  However, with broadcasting it is as if the following
operation were performed:

     x = [1 2 3
          4 5 6
          7 8 9];

     y = [10 20 30
          10 20 30
          10 20 30];

     x + y
     =>    11   22   33
           14   25   36
           17   28   39

That is, the smaller array of size `[1 3]' gets copied along the
singleton dimension (the number of rows) until it is `[3 3]'.  No
actual copying takes place, however.  The internal implementation reuses
elements along the necessary dimension in order to achieve the desired
effect without copying in memory.

   Both arrays can be broadcast across each other, for example, all
pairwise differences of the elements of a vector with itself:

     y - y'
     =>    0   10   20
         -10    0   10
         -20  -10    0

Here the vectors of size `[1 3]' and `[3 1]' both get broadcast into
matrices of size `[3 3]' before ordinary matrix subtraction takes place.

   A special case of broadcasting that may be familiar is when all
dimensions of the array being broadcast are 1, i.e. the array is a
scalar. Thus for example, operations like `x - 42' and `max (x, 2)' are
basic examples of broadcasting.

   For a higher-dimensional example, suppose `img' is an RGB image of
size `[m n 3]' and we wish to multiply each color by a different
scalar.  The following code accomplishes this with broadcasting,

     img .*= permute ([0.8, 0.9, 1.2], [1, 3, 2]);

Note the usage of permute to match the dimensions of the `[0.8, 0.9,
1.2]' vector with `img'.

   For functions that are not written with broadcasting semantics,
`bsxfun' can be useful for coercing them to broadcast.

 -- Loadable Function:  bsxfun (F, A, B)
     The binary singleton expansion function applier performs
     broadcasting, that is, applies a binary function F
     element-by-element to two array arguments A and B, and expands as
     necessary singleton dimensions in either input argument.  F is a
     function handle, inline function, or string containing the name of
     the function to evaluate.  The function F must be capable of
     accepting two column-vector arguments of equal length, or one
     column vector argument and a scalar.

     The dimensions of A and B must be equal or singleton.  The
     singleton dimensions of the arrays will be expanded to the same
     dimensionality as the other array.

     *See also:* *note arrayfun: doc-arrayfun, *note cellfun:
     doc-cellfun.

   Broadcasting is only applied if either of the two broadcasting
conditions hold.  As usual, however, broadcasting does not apply when
two dimensions differ and neither is 1:

     x = [1 2 3
          4 5 6];
     y = [10 20
          30 40];
     x + y

This will produce an error about nonconformant arguments.

   Besides common arithmetic operations, several functions of two
arguments also broadcast.  The full list of functions and operators
that broadcast is

           plus      +  .+
           minus     -  .-
           times     .*
           rdivide   ./
           ldivide   .\
           power     .^  .**
           lt        <
           le        <=
           eq        ==
           gt        >
           ge        >=
           ne        !=  ~=
           and       &
           or        |
           atan2
           hypot
           max
           min
           mod
           rem
           xor

           +=  -=  .+=  .-=  .*=  ./=  .\=  .^=  .**=  &=  |=

   Beware of resorting to broadcasting if a simpler operation will
suffice.  For matrices A and B, consider the following:

     C = sum (permute (A, [1, 3, 2]) .* permute (B, [3, 2, 1]), 3);

This operation broadcasts the two matrices with permuted dimensions
across each other during elementwise multiplication in order to obtain a
larger 3-D array, and this array is then summed along the third
dimension.  A moment of thought will prove that this operation is
simply the much faster ordinary matrix multiplication, `C = A*B;'.

   A note on terminology: "broadcasting" is the term popularized by the
Numpy numerical environment in the Python programming language.  In
other programming languages and environments, broadcasting may also be
known as _binary singleton expansion_ (BSX, in MATLAB, and the origin
of the name of the `bsxfun' function), _recycling_ (R programming
language), _single-instruction multiple data_ (SIMD), or _replication_.

19.2.1 Broadcasting and Legacy Code
-----------------------------------

The new broadcasting semantics almost never affect code that worked in
previous versions of Octave.  Consequently, all code inherited from
MATLAB that worked in previous versions of Octave should still work
without change in Octave.  The only exception is code such as

     try
       c = a.*b;
     catch
       c = a.*a;
     end_try_catch

that may have relied on matrices of different size producing an error.
Due to how broadcasting changes semantics with older versions of Octave,
by default Octave warns if a broadcasting operation is performed.  To
disable this warning, refer to its ID (*note doc-warning_ids::):

     warning ("off", "Octave:broadcast");

If you want to recover the old behavior and produce an error, turn this
warning into an error:

     warning ("error", "Octave:broadcast");

For broadcasting on scalars that worked in previous versions of Octave,
this warning will not be emitted.


File: octave.info,  Node: Function Application,  Next: Accumulation,  Prev: Broadcasting,  Up: Vectorization and Faster Code Execution

19.3 Function Application
=========================

As a general rule, functions should already be written with matrix
arguments in mind and should consider whole matrix operations in a
vectorized manner.  Sometimes, writing functions in this way appears
difficult or impossible for various reasons.  For those situations,
Octave provides facilities for applying a function to each element of an
array, cell, or struct.

 -- Function File:  arrayfun (FUNC, A)
 -- Function File: X = arrayfun (FUNC, A)
 -- Function File: X = arrayfun (FUNC, A, B, ...)
 -- Function File: [X, Y, ...] = arrayfun (FUNC, A, ...)
 -- Function File:  arrayfun (..., "UniformOutput", VAL)
 -- Function File:  arrayfun (..., "ErrorHandler", ERRFUNC)
     Execute a function on each element of an array.  This is useful for
     functions that do not accept array arguments.  If the function does
     accept array arguments it is better to call the function directly.

     The first input argument FUNC can be a string, a function handle,
     an inline function, or an anonymous function.  The input argument
     A can be a logic array, a numeric array, a string array, a
     structure array, or a cell array.  By a call of the function
     `arrayfun' all elements of A are passed on to the named function
     FUNC individually.

     The named function can also take more than two input arguments,
     with the input arguments given as third input argument B, fourth
     input argument C, ...  If given more than one array input argument
     then all input arguments must have the same sizes, for example:

          arrayfun (@atan2, [1, 0], [0, 1])
          => ans = [1.5708   0.0000]

     If the parameter VAL after a further string input argument
     "UniformOutput" is set `true' (the default), then the named
     function FUNC must return a single element which then will be
     concatenated into the return value and is of type matrix.
     Otherwise, if that parameter is set to `false', then the outputs
     are concatenated in a cell array.  For example:

          arrayfun (@(x,y) x:y, "abc", "def", "UniformOutput", false)
          => ans =
              {
                [1,1] = abcd
                [1,2] = bcde
                [1,3] = cdef
              }

     If more than one output arguments are given then the named function
     must return the number of return values that also are expected, for
     example:

          [A, B, C] = arrayfun (@find, [10; 0], "UniformOutput", false)
          =>
          A =
          {
            [1,1] =  1
            [2,1] = [](0x0)
          }
          B =
          {
            [1,1] =  1
            [2,1] = [](0x0)
          }
          C =
          {
            [1,1] =  10
            [2,1] = [](0x0)
          }

     If the parameter ERRFUNC after a further string input argument
     "ErrorHandler" is another string, a function handle, an inline
     function, or an anonymous function, then ERRFUNC defines a
     function to call in the case that FUNC generates an error.  The
     definition of the function must be of the form

          function [...] = errfunc (S, ...)

     where there is an additional input argument to ERRFUNC relative to
     FUNC, given by S.  This is a structure with the elements
     "identifier", "message", and "index" giving, respectively, the
     error identifier, the error message, and the index of the array
     elements that caused the error.  The size of the output argument
     of ERRFUNC must have the same size as the output argument of FUNC,
     otherwise a real error is thrown.  For example:

          function y = ferr (s, x), y = "MyString"; endfunction
          arrayfun (@str2num, [1234],
                     "UniformOutput", false, "ErrorHandler", @ferr)
          => ans =
              {
               [1,1] = MyString
              }

     *See also:* *note spfun: doc-spfun, *note cellfun: doc-cellfun,
     *note structfun: doc-structfun.

 -- Function File: Y = spfun (F, S)
     Compute `f(S)' for the non-zero values of S.  This results in a
     sparse matrix with the same structure as S.  The function F can be
     passed as a string, a function handle, or an inline function.

     *See also:* *note arrayfun: doc-arrayfun, *note cellfun:
     doc-cellfun, *note structfun: doc-structfun.

 -- Loadable Function:  cellfun (NAME, C)
 -- Loadable Function:  cellfun ("size", C, K)
 -- Loadable Function:  cellfun ("isclass", C, CLASS)
 -- Loadable Function:  cellfun (FUNC, C)
 -- Loadable Function:  cellfun (FUNC, C, D)
 -- Loadable Function: [A, ...] = cellfun (...)
 -- Loadable Function:  cellfun (..., 'ErrorHandler', ERRFUNC)
 -- Loadable Function:  cellfun (..., 'UniformOutput', VAL)
     Evaluate the function named NAME on the elements of the cell array
     C.  Elements in C are passed on to the named function
     individually.  The function NAME can be one of the functions

    `isempty'
          Return 1 for empty elements.

    `islogical'
          Return 1 for logical elements.

    `isreal'
          Return 1 for real elements.

    `length'
          Return a vector of the lengths of cell elements.

    `ndims'
          Return the number of dimensions of each element.

    `numel'
    `prodofsize'
          Return the number of elements contained within each cell
          element.  The number is the product of the dimensions of the
          object at each cell element.

    `size'
          Return the size along the K-th dimension.

    `isclass'
          Return 1 for elements of CLASS.

     Additionally, `cellfun' accepts an arbitrary function FUNC in the
     form of an inline function, function handle, or the name of a
     function (in a character string).  In the case of a character
     string argument, the function must accept a single argument named
     X, and it must return a string value.  The function can take one
     or more arguments, with the inputs arguments given by C, D, etc.
     Equally the function can return one or more output arguments.  For
     example:

          cellfun ("atan2", {1, 0}, {0, 1})
               =>ans = [1.57080   0.00000]

     The number of output arguments of `cellfun' matches the number of
     output arguments of the function.  The outputs of the function
     will be collected into the output arguments of `cellfun' like this:

          function [a, b] = twoouts (x)
            a = x;
            b = x*x;
          endfunction
          [aa, bb] = cellfun(@twoouts, {1, 2, 3})
               =>
                  aa =
                     1 2 3
                  bb =
                     1 4 9

     Note that per default the output argument(s) are arrays of the
     same size as the input arguments.  Input arguments that are
     singleton (1x1) cells will be automatically expanded to the size
     of the other arguments.

     If the parameter 'UniformOutput' is set to true (the default),
     then the function must return scalars which will be concatenated
     into the return array(s).  If 'UniformOutput' is false, the
     outputs are concatenated into a cell array (or cell arrays).  For
     example:

          cellfun ("tolower", {"Foo", "Bar", "FooBar"},
                   "UniformOutput",false)
          => ans = {"foo", "bar", "foobar"}

     Given the parameter 'ErrorHandler', then ERRFUNC defines a
     function to call in case FUNC generates an error.  The form of the
     function is

          function [...] = errfunc (S, ...)

     where there is an additional input argument to ERRFUNC relative to
     FUNC, given by S.  This is a structure with the elements
     'identifier', 'message' and 'index', giving respectively the error
     identifier, the error message, and the index into the input
     arguments of the element that caused the error.  For example:

          function y = foo (s, x), y = NaN; endfunction
          cellfun ("factorial", {-1,2}, 'ErrorHandler', @foo)
          => ans = [NaN 2]

     Use `cellfun' intelligently.  The `cellfun' function is a useful
     tool for avoiding loops.  It is often used with anonymous function
     handles; however, calling an anonymous function involves an
     overhead quite comparable to the overhead of an m-file function.
     Passing a handle to a built-in function is faster, because the
     interpreter is not involved in the internal loop.  For example:

          a = {...}
          v = cellfun (@(x) det(x), a); # compute determinants
          v = cellfun (@det, a); # faster

     *See also:* *note arrayfun: doc-arrayfun, *note structfun:
     doc-structfun, *note spfun: doc-spfun.

 -- Function File:  structfun (FUNC, S)
 -- Function File: [A, ...] = structfun (...)
 -- Function File:  structfun (..., "ErrorHandler", ERRFUNC)
 -- Function File:  structfun (..., "UniformOutput", VAL)
     Evaluate the function named NAME on the fields of the structure S.
     The fields of S are passed to the function FUNC individually.

     `structfun' accepts an arbitrary function FUNC in the form of an
     inline function, function handle, or the name of a function (in a
     character string).  In the case of a character string argument, the
     function must accept a single argument named X, and it must return
     a string value.  If the function returns more than one argument,
     they are returned as separate output variables.

     If the parameter "UniformOutput" is set to true (the default),
     then the function must return a single element which will be
     concatenated into the return value.  If "UniformOutput" is false,
     the outputs are placed into a structure with the same fieldnames
     as the input structure.

          s.name1 = "John Smith";
          s.name2 = "Jill Jones";
          structfun (@(x) regexp (x, '(\w+)$', "matches"){1}, s,
                     "UniformOutput", false)
              =>
             {
               name1 = Smith
               name2 = Jones
             }

     Given the parameter "ErrorHandler", ERRFUNC defines a function to
     call in case FUNC generates an error.  The form of the function is

          function [...] = errfunc (SE, ...)

     where there is an additional input argument to ERRFUNC relative to
     FUNC, given by SE.  This is a structure with the elements
     "identifier", "message" and "index", giving respectively the error
     identifier, the error message, and the index into the input
     arguments of the element that caused the error.  For an example on
     how to use an error handler, *note `cellfun': doc-cellfun.

     *See also:* *note cellfun: doc-cellfun, *note arrayfun:
     doc-arrayfun, *note spfun: doc-spfun.


File: octave.info,  Node: Accumulation,  Next: Miscellaneous Techniques,  Prev: Function Application,  Up: Vectorization and Faster Code Execution

19.4 Accumulation
=================

Whenever it's possible to categorize according to indices the elements
of an array when performing a computation, accumulation functions can be
useful.

 -- Function File:  accumarray (SUBS, VALS, SZ, FUNC, FILLVAL, ISSPARSE)
 -- Function File:  accumarray (SUBS, VALS, ...)
     Create an array by accumulating the elements of a vector into the
     positions defined by their subscripts.  The subscripts are defined
     by the rows of the matrix SUBS and the values by VALS.  Each row
     of SUBS corresponds to one of the values in VALS.  If VALS is a
     scalar, it will be used for each of the row of SUBS.  If SUBS is a
     cell array of vectors, all vectors must be of the same length, and
     the subscripts in the Kth vector must correspond to the Kth
     dimension of the result.

     The size of the matrix will be determined by the subscripts
     themselves.  However, if SZ is defined it determines the matrix
     size.  The length of SZ must correspond to the number of columns
     in SUBS.  An exception is if SUBS has only one column, in which
     case SZ may be the dimensions of a vector and the subscripts of
     SUBS are taken as the indices into it.

     The default action of `accumarray' is to sum the elements with the
     same subscripts.  This behavior can be modified by defining the
     FUNC function.  This should be a function or function handle that
     accepts a column vector and returns a scalar.  The result of the
     function should not depend on the order of the subscripts.

     The elements of the returned array that have no subscripts
     associated with them are set to zero.  Defining FILLVAL to some
     other value allows these values to be defined.  This behavior
     changes, however, for certain values of FUNC.  If FUNC is `min'
     (respectively, `max') then the result will be filled with the
     minimum (respectively, maximum) integer if VALS is of integral
     type, logical false (respectively, logical true) if VALS is of
     logical type, zero if FILLVAL is zero and all values are
     non-positive (respectively, non-negative), and NaN otherwise.

     By default `accumarray' returns a full matrix.  If ISSPARSE is
     logically true, then a sparse matrix is returned instead.

     The following `accumarray' example constructs a frequency table
     that in the first column counts how many occurrences each number in
     the second column has, taken from the vector X.  Note the usage of
     `unique'  for assigning to all repeated elements of X the same
     index (*note doc-unique::).

          X = [91, 92, 90, 92, 90, 89, 91, 89, 90, 100, 100, 100];
          [U, ~, J] = unique (X);
          [accumarray(J', 1), U']
          => 2    89
             3    90
             2    91
             2    92
             3   100

     Another example, where the result is a multi-dimensional 3-D array
     and the default value (zero) appears in the output:

          accumarray ([1, 1, 1;
                       2, 1, 2;
                       2, 3, 2;
                       2, 1, 2;
                       2, 3, 2], 101:105)
          => ans(:,:,1) = [101, 0, 0; 0, 0, 0]
             ans(:,:,2) = [0, 0, 0; 206, 0, 208]

     The sparse option can be used as an alternative to the `sparse'
     constructor (*note doc-sparse::). Thus

          sparse (I, J, SV)

     can be written with `accumarray' as

          accumarray ([I, J], SV', [], [], 0, true)

     For repeated indices, `sparse' adds the corresponding value. To
     take the minimum instead, use `min' as an accumulator function:

          accumarray ([I, J], SV', [], @min, 0, true)

     The complexity of accumarray in general for the non-sparse case is
     generally O(M+N), where N is the number of subscripts and M is the
     maximum subscript (linearized in multi-dimensional case).  If FUNC
     is one of `@sum' (default), `@max', `@min' or `@(x) {x}', an
     optimized code path is used.  Note that for general reduction
     function the interpreter overhead can play a major part and it may
     be more efficient to do multiple accumarray calls and compute the
     results in a vectorized manner.

     *See also:* *note accumdim: doc-accumdim, *note unique:
     doc-unique, *note sparse: doc-sparse.

 -- Function File:  accumdim (SUBS, VALS, DIM, N, FUNC, FILLVAL)
     Create an array by accumulating the slices of an array into the
     positions defined by their subscripts along a specified dimension.
     The subscripts are defined by the index vector SUBS.  The
     dimension is specified by DIM.  If not given, it defaults to the
     first non-singleton dimension.  The length of SUBS must be equal
     to `size (VALS, DIM)'.

     The extent of the result matrix in the working dimension will be
     determined by the subscripts themselves.  However, if N is defined
     it determines this extent.

     The default action of `accumdim' is to sum the subarrays with the
     same subscripts.  This behavior can be modified by defining the
     FUNC function.  This should be a function or function handle that
     accepts an array and a dimension, and reduces the array along this
     dimension.  As a special exception, the built-in `min' and `max'
     functions can be used directly, and `accumdim' accounts for the
     middle empty argument that is used in their calling.

     The slices of the returned array that have no subscripts associated
     with them are set to zero.  Defining FILLVAL to some other value
     allows these values to be defined.

     An example of the use of `accumdim' is:

          accumdim ([1, 2, 1, 2, 1], [ 7, -10,   4;
                                      -5, -12,   8;
                                     -12,   2,   8;
                                     -10,   9,  -3;
                                      -5,  -3, -13])
          => ans = [-10,-11,-1;-15,-3,5]

     *See also:* *note accumarray: doc-accumarray.


File: octave.info,  Node: Miscellaneous Techniques,  Next: Examples,  Prev: Accumulation,  Up: Vectorization and Faster Code Execution

19.5 Miscellaneous Techniques
=============================

Here are some other ways of improving the execution speed of Octave
programs.

   * Avoid computing costly intermediate results multiple times.
     Octave currently does not eliminate common subexpressions.  Also,
     certain internal computation results are cached for variables.
     For instance, if a matrix variable is used multiple times as an
     index, checking the indices (and internal conversion to integers)
     is only done once.

   * Be aware of lazy copies (copy-on-write).  When a copy of an object
     is created, the data is not immediately copied, but rather shared.
     The actual copying is postponed until the copied data needs to be
     modified.  For example:

          a = zeros (1000); # create a 1000x1000 matrix
          b = a; # no copying done here
          b(1) = 1; # copying done here

     Lazy copying applies to whole Octave objects such as matrices,
     cells, struct, and also individual cell or struct elements (not
     array elements).

     Additionally, index expressions also use lazy copying when Octave
     can determine that the indexed portion is contiguous in memory.
     For example:

          a = zeros (1000); # create a 1000x1000 matrix
          b = a(:,10:100);  # no copying done here
          b = a(10:100,:);  # copying done here

     This applies to arrays (matrices), cell arrays, and structs indexed
     using `()'.  Index expressions generating comma-separated lists
     can also benefit from shallow copying in some cases.  In
     particular, when A is a struct array, expressions like `{a.x},
     {a(:,2).x}' will use lazy copying, so that data can be shared
     between a struct array and a cell array.

     Most indexing expressions do not live longer than their parent
     objects.  In rare cases, however, a lazily copied slice outlasts
     its parent, in which case it becomes orphaned, still occupying
     unnecessarily more memory than needed.  To provide a remedy
     working in most real cases, Octave checks for orphaned lazy slices
     at certain situations, when a value is stored into a "permanent"
     location, such as a named variable or cell or struct element, and
     possibly economizes them.  For example:

          a = zeros (1000); # create a 1000x1000 matrix
          b = a(:,10:100);  # lazy slice
          a = []; # the original a array is still allocated
          c{1} = b; # b is reallocated at this point

   * Avoid deep recursion.  Function calls to m-file functions carry a
     relatively significant overhead, so rewriting a recursion as a
     loop often helps.  Also, note that the maximum level of recursion
     is limited.

   * Avoid resizing matrices unnecessarily.  When building a single
     result matrix from a series of calculations, set the size of the
     result matrix first, then insert values into it.  Write

          result = zeros (big_n, big_m)
          for i = over:and_over
            ridx = ...
            cidx = ...
            result(ridx, cidx) = new_value ();
          endfor

     instead of

          result = [];
          for i = ever:and_ever
            result = [ result, new_value() ];
          endfor

     Sometimes the number of items can not be computed in advance, and
     stack-like operations are needed.  When elements are being
     repeatedly inserted or removed from the end of an array, Octave
     detects it as stack usage and attempts to use a smarter memory
     management strategy by pre-allocating the array in bigger chunks.
     This strategy is also applied to cell and struct arrays.

          a = [];
          while (condition)
            ...
            a(end+1) = value; # "push" operation
            ...
            a(end) = []; # "pop" operation
            ...
          endwhile

   * Avoid calling `eval' or `feval' excessively.  Parsing input or
     looking up the name of a function in the symbol table are
     relatively expensive operations.

     If you are using `eval' merely as an exception handling mechanism,
     and not because you need to execute some arbitrary text, use the
     `try' statement instead.  *Note The `try' Statement::.

   * Use `ignore_function_time_stamp' when appropriate.  If you are
     calling lots of functions, and none of them will need to change
     during your run, set the variable `ignore_function_time_stamp' to
     `"all"'.  This will stop Octave from checking the time stamp of a
     function file to see if it has been updated while the program is
     being run.


File: octave.info,  Node: Examples,  Prev: Miscellaneous Techniques,  Up: Vectorization and Faster Code Execution

19.6 Examples
=============

The following are examples of vectorization questions asked by actual
users of Octave and their solutions.

   * For a vector `A', the following loop

          n = length (A);
          B = zeros (n, 2);
          for i = 1:length(A)
            ## this will be two columns, the first is the difference and
            ## the second the mean of the two elements used for the diff.
            B(i,:) = [A(i+1)-A(i), (A(i+1) + A(i))/2)];
          endfor

     can be turned into the following one-liner:

          B = [diff(A)(:), 0.5*(A(1:end-1)+A(2:end))(:)]

     Note the usage of colon indexing to flatten an intermediate result
     into a column vector.  This is a common vectorization trick.



File: octave.info,  Node: Nonlinear Equations,  Next: Diagonal and Permutation Matrices,  Prev: Vectorization and Faster Code Execution,  Up: Top

20 Nonlinear Equations
**********************

* Menu:

* Solvers::
* Minimizers::


File: octave.info,  Node: Solvers,  Next: Minimizers,  Up: Nonlinear Equations

20.1 Solvers
============

Octave can solve sets of nonlinear equations of the form

     F (x) = 0

using the function `fsolve', which is based on the MINPACK subroutine
`hybrd'.  This is an iterative technique so a starting point must be
provided.  This also has the consequence that convergence is not
guaranteed even if a solution exists.

 -- Function File:  fsolve (FCN, X0, OPTIONS)
 -- Function File: [X, FVEC, INFO, OUTPUT, FJAC] = fsolve (FCN, ...)
     Solve a system of nonlinear equations defined by the function FCN.
     FCN should accept a vector (array) defining the unknown variables,
     and return a vector of left-hand sides of the equations.
     Right-hand sides are defined to be zeros.  In other words, this
     function attempts to determine a vector X such that `FCN (X)'
     gives (approximately) all zeros.  X0 determines a starting guess.
     The shape of X0 is preserved in all calls to FCN, but otherwise it
     is treated as a column vector.  OPTIONS is a structure specifying
     additional options.  Currently, `fsolve' recognizes these options:
     `"FunValCheck"', `"OutputFcn"', `"TolX"', `"TolFun"', `"MaxIter"',
     `"MaxFunEvals"', `"Jacobian"', `"Updating"', `"ComplexEqn"'
     `"TypicalX"', `"AutoScaling"' and `"FinDiffType"'.

     If `"Jacobian"' is `"on"', it specifies that FCN, called with 2
     output arguments, also returns the Jacobian matrix of right-hand
     sides at the requested point.  `"TolX"' specifies the termination
     tolerance in the unknown variables, while `"TolFun"' is a
     tolerance for equations.  Default is `1e-7' for both `"TolX"' and
     `"TolFun"'.

     If `"AutoScaling"' is on, the variables will be automatically
     scaled according to the column norms of the (estimated) Jacobian.
     As a result, TolF becomes scaling-independent.  By default, this
     option is off, because it may sometimes deliver unexpected (though
     mathematically correct) results.

     If `"Updating"' is "on", the function will attempt to use Broyden
     updates to update the Jacobian, in order to reduce the amount of
     Jacobian calculations.  If your user function always calculates
     the Jacobian (regardless of number of output arguments), this
     option provides no advantage and should be set to false.

     `"ComplexEqn"' is `"on"', `fsolve' will attempt to solve complex
     equations in complex variables, assuming that the equations
     possess a complex derivative (i.e., are holomorphic).  If this is
     not what you want, should unpack the real and imaginary parts of
     the system to get a real system.

     For description of the other options, see `optimset'.

     On return, FVAL contains the value of the function FCN evaluated
     at X, and INFO may be one of the following values:

    1
          Converged to a solution point.  Relative residual error is
          less than specified by TolFun.

    2
          Last relative step size was less that TolX.

    3
          Last relative decrease in residual was less than TolF.

    0
          Iteration limit exceeded.

    -3
          The trust region radius became excessively small.

     Note: If you only have a single nonlinear equation of one
     variable, using `fzero' is usually a much better idea.

     Note about user-supplied Jacobians: As an inherent property of the
     algorithm, Jacobian is always requested for a solution vector
     whose residual vector is already known, and it is the last
     accepted successful step.  Often this will be one of the last two
     calls, but not always.  If the savings by reusing intermediate
     results from residual calculation in Jacobian calculation are
     significant, the best strategy is to employ OutputFcn: After a
     vector is evaluated for residuals, if OutputFcn is called with
     that vector, then the intermediate results should be saved for
     future Jacobian evaluation, and should be kept until a Jacobian
     evaluation is requested or until outputfcn is called with a
     different vector, in which case they should be dropped in favor of
     this most recent vector.  A short example how this can be achieved
     follows:

          function [fvec, fjac] = user_func (x, optimvalues, state)
          persistent sav = [], sav0 = [];
          if (nargin == 1)
            ## evaluation call
            if (nargout == 1)
              sav0.x = x; # mark saved vector
              ## calculate fvec, save results to sav0.
            elseif (nargout == 2)
              ## calculate fjac using sav.
            endif
          else
            ## outputfcn call.
            if (all (x == sav0.x))
              sav = sav0;
            endif
            ## maybe output iteration status, etc.
          endif
          endfunction

           ...

          fsolve (@user_func, x0, optimset ("OutputFcn", @user_func, ...))

     *See also:* *note fzero: doc-fzero, *note optimset: doc-optimset.

   The following is a complete example.  To solve the set of equations

     -2x^2 + 3xy   + 4 sin(y) = 6
      3x^2 - 2xy^2 + 3 cos(x) = -4

you first need to write a function to compute the value of the given
function.  For example:

     function y = f (x)
       y = zeros (2, 1);
       y(1) = -2*x(1)^2 + 3*x(1)*x(2)   + 4*sin(x(2)) - 6;
       y(2) =  3*x(1)^2 - 2*x(1)*x(2)^2 + 3*cos(x(1)) + 4;
     endfunction

   Then, call `fsolve' with a specified initial condition to find the
roots of the system of equations.  For example, given the function `f'
defined above,

     [x, fval, info] = fsolve (@f, [1; 2])

results in the solution

     x =

       0.57983
       2.54621

     fval =

       -5.7184e-10
        5.5460e-10

     info = 1

A value of `info = 1' indicates that the solution has converged.

   When no Jacobian is supplied (as in the example above) it is
approximated numerically.  This requires more function evaluations, and
hence is less efficient.  In the example above we could compute the
Jacobian analytically as

     function [y, jac] = f (x)
       y = zeros (2, 1);
       y(1) = -2*x(1)^2 + 3*x(1)*x(2)   + 4*sin(x(2)) - 6;
       y(2) =  3*x(1)^2 - 2*x(1)*x(2)^2 + 3*cos(x(1)) + 4;
       if (nargout == 2)
         jac = zeros (2, 2);
         jac(1,1) =  3*x(2) - 4*x(1);
         jac(1,2) =  4*cos(x(2)) + 3*x(1);
         jac(2,1) = -2*x(2)^2 - 3*sin(x(1)) + 6*x(1);
         jac(2,2) = -4*x(1)*x(2);
       endif
     endfunction

The Jacobian can then be used with the following call to `fsolve':

     [x, fval, info] = fsolve (@f, [1; 2], optimset ("jacobian", "on"));

which gives the same solution as before.

 -- Function File:  fzero (FUN, X0)
 -- Function File:  fzero (FUN, X0, OPTIONS)
 -- Function File: [X, FVAL, INFO, OUTPUT] = fzero (...)
     Find a zero of a univariate function.

     FUN is a function handle, inline function, or string containing
     the name of the function to evaluate.  X0 should be a two-element
     vector specifying two points which bracket a zero.  In other
     words, there must be a change in sign of the function between
     X0(1) and X0(2).  More mathematically, the following must hold

          sign (FUN(X0(1))) * sign (FUN(X0(2))) <= 0

     If X0 is a single scalar then several nearby and distant values
     are probed in an attempt to obtain a valid bracketing.  If this is
     not successful, the function fails.  OPTIONS is a structure
     specifying additional options.  Currently, `fzero' recognizes
     these options: `"FunValCheck"', `"OutputFcn"', `"TolX"',
     `"MaxIter"', `"MaxFunEvals"'.  For a description of these options,
     see *note optimset: doc-optimset.

     On exit, the function returns X, the approximate zero point and
     FVAL, the function value thereof.  INFO is an exit flag that can
     have these values:

        * 1  The algorithm converged to a solution.

        * 0  Maximum number of iterations or function evaluations has
          been reached.

        * -1 The algorithm has been terminated from user output
          function.

        * -5 The algorithm may have converged to a singular point.

     OUTPUT is a structure containing runtime information about the
     `fzero' algorithm.  Fields in the structure are:

        * iterations  Number of iterations through loop.

        * nfev  Number of function evaluations.

        * bracketx  A two-element vector with the final bracketing of
          the zero along the x-axis.

        * brackety  A two-element vector with the final bracketing of
          the zero along the y-axis.

     *See also:* *note optimset: doc-optimset, *note fsolve: doc-fsolve.


File: octave.info,  Node: Minimizers,  Prev: Solvers,  Up: Nonlinear Equations

20.2 Minimizers
===============

Often it is useful to find the minimum value of a function rather than
just the zeroes where it crosses the x-axis.  `fminbnd' is designed for
the simpler, but very common, case of a univariate function where the
interval to search is bounded.  For unbounded minimization of a
function with potentially many variables use `fminunc'.  *Note
Optimization::, for minimization with the presence of constraint
functions.  Note that searches can be made for maxima by simply
inverting the objective function (`Fto_max = -Fto_min').

 -- Function File: [X, FVAL, INFO, OUTPUT] = fminbnd (FUN, A, B,
          OPTIONS)
     Find a minimum point of a univariate function.  FUN should be a
     function handle or name.  A, B specify a starting interval.
     OPTIONS is a structure specifying additional options.  Currently,
     `fminbnd' recognizes these options: `"FunValCheck"', `"OutputFcn"',
     `"TolX"', `"MaxIter"', `"MaxFunEvals"'.  For description of these
     options, see *note optimset: doc-optimset.

     On exit, the function returns X, the approximate minimum point and
     FVAL, the function value thereof.  INFO is an exit flag that can
     have these values:

        * 1 The algorithm converged to a solution.

        * 0 Maximum number of iterations or function evaluations has
          been exhausted.

        * -1 The algorithm has been terminated from user output
          function.

     *See also:* *note optimset: doc-optimset, *note fzero: doc-fzero,
     *note fminunc: doc-fminunc.

 -- Function File:  fminunc (FCN, X0)
 -- Function File:  fminunc (FCN, X0, OPTIONS)
 -- Function File: [X, FVEC, INFO, OUTPUT, GRAD, HESS] = fminunc (FCN,
          ...)
     Solve an unconstrained optimization problem defined by the function
     FCN.  FCN should accepts a vector (array) defining the unknown
     variables, and return the objective function value, optionally
     with gradient.  In other words, this function attempts to
     determine a vector X such that `FCN (X)' is a local minimum.  X0
     determines a starting guess.  The shape of X0 is preserved in all
     calls to FCN, but otherwise is treated as a column vector.
     OPTIONS is a structure specifying additional options.  Currently,
     `fminunc' recognizes these options: `"FunValCheck"',
     `"OutputFcn"', `"TolX"', `"TolFun"', `"MaxIter"', `"MaxFunEvals"',
     `"GradObj"', `"FinDiffType"', `"TypicalX"', `"AutoScaling"'.

     If `"GradObj"' is `"on"', it specifies that FCN, called with 2
     output arguments, also returns the Jacobian matrix of right-hand
     sides at the requested point.  `"TolX"' specifies the termination
     tolerance in the unknown variables, while `"TolFun"' is a
     tolerance for equations.  Default is `1e-7' for both `"TolX"' and
     `"TolFun"'.

     For description of the other options, see `optimset'.

     On return, FVAL contains the value of the function FCN evaluated
     at X, and INFO may be one of the following values:

    1
          Converged to a solution point.  Relative gradient error is
          less than specified by TolFun.

    2
          Last relative step size was less that TolX.

    3
          Last relative decrease in function value was less than TolF.

    0
          Iteration limit exceeded.

    -3
          The trust region radius became excessively small.

     Optionally, fminunc can also yield a structure with convergence
     statistics (OUTPUT), the output gradient (GRAD) and approximate
     Hessian (HESS).

     Note: If you only have a single nonlinear equation of one
     variable, using `fminbnd' is usually a much better idea.

     *See also:* *note fminbnd: doc-fminbnd, *note optimset:
     doc-optimset.


File: octave.info,  Node: Diagonal and Permutation Matrices,  Next: Sparse Matrices,  Prev: Nonlinear Equations,  Up: Top

21 Diagonal and Permutation Matrices
************************************

* Menu:

* Basic Usage::          Creation and Manipulation of Diagonal and Permutation Matrices
* Matrix Algebra::       Linear Algebra with Diagonal and Permutation Matrices
* Function Support::     Functions That Are Aware of These Matrices
* Example Codes::        Some Examples of Usage
* Zeros Treatment::      The Differences in Treatment of Zero Elements


File: octave.info,  Node: Basic Usage,  Next: Matrix Algebra,  Up: Diagonal and Permutation Matrices

21.1 Creating and Manipulating Diagonal and Permutation Matrices
================================================================

A diagonal matrix is defined as a matrix that has zero entries outside
the main diagonal; that is, `D(i,j) == 0' if `i != j'.  Most often,
square diagonal matrices are considered; however, the definition can
equally be applied to non-square matrices, in which case we usually
speak of a rectangular diagonal matrix.

   A permutation matrix is defined as a square matrix that has a single
element equal to unity in each row and each column; all other elements
are zero.  That is, there exists a permutation (vector) `p' such that
`P(i,j) == 1' if `j == p(i)' and `P(i,j) == 0' otherwise.

   Octave provides special treatment of real and complex rectangular
diagonal matrices, as well as permutation matrices.  They are stored as
special objects, using efficient storage and algorithms, facilitating
writing both readable and efficient matrix algebra expressions in the
Octave language.

* Menu:

* Creating Diagonal Matrices::
* Creating Permutation Matrices::
* Explicit and Implicit Conversions::


File: octave.info,  Node: Creating Diagonal Matrices,  Next: Creating Permutation Matrices,  Up: Basic Usage

21.1.1 Creating Diagonal Matrices
---------------------------------

The most common and easiest way to create a diagonal matrix is using the
built-in function "diag".  The expression `diag (v)', with V a vector,
will create a square diagonal matrix with elements on the main diagonal
given by the elements of V, and size equal to the length of V.  `diag
(v, m, n)' can be used to construct a rectangular diagonal matrix.  The
result of these expressions will be a special diagonal matrix object,
rather than a general matrix object.

   Diagonal matrix with unit elements can be created using "eye".  Some
other built-in functions can also return diagonal matrices.  Examples
include "balance" or "inv".

   Example:

       diag (1:4)
     =>
     Diagonal Matrix

        1   0   0   0
        0   2   0   0
        0   0   3   0
        0   0   0   4

       diag(1:3,5,3)

     =>
     Diagonal Matrix

        1   0   0
        0   2   0
        0   0   3
        0   0   0
        0   0   0


File: octave.info,  Node: Creating Permutation Matrices,  Next: Explicit and Implicit Conversions,  Prev: Creating Diagonal Matrices,  Up: Basic Usage

21.1.2 Creating Permutation Matrices
------------------------------------

For creating permutation matrices, Octave does not introduce a new
function, but rather overrides an existing syntax: permutation matrices
can be conveniently created by indexing an identity matrix by
permutation vectors.  That is, if Q is a permutation vector of length
N, the expression

       P = eye (n) (:, q);

will create a permutation matrix - a special matrix object.

     eye (n) (q, :)

will also work (and create a row permutation matrix), as well as

     eye (n) (q1, q2).

   For example:

       eye (4) ([1,3,2,4],:)
     =>
     Permutation Matrix

        1   0   0   0
        0   0   1   0
        0   1   0   0
        0   0   0   1

       eye (4) (:,[1,3,2,4])
     =>
     Permutation Matrix

        1   0   0   0
        0   0   1   0
        0   1   0   0
        0   0   0   1

   Mathematically, an identity matrix is both diagonal and permutation
matrix.  In Octave, `eye (n)' returns a diagonal matrix, because a
matrix can only have one class.  You can convert this diagonal matrix
to a permutation matrix by indexing it by an identity permutation, as
shown below.  This is a special property of the identity matrix;
indexing other diagonal matrices generally produces a full matrix.

       eye (3)
     =>
     Diagonal Matrix

        1   0   0
        0   1   0
        0   0   1

       eye(3)(1:3,:)
     =>
     Permutation Matrix

        1   0   0
        0   1   0
        0   0   1

   Some other built-in functions can also return permutation matrices.
Examples include "inv" or "lu".


File: octave.info,  Node: Explicit and Implicit Conversions,  Prev: Creating Permutation Matrices,  Up: Basic Usage

21.1.3 Explicit and Implicit Conversions
----------------------------------------

The diagonal and permutation matrices are special objects in their own
right.  A number of operations and built-in functions are defined for
these matrices to use special, more efficient code than would be used
for a full matrix in the same place.  Examples are given in further
sections.

   To facilitate smooth mixing with full matrices, backward
compatibility, and compatibility with MATLAB, the diagonal and
permutation matrices should allow any operation that works on full
matrices, and will either treat it specially, or implicitly convert
themselves to full matrices.

   Instances include matrix indexing, except for extracting a single
element or a leading submatrix, indexed assignment, or applying most
mapper functions, such as "exp".

   An explicit conversion to a full matrix can be requested using the
built-in function "full".  It should also be noted that the diagonal
and permutation matrix objects will cache the result of the conversion
after it is first requested (explicitly or implicitly), so that
subsequent conversions will be very cheap.


File: octave.info,  Node: Matrix Algebra,  Next: Function Support,  Prev: Basic Usage,  Up: Diagonal and Permutation Matrices

21.2 Linear Algebra with Diagonal and Permutation Matrices
==========================================================

As has been already said, diagonal and permutation matrices make it
possible to use efficient algorithms while preserving natural linear
algebra syntax.  This section describes in detail the operations that
are treated specially when performed on these special matrix objects.

* Menu:

* Expressions Involving Diagonal Matrices::
* Expressions Involving Permutation Matrices::


File: octave.info,  Node: Expressions Involving Diagonal Matrices,  Next: Expressions Involving Permutation Matrices,  Up: Matrix Algebra

21.2.1 Expressions Involving Diagonal Matrices
----------------------------------------------

Assume D is a diagonal matrix.  If M is a full matrix, then `D*M' will
scale the rows of M.  That means, if `S = D*M', then for each pair of
indices i,j it holds

     S(i,j) = D(i,i) * M(i,j).

   Similarly, `M*D' will do a column scaling.

   The matrix D may also be rectangular, m-by-n where `m != n'.  If `m
< n', then the expression `D*M' is equivalent to

     D(:,1:m) * M(1:m,:),

i.e., trailing `n-m' rows of M are ignored.  If `m > n', then `D*M' is
equivalent to

     [D(1:n,n) * M; zeros(m-n, columns (M))],

i.e., null rows are appended to the result.  The situation for
right-multiplication `M*D' is analogous.

   The expressions `D \ M' and `M / D' perform inverse scaling.  They
are equivalent to solving a diagonal (or rectangular diagonal) in a
least-squares minimum-norm sense.  In exact arithmetic, this is
equivalent to multiplying by a pseudoinverse.  The pseudoinverse of a
rectangular diagonal matrix is again a rectangular diagonal matrix with
swapped dimensions, where each nonzero diagonal element is replaced by
its reciprocal.  The matrix division algorithms do, in fact, use
division rather than multiplication by reciprocals for better numerical
accuracy; otherwise, they honor the above definition.  Note that a
diagonal matrix is never truncated due to ill-conditioning; otherwise,
it would not be much useful for scaling.  This is typically consistent
with linear algebra needs.  A full matrix that only happens to be
diagonal (an is thus not a special object) is of course treated
normally.

   Multiplication and division by diagonal matrices works efficiently
also when combined with sparse matrices, i.e., `D*S', where D is a
diagonal matrix and S is a sparse matrix scales the rows of the sparse
matrix and returns a sparse matrix.  The expressions `S*D', `D\S', `S/D'
work analogically.

   If D1 and D2 are both diagonal matrices, then the expressions

     D1 + D2
     D1 - D2
     D1 * D2
     D1 / D2
     D1 \ D2

again produce diagonal matrices, provided that normal dimension
matching rules are obeyed.  The relations used are same as described
above.

   Also, a diagonal matrix D can be multiplied or divided by a scalar,
or raised to a scalar power if it is square, producing diagonal matrix
result in all cases.

   A diagonal matrix can also be transposed or conjugate-transposed,
giving the expected result.  Extracting a leading submatrix of a
diagonal matrix, i.e., `D(1:m,1:n)', will produce a diagonal matrix,
other indexing expressions will implicitly convert to full matrix.

   Adding a diagonal matrix to a full matrix only operates on the
diagonal elements.  Thus,

     A = A + eps * eye (n)

is an efficient method of augmenting the diagonal of a matrix.
Subtraction works analogically.

   When involved in expressions with other element-by-element
operators, `.*', `./', `.\' or `.^', an implicit conversion to full
matrix will take place.  This is not always strictly necessary but
chosen to facilitate better consistency with MATLAB.


File: octave.info,  Node: Expressions Involving Permutation Matrices,  Prev: Expressions Involving Diagonal Matrices,  Up: Matrix Algebra

21.2.2 Expressions Involving Permutation Matrices
-------------------------------------------------

If P is a permutation matrix and M a matrix, the expression `P*M' will
permute the rows of M.  Similarly, `M*P' will yield a column
permutation.  Matrix division `P\M' and `M/P' can be used to do inverse
permutation.

   The previously described syntax for creating permutation matrices
can actually help an user to understand the connection between a
permutation matrix and a permuting vector.  Namely, the following
holds, where `I = eye (n)' is an identity matrix:

       I(p,:) * M = (I*M) (p,:) = M(p,:)

   Similarly,

       M * I(:,p) = (M*I) (:,p) = M(:,p)

   The expressions `I(p,:)' and `I(:,p)' are permutation matrices.

   A permutation matrix can be transposed (or conjugate-transposed,
which is the same, because a permutation matrix is never complex),
inverting the permutation, or equivalently, turning a row-permutation
matrix into a column-permutation one.  For permutation matrices,
transpose is equivalent to inversion, thus `P\M' is equivalent to
`P'*M'.  Transpose of a permutation matrix (or inverse) is a
constant-time operation, flipping only a flag internally, and thus the
choice between the two above equivalent expressions for inverse
permuting is completely up to the user's taste.

   Multiplication and division by permutation matrices works
efficiently also when combined with sparse matrices, i.e., `P*S', where
P is a permutation matrix and S is a sparse matrix permutes the rows of
the sparse matrix and returns a sparse matrix.  The expressions `S*P',
`P\S', `S/P' work analogically.

   Two permutation matrices can be multiplied or divided (if their
sizes match), performing a composition of permutations.  Also a
permutation matrix can be indexed by a permutation vector (or two
vectors), giving again a permutation matrix.  Any other operations do
not generally yield a permutation matrix and will thus trigger the
implicit conversion.


File: octave.info,  Node: Function Support,  Next: Example Codes,  Prev: Matrix Algebra,  Up: Diagonal and Permutation Matrices

21.3 Functions That Are Aware of These Matrices
===============================================

This section lists the built-in functions that are aware of diagonal and
permutation matrices on input, or can return them as output.  Passed to
other functions, these matrices will in general trigger an implicit
conversion.  (Of course, user-defined dynamically linked functions may
also work with diagonal or permutation matrices).

* Menu:

* Diagonal Matrix Functions::
* Permutation Matrix Functions::


File: octave.info,  Node: Diagonal Matrix Functions,  Next: Permutation Matrix Functions,  Up: Function Support

21.3.1 Diagonal Matrix Functions
--------------------------------

"inv" and "pinv" can be applied to a diagonal matrix, yielding again a
diagonal matrix.  "det" will use an efficient straightforward
calculation when given a diagonal matrix, as well as "cond".  The
following mapper functions can be applied to a diagonal matrix without
converting it to a full one: "abs", "real", "imag", "conj", "sqrt".  A
diagonal matrix can also be returned from the "balance" and "svd"
functions.  The "sparse" function will convert a diagonal matrix
efficiently to a sparse matrix.


File: octave.info,  Node: Permutation Matrix Functions,  Prev: Diagonal Matrix Functions,  Up: Function Support

21.3.2 Permutation Matrix Functions
-----------------------------------

"inv" and "pinv" will invert a permutation matrix, preserving its
specialness.  "det" can be applied to a permutation matrix, efficiently
calculating the sign of the permutation (which is equal to the
determinant).

   A permutation matrix can also be returned from the built-in functions
"lu" and "qr", if a pivoted factorization is requested.

   The "sparse" function will convert a permutation matrix efficiently
to a sparse matrix.  The "find" function will also work efficiently
with a permutation matrix, making it possible to conveniently obtain
the permutation indices.


File: octave.info,  Node: Example Codes,  Next: Zeros Treatment,  Prev: Function Support,  Up: Diagonal and Permutation Matrices

21.4 Some Examples of Usage
===========================

The following can be used to solve a linear system `A*x = b' using the
pivoted LU factorization:

       [L, U, P] = lu (A); ## now L*U = P*A
       x = U \ L \ P*b;

This is how you normalize columns of a matrix X to unit norm:

       s = norm (X, "columns");
       X = diag (s) \ X;

The following expression is a way to efficiently calculate the sign of a
permutation, given by a permutation vector P.  It will also work in
earlier versions of Octave, but slowly.

       det (eye (length (p))(p, :))

Finally, here's how you solve a linear system `A*x = b' with Tikhonov
regularization (ridge regression) using SVD (a skeleton only):

       m = rows (A); n = columns (A);
       [U, S, V] = svd (A);
       ## determine the regularization factor alpha
       ## alpha = ...
       ## transform to orthogonal basis
       b = U'*b;
       ## Use the standard formula, replacing A with S.
       ## S is diagonal, so the following will be very fast and accurate.
       x = (S'*S + alpha^2 * eye (n)) \ (S' * b);
       ## transform to solution basis
       x = V*x;


File: octave.info,  Node: Zeros Treatment,  Prev: Example Codes,  Up: Diagonal and Permutation Matrices

21.5 The Differences in Treatment of Zero Elements
==================================================

Making diagonal and permutation matrices special matrix objects in
their own right and the consequent usage of smarter algorithms for
certain operations implies, as a side effect, small differences in
treating zeros.  The contents of this section applies also to sparse
matrices, discussed in the following chapter.

   The IEEE standard defines the result of the expressions `0*Inf' and
`0*NaN' as `NaN', as it has been generally agreed that this is the best
compromise.  Numerical software dealing with structured and sparse
matrices (including Octave) however, almost always makes a distinction
between a "numerical zero" and an "assumed zero".  A "numerical zero"
is a zero value occurring in a place where any floating-point value
could occur.  It is normally stored somewhere in memory as an explicit
value.  An "assumed zero", on the contrary, is a zero matrix element
implied by the matrix structure (diagonal, triangular) or a sparsity
pattern; its value is usually not stored explicitly anywhere, but is
implied by the underlying data structure.

   The primary distinction is that an assumed zero, when multiplied by
any number, or divided by any nonzero number, yields *always* a zero,
even when, e.g., multiplied by `Inf' or divided by `NaN'.  The reason
for this behavior is that the numerical multiplication is not actually
performed anywhere by the underlying algorithm; the result is just
assumed to be zero.  Equivalently, one can say that the part of the
computation involving assumed zeros is performed symbolically, not
numerically.

   This behavior not only facilitates the most straightforward and
efficient implementation of algorithms, but also preserves certain
useful invariants, like:
   * scalar * diagonal matrix is a diagonal matrix

   * sparse matrix / scalar preserves the sparsity pattern

   * permutation matrix * matrix is equivalent to permuting rows
   all of these natural mathematical truths would be invalidated by
treating assumed zeros as numerical ones.

   Note that MATLAB does not strictly follow this principle and converts
assumed zeros to numerical zeros in certain cases, while not doing so in
other cases.  As of today, there are no intentions to mimic such
behavior in Octave.

   Examples of effects of assumed zeros vs. numerical zeros:

     Inf * eye (3)
     =>
        Inf     0     0
          0   Inf     0
          0     0   Inf

     Inf * speye (3)
     =>
     Compressed Column Sparse (rows = 3, cols = 3, nnz = 3 [33%])

       (1, 1) -> Inf
       (2, 2) -> Inf
       (3, 3) -> Inf

     Inf * full (eye (3))
     =>
        Inf   NaN   NaN
        NaN   Inf   NaN
        NaN   NaN   Inf

     diag(1:3) * [NaN; 1; 1]
     =>
        NaN
          2
          3

     sparse(1:3,1:3,1:3) * [NaN; 1; 1]
     =>
        NaN
          2
          3
     [1,0,0;0,2,0;0,0,3] * [NaN; 1; 1]
     =>
        NaN
        NaN
        NaN


File: octave.info,  Node: Sparse Matrices,  Next: Numerical Integration,  Prev: Diagonal and Permutation Matrices,  Up: Top

22 Sparse Matrices
******************

* Menu:

* Basics::                      Creation and Manipulation of Sparse Matrices
* Sparse Linear Algebra::       Linear Algebra on Sparse Matrices
* Iterative Techniques::        Iterative Techniques
* Real Life Example::           Using Sparse Matrices


File: octave.info,  Node: Basics,  Next: Sparse Linear Algebra,  Up: Sparse Matrices

22.1 The Creation and Manipulation of Sparse Matrices
=====================================================

The size of mathematical problems that can be treated at any particular
time is generally limited by the available computing resources.  Both,
the speed of the computer and its available memory place limitation on
the problem size.

   There are many classes of mathematical problems which give rise to
matrices, where a large number of the elements are zero.  In this case
it makes sense to have a special matrix type to handle this class of
problems where only the non-zero elements of the matrix are stored.
Not only does this reduce the amount of memory to store the matrix, but
it also means that operations on this type of matrix can take advantage
of the a-priori knowledge of the positions of the non-zero elements to
accelerate their calculations.

   A matrix type that stores only the non-zero elements is generally
called sparse.  It is the purpose of this document to discuss the
basics of the storage and creation of sparse matrices and the
fundamental operations on them.

* Menu:

* Storage of Sparse Matrices::
* Creating Sparse Matrices::
* Information::
* Operators and Functions::


File: octave.info,  Node: Storage of Sparse Matrices,  Next: Creating Sparse Matrices,  Up: Basics

22.1.1 Storage of Sparse Matrices
---------------------------------

It is not strictly speaking necessary for the user to understand how
sparse matrices are stored.  However, such an understanding will help
to get an understanding of the size of sparse matrices.  Understanding
the storage technique is also necessary for those users wishing to
create their own oct-files.

   There are many different means of storing sparse matrix data.  What
all of the methods have in common is that they attempt to reduce the
complexity and storage given a-priori knowledge of the particular class
of problems that will be solved.  A good summary of the available
techniques for storing sparse matrix is given by Saad (1).  With full
matrices, knowledge of the point of an element of the matrix within the
matrix is implied by its position in the computers memory.  However,
this is not the case for sparse matrices, and so the positions of the
non-zero elements of the matrix must equally be stored.

   An obvious way to do this is by storing the elements of the matrix as
triplets, with two elements being their position in the array (rows and
column) and the third being the data itself.  This is conceptually easy
to grasp, but requires more storage than is strictly needed.

   The storage technique used within Octave is the compressed column
format.  In this format the position of each element in a row and the
data are stored as previously.  However, if we assume that all elements
in the same column are stored adjacent in the computers memory, then we
only need to store information on the number of non-zero elements in
each column, rather than their positions.  Thus assuming that the
matrix has more non-zero elements than there are columns in the matrix,
we win in terms of the amount of memory used.

   In fact, the column index contains one more element than the number
of columns, with the first element always being zero.  The advantage of
this is a simplification in the code, in that there is no special case
for the first or last columns.  A short example, demonstrating this in
C is.

       for (j = 0; j < nc; j++)
         for (i = cidx (j); i < cidx(j+1); i++)
            printf ("non-zero element (%i,%i) is %d\n",
                ridx(i), j, data(i));

   A clear understanding might be had by considering an example of how
the above applies to an example matrix.  Consider the matrix

         1   2   0  0
         0   0   0  3
         0   0   0  4

   The non-zero elements of this matrix are

        (1, 1)  => 1
        (1, 2)  => 2
        (2, 4)  => 3
        (3, 4)  => 4

   This will be stored as three vectors CIDX, RIDX and DATA,
representing the column indexing, row indexing and data respectively.
The contents of these three vectors for the above matrix will be

       CIDX = [0, 1, 2, 2, 4]
       RIDX = [0, 0, 1, 2]
       DATA = [1, 2, 3, 4]

   Note that this is the representation of these elements with the
first row and column assumed to start at zero, while in Octave itself
the row and column indexing starts at one.  Thus the number of elements
in the I-th column is given by `CIDX (I + 1) - CIDX (I)'.

   Although Octave uses a compressed column format, it should be noted
that compressed row formats are equally possible.  However, in the
context of mixed operations between mixed sparse and dense matrices, it
makes sense that the elements of the sparse matrices are in the same
order as the dense matrices.  Octave stores dense matrices in column
major ordering, and so sparse matrices are equally stored in this
manner.

   A further constraint on the sparse matrix storage used by Octave is
that all elements in the rows are stored in increasing order of their
row index, which makes certain operations faster.  However, it imposes
the need to sort the elements on the creation of sparse matrices.
Having disordered elements is potentially an advantage in that it makes
operations such as concatenating two sparse matrices together easier
and faster, however it adds complexity and speed problems elsewhere.

   ---------- Footnotes ----------

   (1) Y. Saad "SPARSKIT: A basic toolkit for sparse matrix
computation", 1994,
`http://www-users.cs.umn.edu/~saad/software/SPARSKIT/paper.ps'


File: octave.info,  Node: Creating Sparse Matrices,  Next: Information,  Prev: Storage of Sparse Matrices,  Up: Basics

22.1.2 Creating Sparse Matrices
-------------------------------

There are several means to create sparse matrix.

Returned from a function
     There are many functions that directly return sparse matrices.
     These include "speye", "sprand", "diag", etc.

Constructed from matrices or vectors
     The function "sparse" allows a sparse matrix to be constructed from
     three vectors representing the row, column and data.
     Alternatively, the function "spconvert" uses a three column matrix
     format to allow easy importation of data from elsewhere.

Created and then filled
     The function "sparse" or "spalloc" can be used to create an empty
     matrix that is then filled by the user

From a user binary program
     The user can directly create the sparse matrix within an oct-file.

   There are several basic functions to return specific sparse
matrices.  For example the sparse identity matrix, is a matrix that is
often needed.  It therefore has its own function to create it as `speye
(N)' or `speye (R, C)', which creates an N-by-N or R-by-C sparse
identity matrix.

   Another typical sparse matrix that is often needed is a random
distribution of random elements.  The functions "sprand" and "sprandn"
perform this for uniform and normal random distributions of elements.
They have exactly the same calling convention, where `sprand (R, C, D)',
creates an R-by-C sparse matrix with a density of filled elements of D.

   Other functions of interest that directly create sparse matrices, are
"diag" or its generalization "spdiags", that can take the definition of
the diagonals of the matrix and create the sparse matrix that
corresponds to this.  For example,

     s = diag (sparse(randn(1,n)), -1);

creates a sparse (N+1)-by-(N+1) sparse matrix with a single diagonal
defined.

 -- Function File: [B, C] = spdiags (A)
 -- Function File: B = spdiags (A, C)
 -- Function File: B = spdiags (V, C, A)
 -- Function File: B = spdiags (V, C, M, N)
     A generalization of the function `diag'.  Called with a single
     input argument, the non-zero diagonals C of A are extracted.  With
     two arguments the diagonals to extract are given by the vector C.

     The other two forms of `spdiags' modify the input matrix by
     replacing the diagonals.  They use the columns of V to replace the
     columns represented by the vector C.  If the sparse matrix A is
     defined then the diagonals of this matrix are replaced.  Otherwise
     a matrix of M by N is created with the diagonals given by V.

     Negative values of C represent diagonals below the main diagonal,
     and positive values of C diagonals above the main diagonal.

     For example:

          spdiags (reshape (1:12, 4, 3), [-1 0 1], 5, 4)
          =>    5 10  0  0
                1  6 11  0
                0  2  7 12
                0  0  3  8
                0  0  0  4


 -- Function File: Y = speye (M)
 -- Function File: Y = speye (M, N)
 -- Function File: Y = speye (SZ)
     Return a sparse identity matrix.  This is significantly more
     efficient than `sparse (eye (M))' as the full matrix is not
     constructed.

     Called with a single argument a square matrix of size M by M is
     created.  Otherwise a matrix of M by N is created.  If called with
     a single vector argument, this argument is taken to be the size of
     the matrix to create.

 -- Function File: R = spones (S)
     Replace the non-zero entries of S with ones.  This creates a
     sparse matrix with the same structure as S.

 -- Function File:  sprand (M, N, D)
 -- Function File:  sprand (S)
     Generate a random sparse matrix.  The size of the matrix will be M
     by N, with a density of values given by D.  D should be between 0
     and 1.  Values will be uniformly distributed between 0 and 1.

     If called with a single matrix argument, a random sparse matrix is
     generated wherever the matrix S is non-zero.

     *See also:* *note sprandn: doc-sprandn, *note sprandsym:
     doc-sprandsym.

 -- Function File:  sprandn (M, N, D)
 -- Function File:  sprandn (S)
     Generate a random sparse matrix.  The size of the matrix will be M
     by N, with a density of values given by D.  D should be between 0
     and 1. Values will be normally distributed with mean of zero and
     variance 1.

     If called with a single matrix argument, a random sparse matrix is
     generated wherever the matrix S is non-zero.

     *See also:* *note sprand: doc-sprand, *note sprandsym:
     doc-sprandsym.

 -- Function File:  sprandsym (N, D)
 -- Function File:  sprandsym (S)
     Generate a symmetric random sparse matrix.  The size of the matrix
     will be N by N, with a density of values given by D.  D should be
     between 0 and 1. Values will be normally distributed with mean of
     zero and variance 1.

     If called with a single matrix argument, a random sparse matrix is
     generated wherever the matrix S is non-zero in its lower
     triangular part.

     *See also:* *note sprand: doc-sprand, *note sprandn: doc-sprandn.

   The recommended way for the user to create a sparse matrix, is to
create two vectors containing the row and column index of the data and
a third vector of the same size containing the data to be stored.  For
example,

       ri = ci = d = [];
       for j = 1:c
         ri = [ri; randperm(r,n)'];
         ci = [ci; j*ones(n,1)];
         d = [d; rand(n,1)];
       endfor
       s = sparse (ri, ci, d, r, c);

creates an R-by-C sparse matrix with a random distribution of N (<R)
elements per column.  The elements of the vectors do not need to be
sorted in any particular order as Octave will sort them prior to
storing the data.  However, pre-sorting the data will make the creation
of the sparse matrix faster.

   The function "spconvert" takes a three or four column real matrix.
The first two columns represent the row and column index respectively
and the third and four columns, the real and imaginary parts of the
sparse matrix.  The matrix can contain zero elements and the elements
can be sorted in any order.  Adding zero elements is a convenient way
to define the size of the sparse matrix.  For example:

     s = spconvert ([1 2 3 4; 1 3 4 4; 1 2 3 0]')
     => Compressed Column Sparse (rows=4, cols=4, nnz=3)
           (1 , 1) -> 1
           (2 , 3) -> 2
           (3 , 4) -> 3

   An example of creating and filling a matrix might be

     k = 5;
     nz = r * k;
     s = spalloc (r, c, nz)
     for j = 1:c
       idx = randperm (r);
       s (:, j) = [zeros(r - k, 1); ...
             rand(k, 1)] (idx);
     endfor

   It should be noted, that due to the way that the Octave assignment
functions are written that the assignment will reallocate the memory
used by the sparse matrix at each iteration of the above loop.
Therefore the "spalloc" function ignores the NZ argument and does not
pre-assign the memory for the matrix.  Therefore, it is vitally
important that code using to above structure should be vectorized as
much as possible to minimize the number of assignments and reduce the
number of memory allocations.

 -- Loadable Function: FM = full (SM)
     Return a full storage matrix from a sparse, diagonal, permutation
     matrix or a range.

     *See also:* *note sparse: doc-sparse.

 -- Loadable Function: S = spalloc (M, N, NZ)
     Create an M-by-N sparse matrix with pre-allocated space for at
     most NZ nonzero elements.  This is useful for building the matrix
     incrementally by a sequence of indexed assignments.  Subsequent
     indexed assignments will reuse the pre-allocated memory, provided
     they are of one of the simple forms

        * `S(I:J) = X'

        * `S(:,I:J) = X'

        * `S(K:L,I:J) = X'

     and that the following conditions are met:

        * the assignment does not decrease nnz(S).

        * after the assignment, nnz(S) does not exceed NZ.

        * no index is out of bounds.

     Partial movement of data may still occur, but in general the
     assignment will be more memory and time-efficient under these
     circumstances.  In particular, it is possible to efficiently build
     a pre-allocated sparse matrix from contiguous block of columns.

     The amount of pre-allocated memory for a given matrix may be
     queried using the function `nzmax'.

     *See also:* *note nzmax: doc-nzmax, *note sparse: doc-sparse.

 -- Loadable Function: S = sparse (A)
 -- Loadable Function: S = sparse (I, J, SV, M, N, NZMAX)
 -- Loadable Function: S = sparse (I, J, SV)
 -- Loadable Function: S = sparse (I, J, S, M, N, "unique")
 -- Loadable Function: S = sparse (M, N)
     Create a sparse matrix from the full matrix or row, column, value
     triplets.  If A is a full matrix, convert it to a sparse matrix
     representation, removing all zero values in the process.

     Given the integer index vectors I and J, a 1-by-`nnz' vector of
     real of complex values SV, overall dimensions M and N of the
     sparse matrix.  The argument `nzmax' is ignored but accepted for
     compatibility with MATLAB.  If M or N are not specified their
     values are derived from the maximum index in the vectors I and J
     as given by `M = max (I)', `N = max (J)'.

     *Note*: if multiple values are specified with the same I, J
     indices, the corresponding values in S will be added.

     The following are all equivalent:

          s = sparse (i, j, s, m, n)
          s = sparse (i, j, s, m, n, "summation")
          s = sparse (i, j, s, m, n, "sum")

     Given the option "unique". if more than two values are specified
     for the same I, J indices, the last specified value will be used.

     `sparse(M, N)' is equivalent to `sparse ([], [], [], M, N, 0)'

     If any of SV, I or J are scalars, they are expanded to have a
     common size.

     *See also:* *note full: doc-full.

 -- Function File: X = spconvert (M)
     This function converts for a simple sparse matrix format easily
     produced by other programs into Octave's internal sparse format.
     The input X is either a 3 or 4 column real matrix, containing the
     row, column, real and imaginary parts of the elements of the
     sparse matrix.  An element with a zero real and imaginary part can
     be used to force a particular matrix size.

   The above problem of memory reallocation can be avoided in
oct-files.  However, the construction of a sparse matrix from an
oct-file is more complex than can be discussed here, and you are
referred to chapter *note Dynamically Linked Functions::, to have a
full description of the techniques involved.


File: octave.info,  Node: Information,  Next: Operators and Functions,  Prev: Creating Sparse Matrices,  Up: Basics

22.1.3 Finding out Information about Sparse Matrices
----------------------------------------------------

There are a number of functions that allow information concerning
sparse matrices to be obtained.  The most basic of these is "issparse"
that identifies whether a particular Octave object is in fact a sparse
matrix.

   Another very basic function is "nnz" that returns the number of
non-zero entries there are in a sparse matrix, while the function
"nzmax" returns the amount of storage allocated to the sparse matrix.
Note that Octave tends to crop unused memory at the first opportunity
for sparse objects.  There are some cases of user created sparse
objects where the value returned by "nzmax" will not be the same as
"nnz", but in general they will give the same result.  The function
"spstats" returns some basic statistics on the columns of a sparse
matrix including the number of elements, the mean and the variance of
each column.

 -- Loadable Function:  issparse (X)
     Return true if X is a sparse matrix.

     *See also:* *note ismatrix: doc-ismatrix.

 -- Built-in Function: SCALAR = nnz (A)
     Return the number of non zero elements in A.

     *See also:* *note sparse: doc-sparse.

 -- Function File:  nonzeros (S)
     Return a vector of the non-zero values of the sparse matrix S.

 -- Built-in Function: SCALAR = nzmax (SM)
     Return the amount of storage allocated to the sparse matrix SM.
     Note that Octave tends to crop unused memory at the first
     opportunity for sparse objects.  There are some cases of user
     created sparse objects where the value returned by "nzmax" will
     not be the same as "nnz", but in general they will give the same
     result.

     *See also:* *note sparse: doc-sparse, *note spalloc: doc-spalloc.

 -- Function File: [COUNT, MEAN, VAR] = spstats (S)
 -- Function File: [COUNT, MEAN, VAR] = spstats (S, J)
     Return the stats for the non-zero elements of the sparse matrix S.
     COUNT is the number of non-zeros in each column, MEAN is the mean
     of the non-zeros in each column, and VAR is the variance of the
     non-zeros in each column.

     Called with two input arguments, if S is the data and J is the bin
     number for the data, compute the stats for each bin.  In this
     case, bins can contain data values of zero, whereas with `spstats
     (S)' the zeros may disappear.

   When solving linear equations involving sparse matrices Octave
determines the means to solve the equation based on the type of the
matrix as discussed in *note Sparse Linear Algebra::.  Octave probes the
matrix type when the div (/) or ldiv (\) operator is first used with
the matrix and then caches the type.  However the "matrix_type"
function can be used to determine the type of the sparse matrix prior
to use of the div or ldiv operators.  For example,

     a = tril (sprandn(1024, 1024, 0.02), -1) ...
         + speye(1024);
     matrix_type (a);
     ans = Lower

shows that Octave correctly determines the matrix type for lower
triangular matrices.  "matrix_type" can also be used to force the type
of a matrix to be a particular type.  For example:

     a = matrix_type (tril (sprandn (1024, ...
        1024, 0.02), -1) + speye(1024), 'Lower');

   This allows the cost of determining the matrix type to be avoided.
However, incorrectly defining the matrix type will result in incorrect
results from solutions of linear equations, and so it is entirely the
responsibility of the user to correctly identify the matrix type

   There are several graphical means of finding out information about
sparse matrices.  The first is the "spy" command, which displays the
structure of the non-zero elements of the matrix.  *Note
fig:spmatrix::, for an example of the use of "spy".  More advanced
graphical information can be obtained with the "treeplot", "etreeplot"
and "gplot" commands.


            |  * *
            |  * * * *
            |    * *   * *
            |    *   *     * *
          5 -      *   *       * *
            |      *     *         * *
            |        *     *           * *
            |        *       *             *
            |          *       *
         10 -          *         *
            |            *         *
            |            *           *
            |              *           *
            |              *             *
         15 -                *             *
            |----------|---------|---------|
                       5        10        15
Figure 22.1: Structure of simple sparse matrix.

   One use of sparse matrices is in graph theory, where the
interconnections between nodes are represented as an adjacency matrix.
That is, if the i-th node in a graph is connected to the j-th node.
Then the ij-th node (and in the case of undirected graphs the ji-th
node) of the sparse adjacency matrix is non-zero.  If each node is then
associated with a set of coordinates, then the "gplot" command can be
used to graphically display the interconnections between nodes.

   As a trivial example of the use of "gplot" consider the example,

     A = sparse([2,6,1,3,2,4,3,5,4,6,1,5],
         [1,1,2,2,3,3,4,4,5,5,6,6],1,6,6);
     xy = [0,4,8,6,4,2;5,0,5,7,5,7]';
     gplot(A,xy)

which creates an adjacency matrix `A' where node 1 is connected to
nodes 2 and 6, node 2 with nodes 1 and 3, etc.  The coordinates of the
nodes are given in the n-by-2 matrix `xy'.

   The dependencies between the nodes of a Cholesky factorization can be
calculated in linear time without explicitly needing to calculate the
Cholesky factorization by the `etree' command.  This command returns
the elimination tree of the matrix and can be displayed graphically by
the command `treeplot(etree(A))' if `A' is symmetric or
`treeplot(etree(A+A'))' otherwise.

 -- Function File:  spy (X)
 -- Function File:  spy (..., MARKERSIZE)
 -- Function File:  spy (..., LINE_SPEC)
     Plot the sparsity pattern of the sparse matrix X.  If the argument
     MARKERSIZE is given as a scalar value, it is used to determine the
     point size in the plot.  If the string LINE_SPEC is given it is
     passed to `plot' and determines the appearance of the plot.

     *See also:* *note plot: doc-plot.

 -- Loadable Function: P = etree (S)
 -- Loadable Function: P = etree (S, TYP)
 -- Loadable Function: [P, Q] = etree (S, TYP)
     Return the elimination tree for the matrix S.  By default S is
     assumed to be symmetric and the symmetric elimination tree is
     returned.  The argument TYP controls whether a symmetric or column
     elimination tree is returned.  Valid values of TYP are 'sym' or
     'col', for symmetric or column elimination tree respectively

     Called with a second argument, `etree' also returns the postorder
     permutations on the tree.

 -- Function File:  etreeplot (A)
 -- Function File:  etreeplot (A, NODE_STYLE, EDGE_STYLE)
     Plot the elimination tree of the matrix A or A+A' if A in not
     symmetric.  The optional parameters NODE_STYLE and EDGE_STYLE
     define the output style.

     *See also:* *note treeplot: doc-treeplot, *note gplot: doc-gplot.

 -- Function File:  gplot (A, XY)
 -- Function File:  gplot (A, XY, LINE_STYLE)
 -- Function File: [X, Y] = gplot (A, XY)
     Plot a graph defined by A and XY in the graph theory sense.  A is
     the adjacency matrix of the array to be plotted and XY is an
     N-by-2 matrix containing the coordinates of the nodes of the graph.

     The optional parameter LINE_STYLE defines the output style for the
     plot.  Called with no output arguments the graph is plotted
     directly.  Otherwise, return the coordinates of the plot in X and
     Y.

     *See also:* *note treeplot: doc-treeplot, *note etreeplot:
     doc-etreeplot, *note spy: doc-spy.

 -- Function File:  treeplot (TREE)
 -- Function File:  treeplot (TREE, NODE_STYLE, EDGE_STYLE)
     Produce a graph of tree or forest.  The first argument is vector of
     predecessors, optional parameters NODE_STYLE and EDGE_STYLE define
     the output style.  The complexity of the algorithm is O(n) in
     terms of is time and memory requirements.

     *See also:* *note etreeplot: doc-etreeplot, *note gplot: doc-gplot.

 -- Function File:  treelayout (TREE)
 -- Function File:  treelayout (TREE, PERMUTATION)
     treelayout lays out a tree or a forest.  The first argument TREE
     is a vector of predecessors, optional parameter PERMUTATION is an
     optional postorder permutation.  The complexity of the algorithm
     is O(n) in terms of time and memory requirements.

     *See also:* *note etreeplot: doc-etreeplot, *note gplot:
     doc-gplot, *note treeplot: doc-treeplot.


File: octave.info,  Node: Operators and Functions,  Prev: Information,  Up: Basics

22.1.4 Basic Operators and Functions on Sparse Matrices
-------------------------------------------------------

* Menu:

* Sparse Functions::
* Return Types of Operators and Functions::
* Mathematical Considerations::


File: octave.info,  Node: Sparse Functions,  Next: Return Types of Operators and Functions,  Up: Operators and Functions

22.1.4.1 Sparse Functions
.........................

Many Octave functions have been overloaded to work with either sparse
or full matrices.  There is no difference in calling convention when
using an overloaded function with a sparse matrix, however, there is
also no access to potentially sparse-specific features.  At any time
the sparse matrix specific version of a function can be used by
explicitly calling its function name.

   The table below lists all of the sparse functions of Octave.  Note
that the names of the specific sparse forms of the functions are
typically the same as the general versions with a "sp" prefix.  In the
table below, and in the rest of this article, the specific sparse
versions of functions are used.

Generate sparse matrices:
     "spalloc", "spdiags", "speye", "sprand",   "sprandn", "sprandsym"

Sparse matrix conversion:
     "full", "sparse", "spconvert"

Manipulate sparse matrices
     "issparse", "nnz", "nonzeros", "nzmax",   "spfun", "spones", "spy"

Graph Theory:
     "etree", "etreeplot", "gplot",   "treeplot"

Sparse matrix reordering:
     "amd", "ccolamd", "colamd", "colperm", "csymamd",   "dmperm",
     "symamd", "randperm", "symrcm"

Linear algebra:
     "condest", "eigs", "matrix_type", "normest", "sprank",
     "spaugment", "svds"

Iterative techniques:
     "luinc", "pcg", "pcr"

Miscellaneous:
     "spparms", "symbfact", "spstats"

   In addition all of the standard Octave mapper functions (i.e., basic
math functions that take a single argument) such as "abs", etc.  can
accept sparse matrices.  The reader is referred to the documentation
supplied with these functions within Octave itself for further details.


File: octave.info,  Node: Return Types of Operators and Functions,  Next: Mathematical Considerations,  Prev: Sparse Functions,  Up: Operators and Functions

22.1.4.2 The Return Types of Operators and Functions
....................................................

The two basic reasons to use sparse matrices are to reduce the memory
usage and to not have to do calculations on zero elements.  The two are
closely related in that the computation time on a sparse matrix operator
or function is roughly linear with the number of non-zero elements.

   Therefore, there is a certain density of non-zero elements of a
matrix where it no longer makes sense to store it as a sparse matrix,
but rather as a full matrix.  For this reason operators and functions
that have a high probability of returning a full matrix will always
return one.  For example adding a scalar constant to a sparse matrix
will almost always make it a full matrix, and so the example,

     speye(3) + 0
     =>   1  0  0
       0  1  0
       0  0  1

returns a full matrix as can be seen.

   Additionally, if `sparse_auto_mutate' is true, all sparse functions
test the amount of memory occupied by the sparse matrix to see if the
amount of storage used is larger than the amount used by the full
equivalent.  Therefore `speye (2) * 1' will return a full matrix as the
memory used is smaller for the full version than the sparse version.

   As all of the mixed operators and functions between full and sparse
matrices exist, in general this does not cause any problems.  However,
one area where it does cause a problem is where a sparse matrix is
promoted to a full matrix, where subsequent operations would resparsify
the matrix.  Such cases are rare, but can be artificially created, for
example `(fliplr(speye(3)) + speye(3)) - speye(3)' gives a full matrix
when it should give a sparse one.  In general, where such cases occur,
they impose only a small memory penalty.

   There is however one known case where this behavior of Octave's
sparse matrices will cause a problem.  That is in the handling of the
"diag" function.  Whether "diag" returns a sparse or full matrix
depending on the type of its input arguments.  So

      a = diag (sparse([1,2,3]), -1);

should return a sparse matrix.  To ensure this actually happens, the
"sparse" function, and other functions based on it like "speye", always
returns a sparse matrix, even if the memory used will be larger than
its full representation.

 -- Built-in Function: VAL = sparse_auto_mutate ()
 -- Built-in Function: OLD_VAL = sparse_auto_mutate (NEW_VAL)
 -- Built-in Function:  sparse_auto_mutate (NEW_VAL, "local")
     Query or set the internal variable that controls whether Octave
     will automatically mutate sparse matrices to full matrices to save
     memory.  For example:

          s = speye (3);
          sparse_auto_mutate (false)
          s(:, 1) = 1;
          typeinfo (s)
          => sparse matrix
          sparse_auto_mutate (true)
          s(1, :) = 1;
          typeinfo (s)
          => matrix

     When called from inside a function with the "local" option, the
     variable is changed locally for the function and any subroutines
     it calls.  The original variable value is restored when exiting
     the function.

   Note that the `sparse_auto_mutate' option is incompatible with
MATLAB, and so it is off by default.


File: octave.info,  Node: Mathematical Considerations,  Prev: Return Types of Operators and Functions,  Up: Operators and Functions

22.1.4.3 Mathematical Considerations
....................................

The attempt has been made to make sparse matrices behave in exactly the
same manner as there full counterparts.  However, there are certain
differences and especially differences with other products sparse
implementations.

   Firstly, the "./" and ".^" operators must be used with care.
Consider what the examples

       s = speye (4);
       a1 = s .^ 2;
       a2 = s .^ s;
       a3 = s .^ -2;
       a4 = s ./ 2;
       a5 = 2 ./ s;
       a6 = s ./ s;

will give.  The first example of S raised to the power of 2 causes no
problems.  However S raised element-wise to itself involves a large
number of terms `0 .^ 0' which is 1. There `S .^ S' is a full matrix.

   Likewise `S .^ -2' involves terms like `0 .^ -2' which is infinity,
and so `S .^ -2' is equally a full matrix.

   For the "./" operator `S ./ 2' has no problems, but `2 ./ S'
involves a large number of infinity terms as well and is equally a full
matrix.  The case of `S ./ S' involves terms like `0 ./ 0' which is a
`NaN' and so this is equally a full matrix with the zero elements of S
filled with `NaN' values.

   The above behavior is consistent with full matrices, but is not
consistent with sparse implementations in other products.

   A particular problem of sparse matrices comes about due to the fact
that as the zeros are not stored, the sign-bit of these zeros is
equally not stored.  In certain cases the sign-bit of zero is
important.  For example:

      a = 0 ./ [-1, 1; 1, -1];
      b = 1 ./ a
      => -Inf            Inf
          Inf           -Inf
      c = 1 ./ sparse (a)
      =>  Inf            Inf
          Inf            Inf

   To correct this behavior would mean that zero elements with a
negative sign-bit would need to be stored in the matrix to ensure that
their sign-bit was respected.  This is not done at this time, for
reasons of efficiency, and so the user is warned that calculations
where the sign-bit of zero is important must not be done using sparse
matrices.

   In general any function or operator used on a sparse matrix will
result in a sparse matrix with the same or a larger number of non-zero
elements than the original matrix.  This is particularly true for the
important case of sparse matrix factorizations.  The usual way to
address this is to reorder the matrix, such that its factorization is
sparser than the factorization of the original matrix.  That is the
factorization of `L * U = P * S * Q' has sparser terms `L' and `U' than
the equivalent factorization `L * U = S'.

   Several functions are available to reorder depending on the type of
the matrix to be factorized.  If the matrix is symmetric
positive-definite, then "symamd" or "csymamd" should be used.  Otherwise
"amd", "colamd" or "ccolamd" should be used.  For completeness the
reordering functions "colperm" and "randperm" are also available.

   *Note fig:simplematrix::, for an example of the structure of a simple
positive definite matrix.


            |  * *
            |  * * * *
            |    * *   * *
            |    *   *     * *
          5 -      *   *       * *
            |      *     *         * *
            |        *     *           * *
            |        *       *             *
            |          *       *
         10 -          *         *
            |            *         *
            |            *           *
            |              *           *
            |              *             *
         15 -                *             *
            |----------|---------|---------|
                       5        10        15
Figure 22.2: Structure of simple sparse matrix.

   The standard Cholesky factorization of this matrix can be obtained
by the same command that would be used for a full matrix.  This can be
visualized with the command `r = chol(A); spy(r);'.  *Note
fig:simplechol::.  The original matrix had 43 non-zero terms, while
this Cholesky factorization has 71, with only half of the symmetric
matrix being stored.  This is a significant level of fill in, and
although not an issue for such a small test case, can represents a
large overhead in working with other sparse matrices.

   The appropriate sparsity preserving permutation of the original
matrix is given by "symamd" and the factorization using this reordering
can be visualized using the command `q = symamd(A); r = chol(A(q,q));
spy(r)'.  This gives 29 non-zero terms which is a significant
improvement.

   The Cholesky factorization itself can be used to determine the
appropriate sparsity preserving reordering of the matrix during the
factorization, In that case this might be obtained with three return
arguments as r`[r, p, q] = chol(A); spy(r)'.


            |  * *
            |    * * *
            |      * * * *
            |        * * * * *
          5 -          * * * * * *
            |            * * * * * * *
            |              * * * * * * * *
            |                * * * * * * * *
            |                  * * * * * * *
         10 -                    * * * * * *
            |                      * * * * *
            |                        * * * *
            |                          * * *
            |                            * *
         15 -                              *
            |----------|---------|---------|
                       5        10        15
Figure 22.3: Structure of the un-permuted Cholesky factorization of the
above matrix.


            |  * *
            |    *       *
            |      *   *
            |        * *
          5 -          * *
            |            *                 *
            |              *   *
            |                * *
            |                  *       *
         10 -                    *   *
            |                      * *
            |                        * *
            |                          *   *
            |                            * *
         15 -                              *
            |----------|---------|---------|
                       5        10        15
Figure 22.4: Structure of the permuted Cholesky factorization of the
above matrix.

   In the case of an asymmetric matrix, the appropriate sparsity
preserving permutation is "colamd" and the factorization using this
reordering can be visualized using the command `q = colamd(A); [l, u,
p] = lu(A(:,q)); spy(l+u)'.

   Finally, Octave implicitly reorders the matrix when using the div (/)
and ldiv (\) operators, and so no the user does not need to explicitly
reorder the matrix to maximize performance.

 -- Loadable Function: P = amd (S)
 -- Loadable Function: P = amd (S, OPTS)
     Return the approximate minimum degree permutation of a matrix.
     This permutation such that the Cholesky factorization of `S (P,
     P)' tends to be sparser than the Cholesky factorization of S
     itself.  `amd' is typically faster than `symamd' but serves a
     similar purpose.

     The optional parameter OPTS is a structure that controls the
     behavior of `amd'.  The fields of the structure are

    OPTS.dense
          Determines what `amd' considers to be a dense row or column
          of the input matrix.  Rows or columns with more than `max(16,
          (dense * sqrt (N)' entries, where N is the order of the
          matrix S, are ignored by `amd' during the calculation of the
          permutation The value of dense must be a positive scalar and
          its default value is 10.0

    OPTS.aggressive
          If this value is a non zero scalar, then `amd' performs
          aggressive absorption.  The default is not to perform
          aggressive absorption.

     The author of the code itself is Timothy A. Davis
     <davis@cise.ufl.edu>, University of Florida (see
     `http://www.cise.ufl.edu/research/sparse/amd').

     *See also:* *note symamd: doc-symamd, *note colamd: doc-colamd.

 -- Loadable Function: P = ccolamd (S)
 -- Loadable Function: P = ccolamd (S, KNOBS)
 -- Loadable Function: P = ccolamd (S, KNOBS, CMEMBER)
 -- Loadable Function: [P, STATS] = ccolamd (...)
     Constrained column approximate minimum degree permutation.  `P =
     ccolamd (S)' returns the column approximate minimum degree
     permutation vector for the sparse matrix S.  For a non-symmetric
     matrix S, `S(:, P)' tends to have sparser LU factors than S.
     `chol (S(:, P)' * S(:, P))' also tends to be sparser than `chol
     (S' * S)'.  `P = ccolamd (S, 1)' optimizes the ordering for `lu
     (S(:, P))'.  The ordering is followed by a column elimination tree
     post-ordering.

     KNOBS is an optional 1-element to 5-element input vector, with a
     default value of `[0 10 10 1 0]' if not present or empty.  Entries
     not present are set to their defaults.

    `KNOBS(1)'
          if nonzero, the ordering is optimized for `lu (S(:, p))'.  It
          will be a poor ordering for `chol (S(:, P)' * S(:, P))'.
          This is the most important knob for ccolamd.

    `KNOBS(2)'
          if S is m-by-n, rows with more than `max (16, KNOBS(2) * sqrt
          (n))' entries are ignored.

    `KNOBS(3)'
          columns with more than `max (16, KNOBS(3) * sqrt (min (M,
          N)))' entries are ignored and ordered last in the output
          permutation (subject to the cmember constraints).

    `KNOBS(4)'
          if nonzero, aggressive absorption is performed.

    `KNOBS(5)'
          if nonzero, statistics and knobs are printed.


     CMEMBER is an optional vector of length n.  It defines the
     constraints on the column ordering.  If `CMEMBER(j) = C', then
     column J is in constraint set C (C must be in the range 1 to n).
     In the output permutation P, all columns in set 1 appear first,
     followed by all columns in set 2, and so on.  `CMEMBER =
     ones(1,n)' if not present or empty.  `ccolamd (S, [], 1 : n)'
     returns `1 : n'

     `P = ccolamd (S)' is about the same as `P = colamd (S)'.  KNOBS
     and its default values differ.  `colamd' always does aggressive
     absorption, and it finds an ordering suitable for both `lu (S(:,
     P))' and `chol (S(:, P)' * S(:, P))'; it cannot optimize its
     ordering for `lu (S(:, P))' to the extent that `ccolamd (S, 1)'
     can.

     STATS is an optional 20-element output vector that provides data
     about the ordering and the validity of the input matrix S.
     Ordering statistics are in `STATS(1 : 3)'.  `STATS(1)' and
     `STATS(2)' are the number of dense or empty rows and columns
     ignored by CCOLAMD and `STATS(3)' is the number of garbage
     collections performed on the internal data structure used by
     CCOLAMD (roughly of size `2.2 * nnz (S) + 4 * M + 7 * N' integers).

     `STATS(4 : 7)' provide information if CCOLAMD was able to
     continue.  The matrix is OK if `STATS(4)' is zero, or 1 if
     invalid.  `STATS(5)' is the rightmost column index that is
     unsorted or contains duplicate entries, or zero if no such column
     exists.  `STATS(6)' is the last seen duplicate or out-of-order row
     index in the column index given by `STATS(5)', or zero if no such
     row index exists.  `STATS(7)' is the number of duplicate or
     out-of-order row indices.  `STATS(8 : 20)' is always zero in the
     current version of CCOLAMD (reserved for future use).

     The authors of the code itself are S. Larimore, T. Davis (Univ. of
     Florida) and S. Rajamanickam in collaboration with J. Bilbert and
     E. Ng.  Supported by the National Science Foundation (DMS-9504974,
     DMS-9803599, CCR-0203270), and a grant from Sandia National Lab.
     See `http://www.cise.ufl.edu/research/sparse' for ccolamd,
     csymamd, amd, colamd, symamd, and other related orderings.

     *See also:* *note colamd: doc-colamd, *note csymamd: doc-csymamd.

 -- Loadable Function: P = colamd (S)
 -- Loadable Function: P = colamd (S, KNOBS)
 -- Loadable Function: [P, STATS] = colamd (S)
 -- Loadable Function: [P, STATS] = colamd (S, KNOBS)
     Column approximate minimum degree permutation.  `P = colamd (S)'
     returns the column approximate minimum degree permutation vector
     for the sparse matrix S.  For a non-symmetric matrix S, `S(:,P)'
     tends to have sparser LU factors than S.  The
     Cholesky factorization of `S(:,P)' * S(:,P)' also tends to be
     sparser than that of `S' * S'.

     KNOBS is an optional one- to three-element input vector.  If S is
     m-by-n, then rows with more than `max(16,KNOBS(1)*sqrt(n))'
     entries are ignored.  Columns with more than
     `max(16,KNOBS(2)*sqrt(min(m,n)))' entries are removed prior to
     ordering, and ordered last in the output permutation P.  Only
     completely dense rows or columns are removed if `KNOBS(1)' and
     `KNOBS(2)' are < 0, respectively.  If `KNOBS(3)' is nonzero, STATS
     and KNOBS are printed.  The default is `KNOBS = [10 10 0]'.  Note
     that KNOBS differs from earlier versions of colamd.

     STATS is an optional 20-element output vector that provides data
     about the ordering and the validity of the input matrix S.
     Ordering statistics are in `STATS(1:3)'.  `STATS(1)' and
     `STATS(2)' are the number of dense or empty rows and columns
     ignored by COLAMD and `STATS(3)' is the number of garbage
     collections performed on the internal data structure used by COLAMD
     (roughly of size `2.2 * nnz(S) + 4 * M + 7 * N' integers).

     Octave built-in functions are intended to generate valid sparse
     matrices, with no duplicate entries, with ascending row indices of
     the nonzeros in each column, with a non-negative number of entries
     in each column (!)  and so on.  If a matrix is invalid, then
     COLAMD may or may not be able to continue.  If there are duplicate
     entries (a row index appears two or more times in the same column)
     or if the row indices in a column are out of order, then COLAMD
     can correct these errors by ignoring the duplicate entries and
     sorting each column of its internal copy of the matrix S (the
     input matrix S is not repaired, however).  If a matrix is invalid
     in other ways then COLAMD cannot continue, an error message is
     printed, and no output arguments (P or STATS) are returned.
     COLAMD is thus a simple way to check a sparse matrix to see if it's
     valid.

     `STATS(4:7)' provide information if COLAMD was able to continue.
     The matrix is OK if `STATS(4)' is zero, or 1 if invalid.
     `STATS(5)' is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     `STATS(6)' is the last seen duplicate or out-of-order row index in
     the column index given by `STATS(5)', or zero if no such row index
     exists.  `STATS(7)' is the number of duplicate or out-of-order row
     indices.  `STATS(8:20)' is always zero in the current version of
     COLAMD (reserved for future use).

     The ordering is followed by a column elimination tree
     post-ordering.

     The authors of the code itself are Stefan I. Larimore and Timothy
     A.  Davis <davis@cise.ufl.edu>, University of Florida.  The
     algorithm was developed in collaboration with John Gilbert, Xerox
     PARC, and Esmond Ng, Oak Ridge National Laboratory.  (see
     `http://www.cise.ufl.edu/research/sparse/colamd')

     *See also:* *note colperm: doc-colperm, *note symamd: doc-symamd,
     *note ccolamd: doc-ccolamd.

 -- Function File: P = colperm (S)
     Return the column permutations such that the columns of `S (:, P)'
     are ordered in terms of increase number of non-zero elements.  If
     S is symmetric, then P is chosen such that `S (P, P)' orders the
     rows and columns with increasing number of non zeros elements.

 -- Loadable Function: P = csymamd (S)
 -- Loadable Function: P = csymamd (S, KNOBS)
 -- Loadable Function: P = csymamd (S, KNOBS, CMEMBER)
 -- Loadable Function: [P, STATS] = csymamd (...)
     For a symmetric positive definite matrix S, returns the permutation
     vector P such that `S(P,P)' tends to have a sparser
     Cholesky factor than S.  Sometimes `csymamd' works well for
     symmetric indefinite matrices too.  The matrix S is assumed to be
     symmetric; only the strictly lower triangular part is referenced.
     S must be square.  The ordering is followed by an elimination tree
     post-ordering.

     KNOBS is an optional 1-element to 3-element input vector, with a
     default value of `[10 1 0]' if present or empty.  Entries not
     present are set to their defaults.

    `KNOBS(1)'
          If S is n-by-n, then rows and columns with more than
          `max(16,KNOBS(1)*sqrt(n))' entries are ignored, and ordered
          last in the output permutation (subject to the cmember
          constraints).

    `KNOBS(2)'
          If nonzero, aggressive absorption is performed.

    `KNOBS(3)'
          If nonzero, statistics and knobs are printed.


     CMEMBER is an optional vector of length n. It defines the
     constraints on the ordering.  If `CMEMBER(j) = S', then row/column
     j is in constraint set C (C must be in the range 1 to n).  In the
     output permutation P, rows/columns in set 1 appear first, followed
     by all rows/columns in set 2, and so on.  `CMEMBER = ones(1,n)' if
     not present or empty.  `csymamd(S,[],1:n)' returns `1:n'.

     `P = csymamd(S)' is about the same as `P = symamd(S)'.  KNOBS and
     its default values differ.

     `STATS(4:7)' provide information if CCOLAMD was able to continue.
     The matrix is OK if `STATS(4)' is zero, or 1 if invalid.
     `STATS(5)' is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     `STATS(6)' is the last seen duplicate or out-of-order row index in
     the column index given by `STATS(5)', or zero if no such row index
     exists.  `STATS(7)' is the number of duplicate or out-of-order row
     indices.  `STATS(8:20)' is always zero in the current version of
     CCOLAMD (reserved for future use).

     The authors of the code itself are S. Larimore, T. Davis (Uni of
     Florida) and S. Rajamanickam in collaboration with J. Bilbert and
     E. Ng.  Supported by the National Science Foundation (DMS-9504974,
     DMS-9803599, CCR-0203270), and a grant from Sandia National Lab.
     See `http://www.cise.ufl.edu/research/sparse' for ccolamd,
     csymamd, amd, colamd, symamd, and other related orderings.

     *See also:* *note symamd: doc-symamd, *note ccolamd: doc-ccolamd.

 -- Loadable Function: P = dmperm (S)
 -- Loadable Function: [P, Q, R, S] = dmperm (S)
     Perform a Dulmage-Mendelsohn permutation of the sparse matrix S.
     With a single output argument `dmperm' performs the row
     permutations P such that `S(P,:)' has no zero elements on the
     diagonal.

     Called with two or more output arguments, returns the row and
     column permutations, such that `S(P, Q)' is in block triangular
     form.  The values of R and S define the boundaries of the blocks.
     If S is square then `R == S'.

     The method used is described in: A. Pothen & C.-J. Fan. `Computing
     the Block Triangular Form of a Sparse Matrix'. ACM Trans. Math.
     Software, 16(4):303-324, 1990.

     *See also:* *note colamd: doc-colamd, *note ccolamd: doc-ccolamd.

 -- Loadable Function: P = symamd (S)
 -- Loadable Function: P = symamd (S, KNOBS)
 -- Loadable Function: [P, STATS] = symamd (S)
 -- Loadable Function: [P, STATS] = symamd (S, KNOBS)
     For a symmetric positive definite matrix S, returns the permutation
     vector p such that `S(P, P)' tends to have a sparser
     Cholesky factor than S.  Sometimes `symamd' works well for
     symmetric indefinite matrices too.  The matrix S is assumed to be
     symmetric; only the strictly lower triangular part is referenced.
     S must be square.

     KNOBS is an optional one- to two-element input vector.  If S is
     n-by-n, then rows and columns with more than
     `max(16,KNOBS(1)*sqrt(n))' entries are removed prior to ordering,
     and ordered last in the output permutation P.  No rows/columns are
     removed if `KNOBS(1) < 0'.  If `KNOBS (2)' is nonzero, `stats' and
     KNOBS are printed.  The default is `KNOBS = [10 0]'.  Note that
     KNOBS differs from earlier versions of symamd.

     STATS is an optional 20-element output vector that provides data
     about the ordering and the validity of the input matrix S.
     Ordering statistics are in `STATS(1:3)'.  `STATS(1) = STATS(2)' is
     the number of dense or empty rows and columns ignored by SYMAMD
     and `STATS(3)' is the number of garbage collections performed on
     the internal data structure used by SYMAMD (roughly of size `8.4 *
     nnz (tril (S, -1)) + 9 * N' integers).

     Octave built-in functions are intended to generate valid sparse
     matrices, with no duplicate entries, with ascending row indices of
     the nonzeros in each column, with a non-negative number of entries
     in each column (!)  and so on.  If a matrix is invalid, then
     SYMAMD may or may not be able to continue.  If there are duplicate
     entries (a row index appears two or more times in the same column)
     or if the row indices in a column are out of order, then SYMAMD
     can correct these errors by ignoring the duplicate entries and
     sorting each column of its internal copy of the matrix S (the
     input matrix S is not repaired, however).  If a matrix is invalid
     in other ways then SYMAMD cannot continue, an error message is
     printed, and no output arguments (P or STATS) are returned.
     SYMAMD is thus a simple way to check a sparse matrix to see if
     it's valid.

     `STATS(4:7)' provide information if SYMAMD was able to continue.
     The matrix is OK if `STATS (4)' is zero, or 1 if invalid.
     `STATS(5)' is the rightmost column index that is unsorted or
     contains duplicate entries, or zero if no such column exists.
     `STATS(6)' is the last seen duplicate or out-of-order row index in
     the column index given by `STATS(5)', or zero if no such row index
     exists.  `STATS(7)' is the number of duplicate or out-of-order row
     indices.  `STATS(8:20)' is always zero in the current version of
     SYMAMD (reserved for future use).

     The ordering is followed by a column elimination tree
     post-ordering.

     The authors of the code itself are Stefan I. Larimore and Timothy
     A.  Davis <davis@cise.ufl.edu>, University of Florida.  The
     algorithm was developed in collaboration with John Gilbert, Xerox
     PARC, and Esmond Ng, Oak Ridge National Laboratory.  (see
     `http://www.cise.ufl.edu/research/sparse/colamd')

     *See also:* *note colperm: doc-colperm, *note colamd: doc-colamd.

 -- Loadable Function: P = symrcm (S)
     Return the symmetric reverse Cuthill-McKee permutation of S.  P is
     a permutation vector such that `S(P, P)' tends to have its
     diagonal elements closer to the diagonal than S.  This is a good
     preordering for LU or Cholesky factorization of matrices that come
     from 'long, skinny' problems.  It works for both symmetric and
     asymmetric S.

     The algorithm represents a heuristic approach to the NP-complete
     bandwidth minimization problem.  The implementation is based in the
     descriptions found in

     E. Cuthill, J. McKee. `Reducing the Bandwidth of Sparse Symmetric
     Matrices'. Proceedings of the 24th ACM National Conference, 157-172
     1969, Brandon Press, New Jersey.

     A. George, J.W.H. Liu. `Computer Solution of Large Sparse Positive
     Definite Systems', Prentice Hall Series in Computational
     Mathematics, ISBN 0-13-165274-5, 1981.

     *See also:* *note colperm: doc-colperm, *note colamd: doc-colamd,
     *note symamd: doc-symamd.


File: octave.info,  Node: Sparse Linear Algebra,  Next: Iterative Techniques,  Prev: Basics,  Up: Sparse Matrices

22.2 Linear Algebra on Sparse Matrices
======================================

Octave includes a polymorphic solver for sparse matrices, where the
exact solver used to factorize the matrix, depends on the properties of
the sparse matrix itself.  Generally, the cost of determining the
matrix type is small relative to the cost of factorizing the matrix
itself, but in any case the matrix type is cached once it is
calculated, so that it is not re-determined each time it is used in a
linear equation.

   The selection tree for how the linear equation is solve is

  1. If the matrix is diagonal, solve directly and goto 8

  2. If the matrix is a permuted diagonal, solve directly taking into
     account the permutations.  Goto 8

  3. If the matrix is square, banded and if the band density is less
     than that given by `spparms ("bandden")' continue, else goto 4.

       a. If the matrix is tridiagonal and the right-hand side is not
          sparse continue, else goto 3b.

            1. If the matrix is Hermitian, with a positive real
               diagonal, attempt       Cholesky factorization using
               LAPACK xPTSV.

            2. If the above failed or the matrix is not Hermitian with
               a positive       real diagonal use Gaussian elimination
               with pivoting using       LAPACK xGTSV, and goto 8.

       b. If the matrix is Hermitian with a positive real diagonal,
          attempt       Cholesky factorization using LAPACK xPBTRF.

       c. if the above failed or the matrix is not Hermitian with a
          positive       real diagonal use Gaussian elimination with
          pivoting using       LAPACK xGBTRF, and goto 8.

  4. If the matrix is upper or lower triangular perform a sparse forward
     or backward substitution, and goto 8

  5. If the matrix is an upper triangular matrix with column
     permutations or lower triangular matrix with row permutations,
     perform a sparse forward or backward substitution, and goto 8

  6. If the matrix is square, Hermitian with a real positive diagonal,
     attempt sparse Cholesky factorization using CHOLMOD.

  7. If the sparse Cholesky factorization failed or the matrix is not
     Hermitian with a real positive diagonal, and the matrix is square,
     factorize using UMFPACK.

  8. If the matrix is not square, or any of the previous solvers flags
     a singular or near singular matrix, find a minimum norm solution
     using CXSPARSE(1).

   The band density is defined as the number of non-zero values in the
matrix divided by the number of non-zero values in the matrix.  The
banded matrix solvers can be entirely disabled by using "spparms" to
set `bandden' to 1 (i.e., `spparms ("bandden", 1)').

   The QR solver factorizes the problem with a Dulmage-Mendelsohn, to
separate the problem into blocks that can be treated as over-determined,
multiple well determined blocks, and a final over-determined block.  For
matrices with blocks of strongly connected nodes this is a big win as
LU decomposition can be used for many blocks.  It also significantly
improves the chance of finding a solution to over-determined problems
rather than just returning a vector of "NaN"'s.

   All of the solvers above, can calculate an estimate of the condition
number.  This can be used to detect numerical stability problems in the
solution and force a minimum norm solution to be used.  However, for
narrow banded, triangular or diagonal matrices, the cost of calculating
the condition number is significant, and can in fact exceed the cost of
factoring the matrix.  Therefore the condition number is not calculated
in these cases, and Octave relies on simpler techniques to detect
singular matrices or the underlying LAPACK code in the case of banded
matrices.

   The user can force the type of the matrix with the `matrix_type'
function.  This overcomes the cost of discovering the type of the
matrix.  However, it should be noted that identifying the type of the
matrix incorrectly will lead to unpredictable results, and so
`matrix_type' should be used with care.

 -- Function File: N = normest (A)
 -- Function File: N = normest (A, TOL)
 -- Function File: [N, C] = normest (...)
     Estimate the 2-norm of the matrix A using a power series analysis.
     This is typically used for large matrices, where the cost of
     calculating `norm (A)' is prohibitive and an approximation to the
     2-norm is acceptable.

     TOL is the tolerance to which the 2-norm is calculated.  By default
     TOL is 1e-6.  C returns the number of iterations needed for
     `normest' to converge.

 -- Function File: [EST, V, W, ITER] = onenormest (A, T)
 -- Function File: [EST, V, W, ITER] = onenormest (APPLY, APPLY_T, N, T)
     Apply Higham and Tisseur's randomized block 1-norm estimator to
     matrix A using T test vectors.  If T exceeds 5, then only 5 test
     vectors are used.

     If the matrix is not explicit, e.g., when estimating the norm of
     `inv (A)' given an LU factorization, `onenormest' applies A and
     its conjugate transpose through a pair of functions APPLY and
     APPLY_T, respectively, to a dense matrix of size N by T.  The
     implicit version requires an explicit dimension N.

     Returns the norm estimate EST, two vectors V and W related by norm
     `(W, 1) = EST * norm (V, 1)', and the number of iterations ITER.
     The number of iterations is limited to 10 and is at least 2.

     References:
        * N.J. Higham and F. Tisseur, `A Block Algorithm for Matrix
          1-Norm Estimation, with an Application to 1-Norm
          Pseudospectra'. SIMAX vol 21, no 4, pp 1185-1201.
          `http://dx.doi.org/10.1137/S0895479899356080'

        * N.J. Higham and F. Tisseur, `A Block Algorithm for Matrix
          1-Norm Estimation, with an Application to 1-Norm
          Pseudospectra'. `http://citeseer.ist.psu.edu/223007.html'

     *See also:* *note condest: doc-condest, *note norm: doc-norm,
     *note cond: doc-cond.

 -- Function File:  condest (A)
 -- Function File:  condest (A, T)
 -- Function File: [EST, V] = condest (...)
 -- Function File: [EST, V] = condest (A, SOLVE, SOLVE_T, T)
 -- Function File: [EST, V] = condest (APPLY, APPLY_T, SOLVE, SOLVE_T,
          N, T)
     Estimate the 1-norm condition number of a matrix A using T test
     vectors using a randomized 1-norm estimator.  If T exceeds 5, then
     only 5 test vectors are used.

     If the matrix is not explicit, e.g., when estimating the condition
     number of A given an LU factorization, `condest' uses the
     following functions:

    APPLY
          `A*x' for a matrix `x' of size N by T.

    APPLY_T
          `A'*x' for a matrix `x' of size N by T.

    SOLVE
          `A \ b' for a matrix `b' of size N by T.

    SOLVE_T
          `A' \ b' for a matrix `b' of size N by T.

     The implicit version requires an explicit dimension N.

     `condest' uses a randomized algorithm to approximate the 1-norms.

     `condest' returns the 1-norm condition estimate EST and a vector V
     satisfying `norm (A*v, 1) == norm (A, 1) * norm (V, 1) / EST'.
     When EST is large, V is an approximate null vector.

     References:
        * N.J. Higham and F. Tisseur, `A Block Algorithm for Matrix
          1-Norm Estimation, with an Application to 1-Norm
          Pseudospectra'. SIMAX vol 21, no 4, pp 1185-1201.
          `http://dx.doi.org/10.1137/S0895479899356080'

        * N.J. Higham and F. Tisseur, `A Block Algorithm for Matrix
          1-Norm Estimation, with an Application to 1-Norm
          Pseudospectra'. `http://citeseer.ist.psu.edu/223007.html'

     *See also:* *note cond: doc-cond, *note norm: doc-norm, *note
     onenormest: doc-onenormest.

 -- Loadable Function:   spparms ()
 -- Loadable Function: VALS = spparms ()
 -- Loadable Function: [KEYS, VALS] = spparms ()
 -- Loadable Function: VAL = spparms (KEY)
 -- Loadable Function:   spparms (VALS)
 -- Loadable Function:   spparms ('defaults')
 -- Loadable Function:   spparms ('tight')
 -- Loadable Function:   spparms (KEY, VAL)
     Query or set the parameters used by the sparse solvers and
     factorization functions.  The first four calls above get
     information about the current settings, while the others change
     the current settings.  The parameters are stored as pairs of keys
     and values, where the values are all floats and the keys are one
     of the following strings:

    `spumoni'
          Printing level of debugging information of the solvers
          (default 0)

    `ths_rel'
          Included for compatibility.  Not used.  (default 1)

    `ths_abs'
          Included for compatibility.  Not used.  (default 1)

    `exact_d'
          Included for compatibility.  Not used.  (default 0)

    `supernd'
          Included for compatibility.  Not used.  (default 3)

    `rreduce'
          Included for compatibility.  Not used.  (default 3)

    `wh_frac'
          Included for compatibility.  Not used.  (default 0.5)

    `autommd'
          Flag whether the LU/QR and the '\' and '/' operators will
          automatically use the sparsity preserving mmd functions
          (default 1)

    `autoamd'
          Flag whether the LU and the '\' and '/' operators will
          automatically use the sparsity preserving amd functions
          (default 1)

    `piv_tol'
          The pivot tolerance of the UMFPACK solvers (default 0.1)

    `sym_tol'
          The pivot tolerance of the UMFPACK symmetric solvers (default
          0.001)

    `bandden'
          The density of non-zero elements in a banded matrix before it
          is treated by the LAPACK banded solvers (default 0.5)

    `umfpack'
          Flag whether the UMFPACK or mmd solvers are used for the LU,
          '\' and '/' operations (default 1)

     The value of individual keys can be set with `spparms (KEY, VAL)'.
     The default values can be restored with the special keyword
     'defaults'.  The special keyword 'tight' can be used to set the
     mmd solvers to attempt a sparser solution at the potential cost of
     longer running time.

 -- Loadable Function: P = sprank (S)
     Calculate the structural rank of the sparse matrix S.  Note that
     only the structure of the matrix is used in this calculation based
     on a Dulmage-Mendelsohn permutation to block triangular form.  As
     such the numerical rank of the matrix S is bounded by `sprank (S)
     >= rank (S)'.  Ignoring floating point errors `sprank (S) == rank
     (S)'.

     *See also:* *note dmperm: doc-dmperm.

 -- Loadable Function: [COUNT, H, PARENT, POST, R] = symbfact (S)
 -- Loadable Function: [...] = symbfact (S, TYP)
 -- Loadable Function: [...] = symbfact (S, TYP, MODE)
     Perform a symbolic factorization analysis on the sparse matrix S.
     Where

    S
          S is a complex or real sparse matrix.

    TYP
          Is the type of the factorization and can be one of

         `sym'
               Factorize S.  This is the default.

         `col'
               Factorize `S' * S'.

         `row'
               Factorize S * S'.

         `lo'
               Factorize S'

    MODE
          The default is to return the Cholesky factorization for R,
          and if MODE is 'L', the conjugate transpose of the
          Cholesky factorization is returned.  The conjugate transpose
          version is faster and uses less memory, but returns the same
          values for COUNT, H, PARENT and POST outputs.

     The output variables are

    COUNT
          The row counts of the Cholesky factorization as determined by
          TYP.

    H
          The height of the elimination tree.

    PARENT
          The elimination tree itself.

    POST
          A sparse boolean matrix whose structure is that of the
          Cholesky factorization as determined by TYP.

   For non square matrices, the user can also utilize the `spaugment'
function to find a least squares solution to a linear equation.

 -- Function File: S = spaugment (A, C)
     Create the augmented matrix of A.  This is given by

          [C * eye(M, M),A; A', zeros(N,
          N)]

     This is related to the least squares solution of `A \\ B', by

          S * [ R / C; x] = [B, zeros(N,
          columns(B)]

     where R is the residual error

          R = B - A * X

     As the matrix S is symmetric indefinite it can be factorized with
     `lu', and the minimum norm solution can therefore be found without
     the need for a `qr' factorization.  As the residual error will be
     `zeros (M, M)' for under determined problems, and example can be

          m = 11; n = 10; mn = max(m ,n);
          A = spdiags ([ones(mn,1), 10*ones(mn,1), -ones(mn,1)],
                       [-1, 0, 1], m, n);
          x0 = A \ ones (m,1);
          s = spaugment (A);
          [L, U, P, Q] = lu (s);
          x1 = Q * (U \ (L \ (P  * [ones(m,1); zeros(n,1)])));
          x1 = x1(end - n + 1 : end);

     To find the solution of an overdetermined problem needs an estimate
     of the residual error R and so it is more complex to formulate a
     minimum norm solution using the `spaugment' function.

     In general the left division operator is more stable and faster
     than using the `spaugment' function.

   Finally, the function `eigs' can be used to calculate a limited
number of eigenvalues and eigenvectors based on a selection criteria
and likewise for `svds' which calculates a limited number of singular
values and vectors.

 -- Loadable Function: D = eigs (A)
 -- Loadable Function: D = eigs (A, K)
 -- Loadable Function: D = eigs (A, K, SIGMA)
 -- Loadable Function: D = eigs (A, K, SIGMA, OPTS)
 -- Loadable Function: D = eigs (A, B)
 -- Loadable Function: D = eigs (A, B, K)
 -- Loadable Function: D = eigs (A, B, K, SIGMA)
 -- Loadable Function: D = eigs (A, B, K, SIGMA, OPTS)
 -- Loadable Function: D = eigs (AF, N)
 -- Loadable Function: D = eigs (AF, N, B)
 -- Loadable Function: D = eigs (AF, N, K)
 -- Loadable Function: D = eigs (AF, N, B, K)
 -- Loadable Function: D = eigs (AF, N, K, SIGMA)
 -- Loadable Function: D = eigs (AF, N, B, K, SIGMA)
 -- Loadable Function: D = eigs (AF, N, K, SIGMA, OPTS)
 -- Loadable Function: D = eigs (AF, N, B, K, SIGMA, OPTS)
 -- Loadable Function: [V, D] = eigs (A, ...)
 -- Loadable Function: [V, D] = eigs (AF, N, ...)
 -- Loadable Function: [V, D, FLAG] = eigs (A, ...)
 -- Loadable Function: [V, D, FLAG] = eigs (AF, N, ...)
     Calculate a limited number of eigenvalues and eigenvectors of A,
     based on a selection criteria.  The number of eigenvalues and
     eigenvectors to calculate is given by K and defaults to 6.

     By default, `eigs' solve the equation `A * v = lambda * v', where
     `lambda' is a scalar representing one of the eigenvalues, and `v'
     is the corresponding eigenvector.  If given the positive definite
     matrix B then `eigs' solves the general eigenvalue equation `A * v
     = lambda * B * v'.

     The argument SIGMA determines which eigenvalues are returned.
     SIGMA can be either a scalar or a string.  When SIGMA is a scalar,
     the K eigenvalues closest to SIGMA are returned.  If SIGMA is a
     string, it must have one of the following values.

    'lm'
          Largest Magnitude (default).

    'sm'
          Smallest Magnitude.

    'la'
          Largest Algebraic (valid only for real symmetric problems).

    'sa'
          Smallest Algebraic (valid only for real symmetric problems).

    'be'
          Both Ends, with one more from the high-end if K is odd (valid
          only for real symmetric problems).

    'lr'
          Largest Real part (valid only for complex or unsymmetric
          problems).

    'sr'
          Smallest Real part (valid only for complex or unsymmetric
          problems).

    'li'
          Largest Imaginary part (valid only for complex or unsymmetric
          problems).

    'si'
          Smallest Imaginary part (valid only for complex or
          unsymmetric problems).

     If OPTS is given, it is a structure defining possible options that
     `eigs' should use.  The fields of the OPTS structure are:

    `issym'
          If AF is given, then flags whether the function AF defines a
          symmetric problem.  It is ignored if A is given.  The default
          is false.

    `isreal'
          If AF is given, then flags whether the function AF defines a
          real problem.  It is ignored if A is given.  The default is
          true.

    `tol'
          Defines the required convergence tolerance, calculated as
          `tol * norm (A)'.  The default is `eps'.

    `maxit'
          The maximum number of iterations.  The default is 300.

    `p'
          The number of Lanzcos basis vectors to use.  More vectors
          will result in faster convergence, but a greater use of
          memory.  The optimal value of `p' is problem dependent and
          should be in the range K to N.  The default value is `2 * K'.

    `v0'
          The starting vector for the algorithm.  An initial vector
          close to the final vector will speed up convergence.  The
          default is for ARPACK to randomly generate a starting vector.
          If specified, `v0' must be an N-by-1 vector where `N = rows
          (A)'

    `disp'
          The level of diagnostic printout (0|1|2).  If `disp' is 0 then
          diagnostics are disabled.  The default value is 0.

    `cholB'
          Flag if `chol (B)' is passed rather than B.  The default is
          false.

    `permB'
          The permutation vector of the Cholesky factorization of B if
          `cholB' is true.  That is `chol (B(permB, permB))'.  The
          default is `1:N'.

     It is also possible to represent A by a function denoted AF.  AF
     must be followed by a scalar argument N defining the length of the
     vector argument accepted by AF.  AF can be a function handle, an
     inline function, or a string.  When AF is a string it holds the
     name of the function to use.

     AF is a function of the form `y = af (x)' where the required
     return value of AF is determined by the value of SIGMA.  The four
     possible forms are

    `A * x'
          if SIGMA is not given or is a string other than 'sm'.

    `A \ x'
          if SIGMA is 0 or 'sm'.

    `(A - sigma * I) \ x'
          for the standard eigenvalue problem, where `I' is the
          identity matrix of the same size as A.

    `(A - sigma * B) \ x'
          for the general eigenvalue problem.

     The return arguments of `eigs' depend on the number of return
     arguments requested.  With a single return argument, a vector D of
     length K is returned containing the K eigenvalues that have been
     found.  With two return arguments, V is a N-by-K matrix whose
     columns are the K eigenvectors corresponding to the returned
     eigenvalues.  The eigenvalues themselves are returned in D in the
     form of a N-by-K matrix, where the elements on the diagonal are the
     eigenvalues.

     Given a third return argument FLAG, `eigs' returns the status of
     the convergence.  If FLAG is 0 then all eigenvalues have converged.
     Any other value indicates a failure to converge.

     This function is based on the ARPACK package, written by R.
     Lehoucq, K. Maschhoff, D. Sorensen, and C. Yang.  For more
     information see `http://www.caam.rice.edu/software/ARPACK/'.

     *See also:* *note eig: doc-eig, *note svds: doc-svds.

 -- Function File: S = svds (A)
 -- Function File: S = svds (A, K)
 -- Function File: S = svds (A, K, SIGMA)
 -- Function File: S = svds (A, K, SIGMA, OPTS)
 -- Function File: [U, S, V] = svds (...)
 -- Function File: [U, S, V, FLAG] = svds (...)
     Find a few singular values of the matrix A.  The singular values
     are calculated using

          [M, N] = size(A)
          S = eigs([sparse(M, M), A;
                              A', sparse(N, N)])

     The eigenvalues returned by `eigs' correspond to the singular
     values of A.  The number of singular values to calculate is given
     by K and defaults to 6.

     The argument SIGMA specifies which singular values to find.  When
     SIGMA is the string 'L', the default, the largest singular values
     of A are found.  Otherwise, SIGMA must be a real scalar and the
     singular values closest to SIGMA are found.  As a corollary,
     `SIGMA = 0' finds the smallest singular values.  Note that for
     relatively small values of SIGMA, there is a chance that the
     requested number of singular values will not be found.  In that
     case SIGMA should be increased.

     OPTS is a structure defining options that `svds' will pass to
     `eigs'.  The possible fields of this structure are documented in
     `eigs'.  By default, `svds' sets the following three fields:

    `tol'
          The required convergence tolerance for the singular values.
          The default value is 1e-10.  `eigs' is passed `TOL / sqrt(2)'.

    `maxit'
          The maximum number of iterations.  The default is 300.

    `disp'
          The level of diagnostic printout (0|1|2).  If `disp' is 0 then
          diagnostics are disabled.  The default value is 0.

     If more than one output is requested then `svds' will return an
     approximation of the singular value decomposition of A

          A_approx = U*S*V'

     where A_approx is a matrix of size A but only rank K.

     FLAG returns 0 if the algorithm has succesfully converged, and 1
     otherwise.  The test for convergence is

          norm (A*V - U*S, 1) <= TOL * norm (A, 1)

     `svds' is best for finding only a few singular values from a large
     sparse matrix.  Otherwise, `svd (full(A))' will likely be more
     efficient.

*See also:* *note svd: doc-svd, *note eigs: doc-eigs.

   ---------- Footnotes ----------

   (1) The CHOLMOD, UMFPACK and CXSPARSE packages were written by Tim
Davis and are available at http://www.cise.ufl.edu/research/sparse/


File: octave.info,  Node: Iterative Techniques,  Next: Real Life Example,  Prev: Sparse Linear Algebra,  Up: Sparse Matrices

22.3 Iterative Techniques applied to sparse matrices
====================================================

The left division `\' and right division `/' operators, discussed in
the previous section, use direct solvers to resolve a linear equation
of the form `X = A \ B' or `X = B / A'.  Octave equally includes a
number of functions to solve sparse linear equations using iterative
techniques.

 -- Function File: X = pcg (A, B, TOL, MAXIT, M1, M2, X0, ...)
 -- Function File: [X, FLAG, RELRES, ITER, RESVEC, EIGEST] = pcg (...)
     Solve the linear system of equations `A * X = B' by means of the
     Preconditioned Conjugate Gradient iterative method.  The input
     arguments are

        * A can be either a square (preferably sparse) matrix or a
          function handle, inline function or string containing the name
          of a function which computes `A * X'.  In principle A should
          be symmetric and positive definite; if `pcg' finds A to not
          be positive definite, you will get a warning message and the
          FLAG output parameter will be set.

        * B is the right hand side vector.

        * TOL is the required relative tolerance for the residual error,
          `B - A * X'.  The iteration stops if `norm (B - A * X) <= TOL
          * norm (B - A * X0)'.  If TOL is empty or is omitted, the
          function sets `TOL = 1e-6' by default.

        * MAXIT is the maximum allowable number of iterations; if `[]'
          is supplied for `maxit', or `pcg' has less arguments, a
          default value equal to 20 is used.

        * M = M1 * M2 is the (left) preconditioning matrix, so that the
          iteration is (theoretically) equivalent to solving by `pcg'
          `P * X = M \ B', with `P = M \ A'.  Note that a proper choice
          of the preconditioner may dramatically improve the overall
          performance of the method.  Instead of matrices M1 and M2,
          the user may pass two functions which return the results of
          applying the inverse of M1 and M2 to a vector (usually this
          is the preferred way of using the preconditioner).  If `[]'
          is supplied for M1, or M1 is omitted, no preconditioning is
          applied.  If M2 is omitted, M = M1 will be used as
          preconditioner.

        * X0 is the initial guess.  If X0 is empty or omitted, the
          function sets X0 to a zero vector by default.

     The arguments which follow X0 are treated as parameters, and
     passed in a proper way to any of the functions (A or M) which are
     passed to `pcg'.  See the examples below for further details.  The
     output arguments are

        * X is the computed approximation to the solution of `A * X =
          B'.

        * FLAG reports on the convergence.  `FLAG = 0' means the
          solution converged and the tolerance criterion given by TOL
          is satisfied.  `FLAG = 1' means that the MAXIT limit for the
          iteration count was reached.  `FLAG = 3' reports that the
          (preconditioned) matrix was found not positive definite.

        * RELRES is the ratio of the final residual to its initial
          value, measured in the Euclidean norm.

        * ITER is the actual number of iterations performed.

        * RESVEC describes the convergence history of the method.
          `RESVEC (i,1)' is the Euclidean norm of the residual, and
          `RESVEC (i,2)' is the preconditioned residual norm, after the
          (I-1)-th iteration, `I = 1, 2, ..., ITER+1'.  The
          preconditioned residual norm is defined as `norm (R) ^ 2 = R'
          * (M \ R)' where `R = B - A * X', see also the description of
          M.  If EIGEST is not required, only `RESVEC (:,1)' is
          returned.

        * EIGEST returns the estimate for the smallest `EIGEST (1)' and
          largest `EIGEST (2)' eigenvalues of the preconditioned matrix
          `P = M \ A'.  In particular, if no preconditioning is used,
          the estimates for the extreme eigenvalues of A are returned.
          `EIGEST (1)' is an overestimate and `EIGEST (2)' is an
          underestimate, so that `EIGEST (2) / EIGEST (1)' is a lower
          bound for `cond (P, 2)', which nevertheless in the limit
          should theoretically be equal to the actual value of the
          condition number.  The method which computes EIGEST works
          only for symmetric positive definite A and M, and the user is
          responsible for verifying this assumption.

     Let us consider a trivial problem with a diagonal matrix (we
     exploit the sparsity of A)

               n = 10;
               A = diag (sparse (1:n));
               b = rand (n, 1);
               [l, u, p, q] = luinc (A, 1.e-3);

     EXAMPLE 1: Simplest use of `pcg'

            x = pcg(A,b)

     EXAMPLE 2: `pcg' with a function which computes `A * X'

            function y = apply_a (x)
              y = [1:N]'.*x;
            endfunction

            x = pcg ("apply_a", b)

     EXAMPLE 3: `pcg' with a preconditioner: L * U

          x = pcg (A, b, 1.e-6, 500, l*u);

     EXAMPLE 4: `pcg' with a preconditioner: L * U.  Faster than
     EXAMPLE 3 since lower and upper triangular matrices are easier to
     invert

          x = pcg (A, b, 1.e-6, 500, l, u);

     EXAMPLE 5: Preconditioned iteration, with full diagnostics.  The
     preconditioner (quite strange, because even the original matrix A
     is trivial) is defined as a function

            function y = apply_m (x)
              k = floor (length (x) - 2);
              y = x;
              y(1:k) = x(1:k)./[1:k]';
            endfunction

            [x, flag, relres, iter, resvec, eigest] = ...
                               pcg (A, b, [], [], "apply_m");
            semilogy (1:iter+1, resvec);

     EXAMPLE 6: Finally, a preconditioner which depends on a parameter
     K.

            function y = apply_M (x, varargin)
            K = varargin{1};
            y = x;
            y(1:K) = x(1:K)./[1:K]';
            endfunction

            [x, flag, relres, iter, resvec, eigest] = ...
                 pcg (A, b, [], [], "apply_m", [], [], 3)

     References:

       1. C.T. Kelley, `Iterative Methods for Linear and Nonlinear
          Equations', SIAM, 1995. (the base PCG algorithm)

       2. Y. Saad, `Iterative Methods for Sparse Linear Systems', PWS
          1996.  (condition number estimate from PCG) Revised version
          of this book is available online at
          `http://www-users.cs.umn.edu/~saad/books.html'

     *See also:* *note sparse: doc-sparse, *note pcr: doc-pcr.

 -- Function File: X = pcr (A, B, TOL, MAXIT, M, X0, ...)
 -- Function File: [X, FLAG, RELRES, ITER, RESVEC] = pcr (...)
     Solve the linear system of equations `A * X = B' by means of the
     Preconditioned Conjugate Residuals iterative method.  The input
     arguments are

        * A can be either a square (preferably sparse) matrix or a
          function handle, inline function or string containing the name
          of a function which computes `A * X'.  In principle A should
          be symmetric and non-singular; if `pcr' finds A to be
          numerically singular, you will get a warning message and the
          FLAG output parameter will be set.

        * B is the right hand side vector.

        * TOL is the required relative tolerance for the residual error,
          `B - A * X'.  The iteration stops if `norm (B - A * X) <= TOL
          * norm (B - A * X0)'.  If TOL is empty or is omitted, the
          function sets `TOL = 1e-6' by default.

        * MAXIT is the maximum allowable number of iterations; if `[]'
          is supplied for `maxit', or `pcr' has less arguments, a
          default value equal to 20 is used.

        * M is the (left) preconditioning matrix, so that the iteration
          is (theoretically) equivalent to solving by `pcr' `P * X = M
          \ B', with `P = M \ A'.  Note that a proper choice of the
          preconditioner may dramatically improve the overall
          performance of the method.  Instead of matrix M, the user may
          pass a function which returns the results of applying the
          inverse of M to a vector (usually this is the preferred way
          of using the preconditioner).  If `[]' is supplied for M, or
          M is omitted, no preconditioning is applied.

        * X0 is the initial guess.  If X0 is empty or omitted, the
          function sets X0 to a zero vector by default.

     The arguments which follow X0 are treated as parameters, and
     passed in a proper way to any of the functions (A or M) which are
     passed to `pcr'.  See the examples below for further details.  The
     output arguments are

        * X is the computed approximation to the solution of `A * X =
          B'.

        * FLAG reports on the convergence.  `FLAG = 0' means the
          solution converged and the tolerance criterion given by TOL
          is satisfied.  `FLAG = 1' means that the MAXIT limit for the
          iteration count was reached.  `FLAG = 3' reports t `pcr'
          breakdown, see [1] for details.

        * RELRES is the ratio of the final residual to its initial
          value, measured in the Euclidean norm.

        * ITER is the actual number of iterations performed.

        * RESVEC describes the convergence history of the method, so
          that `RESVEC (i)' contains the Euclidean norms of the
          residual after the (I-1)-th iteration, `I = 1,2, ..., ITER+1'.

     Let us consider a trivial problem with a diagonal matrix (we
     exploit the sparsity of A)

               n = 10;
               A = sparse (diag (1:n));
               b = rand (N, 1);

     EXAMPLE 1: Simplest use of `pcr'

            x = pcr(A, b)

     EXAMPLE 2: `pcr' with a function which computes `A * X'.

            function y = apply_a (x)
              y = [1:10]'.*x;
            endfunction

            x = pcr ("apply_a", b)

     EXAMPLE 3:  Preconditioned iteration, with full diagnostics.  The
     preconditioner (quite strange, because even the original matrix A
     is trivial) is defined as a function

            function y = apply_m (x)
              k = floor (length(x)-2);
              y = x;
              y(1:k) = x(1:k)./[1:k]';
            endfunction

            [x, flag, relres, iter, resvec] = ...
                               pcr (A, b, [], [], "apply_m")
            semilogy([1:iter+1], resvec);

     EXAMPLE 4: Finally, a preconditioner which depends on a parameter
     K.

            function y = apply_m (x, varargin)
              k = varargin{1};
              y = x; y(1:k) = x(1:k)./[1:k]';
            endfunction

            [x, flag, relres, iter, resvec] = ...
                               pcr (A, b, [], [], "apply_m"', [], 3)

     References:

     [1] W. Hackbusch, `Iterative Solution of Large Sparse Systems of
       Equations', section 9.5.4; Springer, 1994

     *See also:* *note sparse: doc-sparse, *note pcg: doc-pcg.

   The speed with which an iterative solver converges to a solution can
be accelerated with the use of a pre-conditioning matrix M.  In this
case the linear equation `M^-1 * X = M^-1 * A \ B' is solved instead.
Typical pre-conditioning matrices are partial factorizations of the
original matrix.

 -- Loadable Function: [L, U, P, Q] = luinc (A, '0')
 -- Loadable Function: [L, U, P, Q] = luinc (A, DROPTOL)
 -- Loadable Function: [L, U, P, Q] = luinc (A, OPTS)
     Produce the incomplete LU factorization of the sparse matrix A.
     Two types of incomplete factorization are possible, and the type
     is determined by the second argument to `luinc'.

     Called with a second argument of '0', the zero-level incomplete
     LU factorization is produced.  This creates a factorization of A
     where the position of the non-zero arguments correspond to the same
     positions as in the matrix A.

     Alternatively, the fill-in of the incomplete LU factorization can
     be controlled through the variable DROPTOL or the structure OPTS.
     The UMFPACK multifrontal factorization code by Tim A.  Davis is
     used for the incomplete LU factorization, (availability
     `http://www.cise.ufl.edu/research/sparse/umfpack/')

     DROPTOL determines the values below which the values in the LU 
     factorization are dropped and replaced by zero.  It must be a
     positive scalar, and any values in the factorization whose
     absolute value are less than this value are dropped, expect if
     leaving them increase the sparsity of the matrix.  Setting DROPTOL
     to zero results in a complete LU factorization which is the
     default.

     OPTS is a structure containing one or more of the fields

    `droptol'
          The drop tolerance as above.  If OPTS only contains `droptol'
          then this is equivalent to using the variable DROPTOL.

    `milu'
          A logical variable flagging whether to use the modified
          incomplete LU  factorization.  In the case that `milu' is
          true, the dropped values are subtracted from the diagonal of
          the matrix U of the factorization.  The default is `false'.

    `udiag'
          A logical variable that flags whether zero elements on the
          diagonal of U should be replaced with DROPTOL to attempt to
          avoid singular factors.  The default is `false'.

    `thresh'
          Defines the pivot threshold in the interval [0,1].  Values
          outside that range are ignored.

     All other fields in OPTS are ignored.  The outputs from `luinc'
     are the same as for `lu'.

     Given the string argument 'vector', `luinc' returns the values of
     P Q as vector values.

     *See also:* *note sparse: doc-sparse, *note lu: doc-lu.


File: octave.info,  Node: Real Life Example,  Prev: Iterative Techniques,  Up: Sparse Matrices

22.4 Real Life Example of the use of Sparse Matrices
====================================================

A common application for sparse matrices is in the solution of Finite
Element Models.  Finite element models allow numerical solution of
partial differential equations that do not have closed form solutions,
typically because of the complex shape of the domain.

   In order to motivate this application, we consider the boundary value
Laplace equation.  This system can model scalar potential fields, such
as heat or electrical potential.  Given a medium Omega with boundary
dOmega.  At all points on the dOmega the boundary conditions are known,
and we wish to calculate the potential in Omega.  Boundary conditions
may specify the potential (Dirichlet boundary condition), its normal
derivative across the boundary (Neumann boundary condition), or a
weighted sum of the potential and its derivative (Cauchy boundary
condition).

   In a thermal model, we want to calculate the temperature in Omega
and know the boundary temperature (Dirichlet condition) or heat flux
(from which we can calculate the Neumann condition by dividing by the
thermal conductivity at the boundary).  Similarly, in an electrical
model, we want to calculate the voltage in Omega and know the boundary
voltage (Dirichlet) or current (Neumann condition after diving by the
electrical conductivity).  In an electrical model, it is common for
much of the boundary to be electrically isolated; this is a Neumann
boundary condition with the current equal to zero.

   The simplest finite element models will divide Omega into simplexes
(triangles in 2D, pyramids in 3D).

   The following example creates a simple rectangular 2-D electrically
conductive medium with 10 V and 20 V imposed on opposite sides
(Dirichlet boundary conditions).  All other edges are electrically
isolated.

        node_y= [1;1.2;1.5;1.8;2]*ones(1,11);
        node_x= ones(5,1)*[1,1.05,1.1,1.2, ...
                  1.3,1.5,1.7,1.8,1.9,1.95,2];
        nodes= [node_x(:), node_y(:)];

        [h,w]= size(node_x);
        elems= [];
        for idx= 1:w-1
          widx= (idx-1)*h;
          elems= [elems; ...
            widx+[(1:h-1);(2:h);h+(1:h-1)]'; ...
            widx+[(2:h);h+(2:h);h+(1:h-1)]' ];
        endfor

        E= size(elems,1); # No. of simplices
        N= size(nodes,1); # No. of vertices
        D= size(elems,2); # dimensions+1

   This creates a N-by-2 matrix `nodes' and a E-by-3 matrix `elems'
with values, which define finite element triangles:

       nodes(1:7,:)'
         1.00 1.00 1.00 1.00 1.00 1.05 1.05 ...
         1.00 1.20 1.50 1.80 2.00 1.00 1.20 ...

       elems(1:7,:)'
         1    2    3    4    2    3    4 ...
         2    3    4    5    7    8    9 ...
         6    7    8    9    6    7    8 ...

   Using a first order FEM, we approximate the electrical conductivity
distribution in Omega as constant on each simplex (represented by the
vector `conductivity').  Based on the finite element geometry, we first
calculate a system (or stiffness) matrix for each simplex (represented
as 3-by-3 elements on the diagonal of the element-wise system matrix
`SE'.  Based on `SE' and a N-by-DE connectivity matrix `C',
representing the connections between simplices and vertices, the global
connectivity matrix `S' is calculated.

       # Element conductivity
       conductivity= [1*ones(1,16), ...
              2*ones(1,48), 1*ones(1,16)];

       # Connectivity matrix
       C = sparse ((1:D*E), reshape (elems', ...
              D*E, 1), 1, D*E, N);

       # Calculate system matrix
       Siidx = floor ([0:D*E-1]'/D) * D * ...
              ones(1,D) + ones(D*E,1)*(1:D) ;
       Sjidx = [1:D*E]'*ones(1,D);
       Sdata = zeros(D*E,D);
       dfact = factorial(D-1);
       for j=1:E
          a = inv([ones(D,1), ...
              nodes(elems(j,:), :)]);
          const = conductivity(j) * 2 / ...
              dfact / abs(det(a));
          Sdata(D*(j-1)+(1:D),:) = const * ...
              a(2:D,:)' * a(2:D,:);
       endfor
       # Element-wise system matrix
       SE= sparse(Siidx,Sjidx,Sdata);
       # Global system matrix
       S= C'* SE *C;

   The system matrix acts like the conductivity `S' in Ohm's law `S * V
= I'.  Based on the Dirichlet and Neumann boundary conditions, we are
able to solve for the voltages at each vertex `V'.

       # Dirichlet boundary conditions
       D_nodes=[1:5, 51:55];
       D_value=[10*ones(1,5), 20*ones(1,5)];

       V= zeros(N,1);
       V(D_nodes) = D_value;
       idx = 1:N; # vertices without Dirichlet
                  # boundary condns
       idx(D_nodes) = [];

       # Neumann boundary conditions.  Note that
       # N_value must be normalized by the
       # boundary length and element conductivity
       N_nodes=[];
       N_value=[];

       Q = zeros(N,1);
       Q(N_nodes) = N_value;

       V(idx) = S(idx,idx) \ ( Q(idx) - ...
                 S(idx,D_nodes) * V(D_nodes));

   Finally, in order to display the solution, we show each solved
voltage value in the z-axis for each simplex vertex.

       elemx = elems(:,[1,2,3,1])';
       xelems = reshape (nodes(elemx, 1), 4, E);
       yelems = reshape (nodes(elemx, 2), 4, E);
       velems = reshape (V(elemx), 4, E);
       plot3 (xelems,yelems,velems,'k');
       print ('grid.eps');


File: octave.info,  Node: Numerical Integration,  Next: Differential Equations,  Prev: Sparse Matrices,  Up: Top

23 Numerical Integration
************************

Octave comes with several built-in functions for computing the integral
of a function numerically (termed quadrature).  These functions all
solve 1-dimensional integration problems.

* Menu:

* Functions of One Variable::
* Orthogonal Collocation::
* Functions of Multiple Variables::


File: octave.info,  Node: Functions of One Variable,  Next: Orthogonal Collocation,  Up: Numerical Integration

23.1 Functions of One Variable
==============================

Octave supports five different algorithms for computing the integral of
a function f over the interval from a to b.  These are

`quad'
     Numerical integration based on Gaussian quadrature.

`quadv'
     Numerical integration using an adaptive vectorized Simpson's rule.

`quadl'
     Numerical integration using an adaptive Lobatto rule.

`quadgk'
     Numerical integration using an adaptive Gauss-Konrod rule.

`quadcc'
     Numerical integration using adaptive Clenshaw-Curtis rules.

`trapz, cumtrapz'
     Numerical integration of data using the trapezoidal method.

The best quadrature algorithm to use depends on the integrand.  If you
have empirical data, rather than a function, the choice is `trapz' or
`cumtrapz'.  If you are uncertain about the characteristics of the
integrand, `quadcc' will be the most robust as it can handle
discontinuities, singularities, oscillatory functions, and infinite
intervals.  When the integrand is smooth `quadgk' may be the fastest of
the algorithms.

     Function    Characteristics
---------------------------------------------------------------------------- 
     quad        Low accuracy with nonsmooth integrands
     quadv       Medium accuracy with smooth integrands
     quadl       Medium accuracy with smooth integrands.  Slower than
                 quadgk.
     quadgk      Medium accuracy (1e^-6-1e^-9) with smooth integrands.
                 Handles oscillatory functions and infinite bounds
     quadcc      Low to High accuracy with nonsmooth/smooth integrands
                 Handles oscillatory functions, singularities, and
                 infinite bounds

   Here is an example of using `quad' to integrate the function

       F(X) = X * sin (1/X) * sqrt (abs (1 - X))

from X = 0 to X = 3.

   This is a fairly difficult integration (plot the function over the
range of integration to see why).

   The first step is to define the function:

     function y = f (x)
       y = x .* sin (1./x) .* sqrt (abs (1 - x));
     endfunction

   Note the use of the `dot' forms of the operators.  This is not
necessary for the `quad' integrator, but is required by the other
integrators.  In any case, it makes it much easier to generate a set of
points for plotting because it is possible to call the function with a
vector argument to produce a vector result.

   The second step is to call quad with the limits of integration:

     [q, ier, nfun, err] = quad ("f", 0, 3)
          => 1.9819
          => 1
          => 5061
          => 1.1522e-07

   Although `quad' returns a nonzero value for IER, the result is
reasonably accurate (to see why, examine what happens to the result if
you move the lower bound to 0.1, then 0.01, then 0.001, etc.).

   The function "f" can be the string name of a function, a function
handle, or an inline function.  These options make it quite easy to do
integration without having to fully define a function in an m-file.
For example:

     # Verify integral (x^3) = x^4/4
     f = inline ("x.^3");
     quadgk (f, 0, 1)
          => 0.25000

     # Verify gamma function = (n-1)! for n = 4
     f = @(x) x.^3 .* exp (-x);
     quadcc (f, 0, Inf)
          => 6.0000

 -- Loadable Function: Q = quad (F, A, B)
 -- Loadable Function: Q = quad (F, A, B, TOL)
 -- Loadable Function: Q = quad (F, A, B, TOL, SING)
 -- Loadable Function: [Q, IER, NFUN, ERR] = quad (...)
     Numerically evaluate the integral of F from A to B using Fortran
     routines from QUADPACK.  F is a function handle, inline function,
     or a string containing the name of the function to evaluate.  The
     function must have the form `y = f (x)' where Y and X are scalars.

     A and B are the lower and upper limits of integration.  Either or
     both may be infinite.

     The optional argument TOL is a vector that specifies the desired
     accuracy of the result.  The first element of the vector is the
     desired absolute tolerance, and the second element is the desired
     relative tolerance.  To choose a relative test only, set the
     absolute tolerance to zero.  To choose an absolute test only, set
     the relative tolerance to zero.  Both tolerances default to
     `sqrt(eps)' or approximately 1.5e^-8.

     The optional argument SING is a vector of values at which the
     integrand is known to be singular.

     The result of the integration is returned in Q.  IER contains an
     integer error code (0 indicates a successful integration).  NFUN
     indicates the number of function evaluations that were made, and
     ERR contains an estimate of the error in the solution.

     The function `quad_options' can set other optional parameters for
     `quad'.

     Note: because `quad' is written in Fortran it cannot be called
     recursively.  This prevents its use in integrating over more than
     one variable by routines `dblquad' and `triplequad'.

     *See also:* *note quad_options: doc-quad_options, *note quadv:
     doc-quadv, *note quadl: doc-quadl, *note quadgk: doc-quadgk, *note
     quadcc: doc-quadcc, *note trapz: doc-trapz, *note dblquad:
     doc-dblquad, *note triplequad: doc-triplequad.

 -- Loadable Function:  quad_options ()
 -- Loadable Function: val = quad_options (OPT)
 -- Loadable Function:  quad_options (OPT, VAL)
     Query or set options for the function `quad'.  When called with no
     arguments, the names of all available options and their current
     values are displayed.  Given one argument, return the value of the
     corresponding option.  When called with two arguments,
     `quad_options' set the option OPT to value VAL.

     Options include

    `"absolute tolerance"'
          Absolute tolerance; may be zero for pure relative error test.

    `"relative tolerance"'
          Non-negative relative tolerance.  If the absolute tolerance
          is zero, the relative tolerance must be greater than or equal
          to `max (50*eps, 0.5e-28)'.

    `"single precision absolute tolerance"'
          Absolute tolerance for single precision; may be zero for pure
          relative error test.

    `"single precision relative tolerance"'
          Non-negative relative tolerance for single precision.  If the
          absolute tolerance is zero, the relative tolerance must be
          greater than or equal to `max (50*eps, 0.5e-28)'.

 -- Function File: Q = quadv (F, A, B)
 -- Function File: Q = quadv (F, A, B, TOL)
 -- Function File: Q = quadv (F, A, B, TOL, TRACE)
 -- Function File: Q = quadv (F, A, B, TOL, TRACE, P1, P2, ...)
 -- Function File: [Q, NFUN] = quadv (...)
     Numerically evaluate the integral of F from A to B using an
     adaptive Simpson's rule.  F is a function handle, inline function,
     or string containing the name of the function to evaluate.
     `quadv' is a vectorized version of `quad' and the function defined
     by F must accept a scalar or vector as input and return a scalar,
     vector, or array as output.

     A and B are the lower and upper limits of integration.  Both
     limits must be finite.

     The optional argument TOL defines the tolerance used to stop the
     adaptation procedure.  The default value is 1e^-6.

     The algorithm used by `quadv' involves recursively subdividing the
     integration interval and applying Simpson's rule on each
     subinterval.  If TRACE is true then after computing each of these
     partial integrals display: (1) the total number of function
     evaluations, (2) the left end of the subinterval, (3) the length
     of the subinterval, (4) the approximation of the integral over the
     subinterval.

     Additional arguments P1, etc., are passed directly to the function
     F.  To use default values for TOL and TRACE, one may pass empty
     matrices ([]).

     The result of the integration is returned in Q.  NFUN indicates
     the number of function evaluations that were made.

     Note: `quadv' is written in Octave's scripting language and can be
     used recursively in `dblquad' and `triplequad', unlike the similar
     `quad' function.

     *See also:* *note quad: doc-quad, *note quadl: doc-quadl, *note
     quadgk: doc-quadgk, *note quadcc: doc-quadcc, *note trapz:
     doc-trapz, *note dblquad: doc-dblquad, *note triplequad:
     doc-triplequad.

 -- Function File: Q = quadl (F, A, B)
 -- Function File: Q = quadl (F, A, B, TOL)
 -- Function File: Q = quadl (F, A, B, TOL, TRACE)
 -- Function File: Q = quadl (F, A, B, TOL, TRACE, P1, P2, ...)
     Numerically evaluate the integral of F from A to B using an
     adaptive Lobatto rule.  F is a function handle, inline function,
     or string containing the name of the function to evaluate.  The
     function F must be vectorized and return a vector of output values
     if given a vector of input values.

     A and B are the lower and upper limits of integration.  Both
     limits must be finite.

     The optional argument TOL defines the relative tolerance with which
     to perform the integration.  The default value is `eps'.

     The algorithm used by `quadl' involves recursively subdividing the
     integration interval.  If TRACE is defined then for each
     subinterval display: (1) the left end of the subinterval, (2) the
     length of the subinterval, (3) the approximation of the integral
     over the subinterval.

     Additional arguments P1, etc., are passed directly to the function
     F.  To use default values for TOL and TRACE, one may pass empty
     matrices ([]).

     Reference: W. Gander and W. Gautschi, `Adaptive Quadrature -
     Revisited', BIT Vol. 40, No. 1, March 2000, pp. 84-101.
     `http://www.inf.ethz.ch/personal/gander/'

     *See also:* *note quad: doc-quad, *note quadv: doc-quadv, *note
     quadgk: doc-quadgk, *note quadcc: doc-quadcc, *note trapz:
     doc-trapz, *note dblquad: doc-dblquad, *note triplequad:
     doc-triplequad.

 -- Function File: Q = quadgk (F, A, B)
 -- Function File: Q = quadgk (F, A, B, ABSTOL)
 -- Function File: Q = quadgk (F, A, B, ABSTOL, TRACE)
 -- Function File: Q = quadgk (F, A, B, PROP, VAL, ...)
 -- Function File: [Q, ERR] = quadgk (...)
     Numerically evaluate the integral of F from A to B using adaptive
     Gauss-Konrod quadrature.  F is a function handle, inline function,
     or string containing the name of the function to evaluate.  The
     formulation is based on a proposal by L.F. Shampine, `"Vectorized
     adaptive quadrature in MATLAB", Journal of Computational and
     Applied Mathematics, pp131-140, Vol 211, Issue 2, Feb 2008' where
     all function evaluations at an iteration are calculated with a
     single call to F.  Therefore, the function F must be vectorized
     and must accept a vector of input values X and return an output
     vector representing the function evaluations at the given values
     of X.

     A and B are the lower and upper limits of integration.  Either or
     both limits may be infinite or contain weak end singularities.
     Variable transformation will be used to treat any infinite
     intervals and weaken the singularities.  For example:

          quadgk(@(x) 1 ./ (sqrt (x) .* (x + 1)), 0, Inf)

     Note that the formulation of the integrand uses the
     element-by-element operator `./' and all user functions to
     `quadgk' should do the same.

     The optional argument TOL defines the absolute tolerance used to
     stop the integration procedure.  The default value is 1e^-10.

     The algorithm used by `quadgk' involves subdividing the
     integration interval and evaluating each subinterval.  If TRACE is
     true then after computing each of these partial integrals display:
     (1) the number of subintervals at this step, (2) the current
     estimate of the error ERR, (3) the current estimate for the
     integral Q.

     Alternatively, properties of `quadgk' can be passed to the
     function as pairs `"PROP", VAL'.  Valid properties are

    `AbsTol'
          Define the absolute error tolerance for the quadrature.  The
          default absolute tolerance is 1e-10.

    `RelTol'
          Define the relative error tolerance for the quadrature.  The
          default relative tolerance is 1e-5.

    `MaxIntervalCount'
          `quadgk' initially subdivides the interval on which to perform
          the quadrature into 10 intervals.  Subintervals that have an
          unacceptable error are subdivided and re-evaluated.  If the
          number of subintervals exceeds 650 subintervals at any point
          then a poor convergence is signaled and the current estimate
          of the integral is returned.  The property 'MaxIntervalCount'
          can be used to alter the number of subintervals that can
          exist before exiting.

    `WayPoints'
          Discontinuities in the first derivative of the function to
          integrate can be flagged with the  `"WayPoints"' property.
          This forces the ends of a subinterval to fall on the
          breakpoints of the function and can result in significantly
          improved estimation of the error in the integral, faster
          computation, or both.  For example,

               quadgk (@(x) abs (1 - x.^2), 0, 2, 'Waypoints', 1)

          signals the breakpoint in the integrand at `X = 1'.

    `Trace'
          If logically true `quadgk' prints information on the
          convergence of the quadrature at each iteration.

     If any of A, B, or WAYPOINTS is complex then the quadrature is
     treated as a contour integral along a piecewise continuous path
     defined by the above.  In this case the integral is assumed to
     have no edge singularities.  For example,

          quadgk (@(z) log (z), 1+1i, 1+1i, "WayPoints",
                  [1-1i, -1,-1i, -1+1i])

     integrates `log (z)' along the square defined by `[1+1i,  1-1i,
     -1-1i, -1+1i]'

     The result of the integration is returned in Q.  ERR is an
     approximate bound on the error in the integral `abs (Q - I)',
     where I is the exact value of the integral.

     *See also:* *note quad: doc-quad, *note quadv: doc-quadv, *note
     quadl: doc-quadl, *note quadcc: doc-quadcc, *note trapz:
     doc-trapz, *note dblquad: doc-dblquad, *note triplequad:
     doc-triplequad.

 -- Function File: Q = quadcc (F, A, B)
 -- Function File: Q = quadcc (F, A, B, TOL)
 -- Function File: Q = quadcc (F, A, B, TOL, SING)
 -- Function File: [Q, ERR, NR_POINTS] = quadcc (...)
     Numerically evaluate the integral of F from A to B using the
     doubly-adaptive Clenshaw-Curtis quadrature described by P. Gonnet
     in `Increasing the Reliability of Adaptive Quadrature Using
     Explicit Interpolants'.  F is a function handle, inline function,
     or string containing the name of the function to evaluate.  The
     function F must be vectorized and must return a vector of output
     values if given a vector of input values.  For example,

             f = @(x) x .* sin (1./x) .* sqrt (abs (1 - x));

     which uses the element-by-element `dot' form for all operators.

     A and B are the lower and upper limits of integration.  Either or
     both limits may be infinite.  `quadcc' handles an inifinite limit
     by substituting the variable of integration with `x=tan(pi/2*u)'.

     The optional argument TOL defines the relative tolerance used to
     stop the integration procedure.  The default value is 1e^-6.

     The optional argument SING contains a list of points where the
     integrand has known singularities, or discontinuities in any of
     its derivatives, inside the integration interval.  For the example
     above, which has a discontinuity at x=1, the call to `quadcc'
     would be as follows

             int = quadcc (f, a, b, 1.0e-6, [ 1 ]);

     The result of the integration is returned in Q.  ERR is an
     estimate of the absolute integration error and NR_POINTS is the
     number of points at which the integrand was evaluated.  If the
     adaptive integration did not converge, the value of ERR will be
     larger than the requested tolerance.  Therefore, it is recommended
     to verify this value for difficult integrands.

     `quadcc' is capable of dealing with non-numeric values of the
     integrand such as `NaN' or `Inf'.  If the integral diverges, and
     `quadcc' detects this, then a warning is issued and `Inf' or
     `-Inf' is returned.

     Note: `quadcc' is a general purpose quadrature algorithm and, as
     such, may be less efficient for a smooth or otherwise well-behaved
     integrand than other methods such as `quadgk'.

     The algorithm uses Clenshaw-Curtis quadrature rules of increasing
     degree in each interval and bisects the interval if either the
     function does not appear to be smooth or a rule of maximum degree
     has been reached.  The error estimate is computed from the L2-norm
     of the difference between two successive interpolations of the
     integrand over the nodes of the respective quadrature rules.

     Reference: P. Gonnet, `Increasing the Reliability of Adaptive
     Quadrature Using Explicit Interpolants', ACM Transactions on
     Mathematical Software, Vol. 37, Issue 3, Article No. 3, 2010.

     *See also:* *note quad: doc-quad, *note quadv: doc-quadv, *note
     quadl: doc-quadl, *note quadgk: doc-quadgk, *note trapz:
     doc-trapz, *note dblquad: doc-dblquad, *note triplequad:
     doc-triplequad.

   Sometimes one does not have the function, but only the raw (x, y)
points from which to perform an integration.  This can occur when
collecting data in an experiment.  The `trapz' function can integrate
these values as shown in the following example where "data" has been
collected on the cosine function over the range [0, pi/2).

     x = 0:0.1:pi/2;  # Uniformly spaced points
     y = cos (x);
     trapz (x, y)
          => 0.99666

   The answer is reasonably close to the exact value of 1.  Ordinary
quadrature is sensitive to the characteristics of the integrand.
Empirical integration depends not just on the integrand, but also on
the particular points chosen to represent the function.  Repeating the
example above with the sine function over the range [0, pi/2) produces
far inferior results.

     x = 0:0.1:pi/2;  # Uniformly spaced points
     y = sin (x);
     trapz (x, y)
          => 0.92849

   However, a slightly different choice of data points can change the
result significantly.  The same integration, with the same number of
points, but spaced differently produces a more accurate answer.

     x = linspace (0, pi/2, 16);  # Uniformly spaced, but including endpoint
     y = sin (x);
     trapz (x, y)
          => 0.99909

   In general there may be no way of knowing the best distribution of
points ahead of time.  Or the points may come from an experiment where
there is no freedom to select the best distribution.  In any case, one
must remain aware of this issue when using `trapz'.

 -- Function File: Q = trapz (Y)
 -- Function File: Q = trapz (X, Y)
 -- Function File: Q = trapz (..., DIM)
     Numerically evaluate the integral of points Y using the trapezoidal
     method.  `trapz (Y)' computes the integral of Y along the first
     non-singleton dimension.  When the argument X is omitted an
     equally spaced X vector with unit spacing (1) is assumed.  `trapz
     (X, Y)' evaluates the integral with respect to the spacing in X
     and the values in Y.  This is useful if the points in Y have been
     sampled unevenly.  If the optional DIM argument is given, operate
     along this dimension.

     If X is not specified then unit spacing will be used.  To scale
     the integral to the correct value you must multiply by the actual
     spacing value (deltaX).  As an example, the integral of x^3 over
     the range [0, 1] is x^4/4 or 0.25.  The following code uses
     `trapz' to calculate the integral in three different ways.

          x = 0:0.1:1;
          y = x.^3;
          q = trapz (y)
            => q = 2.525   # No scaling
          q * 0.1
            => q = 0.2525  # Approximation to integral by scaling
          trapz (x, y)
            => q = 0.2525  # Same result by specifying X

     *See also:* *note cumtrapz: doc-cumtrapz.

 -- Function File: Q = cumtrapz (Y)
 -- Function File: Q = cumtrapz (X, Y)
 -- Function File: Q = cumtrapz (..., DIM)
     Cumulative numerical integration of points Y using the trapezoidal
     method.  `cumtrapz (Y)' computes the cumulative integral of Y
     along the first non-singleton dimension.  Where `trapz' reports
     only the overall integral sum, `cumtrapz' reports the current
     partial sum value at each point of Y.  When the argument X is
     omitted an equally spaced X vector with unit spacing (1) is
     assumed.  `cumtrapz (X, Y)' evaluates the integral with respect to
     the spacing in X and the values in Y.  This is useful if the
     points in Y have been sampled unevenly.  If the optional DIM
     argument is given, operate along this dimension.

     If X is not specified then unit spacing will be used.  To scale
     the integral to the correct value you must multiply by the actual
     spacing value (deltaX).

     *See also:* *note trapz: doc-trapz, *note cumsum: doc-cumsum.


File: octave.info,  Node: Orthogonal Collocation,  Next: Functions of Multiple Variables,  Prev: Functions of One Variable,  Up: Numerical Integration

23.2 Orthogonal Collocation
===========================

 -- Loadable Function: [R, AMAT, BMAT, Q] = colloc (N, "left", "right")
     Compute derivative and integral weight matrices for orthogonal
     collocation using the subroutines given in J. Villadsen and M. L.
     Michelsen, `Solution of Differential Equation Models by Polynomial
     Approximation'.

   Here is an example of using `colloc' to generate weight matrices for
solving the second order differential equation U' - ALPHA * U" = 0 with
the boundary conditions U(0) = 0 and U(1) = 1.

   First, we can generate the weight matrices for N points (including
the endpoints of the interval), and incorporate the boundary conditions
in the right hand side (for a specific value of ALPHA).

     n = 7;
     alpha = 0.1;
     [r, a, b] = colloc (n-2, "left", "right");
     at = a(2:n-1,2:n-1);
     bt = b(2:n-1,2:n-1);
     rhs = alpha * b(2:n-1,n) - a(2:n-1,n);

   Then the solution at the roots R is

     u = [ 0; (at - alpha * bt) \ rhs; 1]
          => [ 0.00; 0.004; 0.01 0.00; 0.12; 0.62; 1.00 ]


File: octave.info,  Node: Functions of Multiple Variables,  Prev: Orthogonal Collocation,  Up: Numerical Integration

23.3 Functions of Multiple Variables
====================================

Octave does not have built-in functions for computing the integral of
functions of multiple variables directly.  It is possible, however, to
compute the integral of a function of multiple variables using the
existing functions for one-dimensional integrals.

   To illustrate how the integration can be performed, we will integrate
the function

     f(x, y) = sin(pi*x*y)*sqrt(x*y)

   for x and y between 0 and 1.

   The first approach creates a function that integrates f with respect
to x, and then integrates that function with respect to y.  Because
`quad' is written in Fortran it cannot be called recursively.  This
means that `quad' cannot integrate a function that calls `quad', and
hence cannot be used to perform the double integration.  Any of the
other integrators, however, can be used which is what the following
code demonstrates.

     function q = g(y)
       q = ones (size (y));
       for i = 1:length (y)
         f = @(x) sin (pi*x.*y(i)) .* sqrt (x.*y(i));
         q(i) = quadgk (f, 0, 1);
       endfor
     endfunction

     I = quadgk ("g", 0, 1)
           => 0.30022

   The above process can be simplified with the `dblquad' and
`triplequad' functions for integrals over two and three variables.  For
example:

     I = dblquad (@(x, y) sin (pi*x.*y) .* sqrt (x.*y), 0, 1, 0, 1)
           => 0.30022

 -- Function File:  dblquad (F, XA, XB, YA, YB)
 -- Function File:  dblquad (F, XA, XB, YA, YB, TOL)
 -- Function File:  dblquad (F, XA, XB, YA, YB, TOL, QUADF)
 -- Function File:  dblquad (F, XA, XB, YA, YB, TOL, QUADF, ...)
     Numerically evaluate the double integral of F.  F is a function
     handle, inline function, or string containing the name of the
     function to evaluate.  The function F must have the form z =
     f(x,y) where X is a vector and Y is a scalar.  It should return a
     vector of the same length and orientation as X.

     XA, YA and XB, YB are the lower and upper limits of integration
     for x and y respectively.  The underlying integrator determines
     whether infinite bounds are accepted.

     The optional argument TOL defines the absolute tolerance used to
     integrate each sub-integral.  The default value is 1e^-6.

     The optional argument QUADF specifies which underlying integrator
     function to use.  Any choice but `quad' is available and the
     default is `quadcc'.

     Additional arguments, are passed directly to F.  To use the default
     value for TOL or QUADF one may pass ':' or an empty matrix ([]).

     *See also:* *note triplequad: doc-triplequad, *note quad:
     doc-quad, *note quadv: doc-quadv, *note quadl: doc-quadl, *note
     quadgk: doc-quadgk, *note quadcc: doc-quadcc, *note trapz:
     doc-trapz.

 -- Function File:  triplequad (F, XA, XB, YA, YB, ZA, ZB)
 -- Function File:  triplequad (F, XA, XB, YA, YB, ZA, ZB, TOL)
 -- Function File:  triplequad (F, XA, XB, YA, YB, ZA, ZB, TOL, QUADF)
 -- Function File:  triplequad (F, XA, XB, YA, YB, ZA, ZB, TOL, QUADF,
          ...)
     Numerically evaluate the triple integral of F.  F is a function
     handle, inline function, or string containing the name of the
     function to evaluate.  The function F must have the form w =
     f(x,y,z) where either X or Y is a vector and the remaining inputs
     are scalars.  It should return a vector of the same length and
     orientation as X or Y.

     XA, YA, ZA and XB, YB, ZB are the lower and upper limits of
     integration for x, y, and z respectively.  The underlying
     integrator determines whether infinite bounds are accepted.

     The optional argument TOL defines the absolute tolerance used to
     integrate each sub-integral.  The default value is 1e^-6.

     The optional argument QUADF specifies which underlying integrator
     function to use.  Any choice but `quad' is available and the
     default is `quadcc'.

     Additional arguments, are passed directly to F.  To use the default
     value for TOL or QUADF one may pass ':' or an empty matrix ([]).

     *See also:* *note dblquad: doc-dblquad, *note quad: doc-quad,
     *note quadv: doc-quadv, *note quadl: doc-quadl, *note quadgk:
     doc-quadgk, *note quadcc: doc-quadcc, *note trapz: doc-trapz.

   The above mentioned approach works, but is fairly slow, and that
problem increases exponentially with the dimensionality of the
integral.  Another possible solution is to use Orthogonal Collocation
as described in the previous section (*note Orthogonal Collocation::).
The integral of a function f(x,y) for x and y between 0 and 1 can be
approximated using n points by the sum over `i=1:n' and `j=1:n' of
`q(i)*q(j)*f(r(i),r(j))', where q and r is as returned by `colloc(n)'.
The generalization to more than two variables is straight forward.  The
following code computes the studied integral using n=8 points.

     f = @(x,y) sin (pi*x*y') .* sqrt (x*y');
     n = 8;
     [t, ~, ~, q] = colloc (n);
     I = q'*f(t,t)*q;
           => 0.30022

It should be noted that the number of points determines the quality of
the approximation.  If the integration needs to be performed between a
and b, instead of 0 and 1, then a change of variables is needed.


File: octave.info,  Node: Differential Equations,  Next: Optimization,  Prev: Numerical Integration,  Up: Top

24 Differential Equations
*************************

Octave has built-in functions for solving ordinary differential
equations, and differential-algebraic equations.  All solvers are based
on reliable ODE routines written in Fortran.

* Menu:

* Ordinary Differential Equations::
* Differential-Algebraic Equations::


File: octave.info,  Node: Ordinary Differential Equations,  Next: Differential-Algebraic Equations,  Up: Differential Equations

24.1 Ordinary Differential Equations
====================================

The function `lsode' can be used to solve ODEs of the form

     dx
     -- = f (x, t)
     dt

using Hindmarsh's ODE solver LSODE.

 -- Loadable Function: [X, ISTATE, MSG] = lsode (FCN, X_0, T)
 -- Loadable Function: [X, ISTATE, MSG] = lsode (FCN, X_0, T, T_CRIT)
     Solve the set of differential equations

          dx
          -- = f(x, t)
          dt

     with

          x(t_0) = x_0

     The solution is returned in the matrix X, with each row
     corresponding to an element of the vector T.  The first element of
     T should be t_0 and should correspond to the initial state of the
     system X_0, so that the first row of the output is X_0.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of right
     hand sides for the set of equations.  The function must have the
     form

          XDOT = f (X, T)

     in which XDOT and X are vectors and T is a scalar.

     If FCN is a two-element string array or a two-element cell array
     of strings, inline functions, or function handles, the first
     element names the function f described above, and the second
     element names a function to compute the Jacobian of f.  The
     Jacobian function must have the form

          JAC = j (X, T)

     in which JAC is the matrix of partial derivatives

                       | df_1  df_1       df_1 |
                       | ----  ----  ...  ---- |
                       | dx_1  dx_2       dx_N |
                       |                       |
                       | df_2  df_2       df_2 |
                       | ----  ----  ...  ---- |
                df_i   | dx_1  dx_2       dx_N |
          jac = ---- = |                       |
                dx_j   |  .    .     .    .    |
                       |  .    .      .   .    |
                       |  .    .       .  .    |
                       |                       |
                       | df_N  df_N       df_N |
                       | ----  ----  ...  ---- |
                       | dx_1  dx_2       dx_N |

     The second and third arguments specify the initial state of the
     system, x_0, and the initial value of the independent variable t_0.

     The fourth argument is optional, and may be used to specify a set
     of times that the ODE solver should not integrate past.  It is
     useful for avoiding difficulties with singularities and points
     where there is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be 2
     (consistent with the Fortran version of LSODE).

     If the computation is not successful, ISTATE will be something
     other than 2 and MSG will contain additional information.

     You can use the function `lsode_options' to set optional
     parameters for `lsode'.

     *See also:* *note daspk: doc-daspk, *note dassl: doc-dassl, *note
     dasrt: doc-dasrt.

 -- Loadable Function:  lsode_options ()
 -- Loadable Function: val = lsode_options (OPT)
 -- Loadable Function:  lsode_options (OPT, VAL)
     Query or set options for the function `lsode'.  When called with
     no arguments, the names of all available options and their current
     values are displayed.  Given one argument, return the value of the
     corresponding option.  When called with two arguments,
     `lsode_options' set the option OPT to value VAL.

     Options include

    `"absolute tolerance"'
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector.

    `"relative tolerance"'
          Relative tolerance parameter.  Unlike the absolute tolerance,
          this parameter may only be a scalar.

          The local error test applied at each integration step is

                 abs (local error in x(i)) <= ...
                     rtol * abs (y(i)) + atol(i)

    `"integration method"'
          A string specifying the method of integration to use to solve
          the ODE system.  Valid values are

         "adams"
         "non-stiff"
               No Jacobian used (even if it is available).

         "bdf"
         "stiff"
               Use stiff backward differentiation formula (BDF) method.
               If a function to compute the Jacobian is not supplied,
               `lsode' will compute a finite difference approximation
               of the Jacobian matrix.

    `"initial step size"'
          The step size to be attempted on the first step (default is
          determined automatically).

    `"maximum order"'
          Restrict the maximum order of the solution method.  If using
          the Adams method, this option must be between 1 and 12.
          Otherwise, it must be between 1 and 5, inclusive.

    `"maximum step size"'
          Setting the maximum stepsize will avoid passing over very
          large regions  (default is not specified).

    `"minimum step size"'
          The minimum absolute step size allowed (default is 0).

    `"step limit"'
          Maximum number of steps allowed (default is 100000).

   Here is an example of solving a set of three differential equations
using `lsode'.  Given the function

     function xdot = f (x, t)

       xdot = zeros (3,1);

       xdot(1) = 77.27 * (x(2) - x(1)*x(2) + x(1) \
                 - 8.375e-06*x(1)^2);
       xdot(2) = (x(3) - x(1)*x(2) - x(2)) / 77.27;
       xdot(3) = 0.161*(x(1) - x(3));

     endfunction

and the initial condition `x0 = [ 4; 1.1; 4 ]', the set of equations
can be integrated using the command

     t = linspace (0, 500, 1000);

     y = lsode ("f", x0, t);

   If you try this, you will see that the value of the result changes
dramatically between T = 0 and 5, and again around T = 305.  A more
efficient set of output points might be

     t = [0, logspace (-1, log10(303), 150), \
             logspace (log10(304), log10(500), 150)];

   See Alan C. Hindmarsh, `ODEPACK, A Systematized Collection of ODE
Solvers', in Scientific Computing, R. S. Stepleman, editor, (1983) for
more information about the inner workings of `lsode'.


File: octave.info,  Node: Differential-Algebraic Equations,  Prev: Ordinary Differential Equations,  Up: Differential Equations

24.2 Differential-Algebraic Equations
=====================================

The function `daspk' can be used to solve DAEs of the form

     0 = f (x-dot, x, t),    x(t=0) = x_0, x-dot(t=0) = x-dot_0

where x-dot is the derivative of x.  The equation is solved using
Petzold's DAE solver DASPK.

 -- Loadable Function: [X, XDOT, ISTATE, MSG] = daspk (FCN, X_0,
          XDOT_0, T, T_CRIT)
     Solve the set of differential-algebraic equations

          0 = f (x, xdot, t)

     with

          x(t_0) = x_0, xdot(t_0) = xdot_0

     The solution is returned in the matrices X and XDOT, with each row
     in the result matrices corresponding to one of the elements in the
     vector T.  The first element of T should be t_0 and correspond to
     the initial state of the system X_0 and its derivative XDOT_0, so
     that the first row of the output X is X_0 and the first row of the
     output XDOT is XDOT_0.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of
     residuals for the set of equations.  It must have the form

          RES = f (X, XDOT, T)

     in which X, XDOT, and RES are vectors, and T is a scalar.

     If FCN is a two-element string array or a two-element cell array
     of strings, inline functions, or function handles, the first
     element names the function f described above, and the second
     element names a function to compute the modified Jacobian

                df       df
          jac = -- + c ------
                dx     d xdot

     The modified Jacobian function must have the form


          JAC = j (X, XDOT, T, C)

     The second and third arguments to `daspk' specify the initial
     condition of the states and their derivatives, and the fourth
     argument specifies a vector of output times at which the solution
     is desired, including the time corresponding to the initial
     condition.

     The set of initial states and derivatives are not strictly
     required to be consistent.  If they are not consistent, you must
     use the `daspk_options' function to provide additional information
     so that `daspk' can compute a consistent starting point.

     The fifth argument is optional, and may be used to specify a set of
     times that the DAE solver should not integrate past.  It is useful
     for avoiding difficulties with singularities and points where
     there is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be
     greater than zero (consistent with the Fortran version of DASPK).

     If the computation is not successful, the value of ISTATE will be
     less than zero and MSG will contain additional information.

     You can use the function `daspk_options' to set optional
     parameters for `daspk'.

     *See also:* *note dassl: doc-dassl.

 -- Loadable Function:  daspk_options ()
 -- Loadable Function: val = daspk_options (OPT)
 -- Loadable Function:  daspk_options (OPT, VAL)
     Query or set options for the function `daspk'.  When called with
     no arguments, the names of all available options and their current
     values are displayed.  Given one argument, return the value of the
     corresponding option.  When called with two arguments,
     `daspk_options' set the option OPT to value VAL.

     Options include

    `"absolute tolerance"'
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the relative tolerance must also be a vector of the same
          length.

    `"relative tolerance"'
          Relative tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the absolute tolerance must also be a vector of the same
          length.

          The local error test applied at each integration step is

                 abs (local error in x(i))
                      <= rtol(i) * abs (Y(i)) + atol(i)

    `"compute consistent initial condition"'
          Denoting the differential variables in the state vector by
          `Y_d' and the algebraic variables by `Y_a', `ddaspk' can solve
          one of two initialization problems:

            1. Given Y_d, calculate Y_a and Y'_d

            2. Given Y', calculate Y.

          In either case, initial values for the given components are
          input, and initial guesses for the unknown components must
          also be provided as input.  Set this option to 1 to solve the
          first problem, or 2 to solve the second (the default is 0, so
          you must provide a set of initial conditions that are
          consistent).

          If this option is set to a nonzero value, you must also set
          the `"algebraic variables"' option to declare which variables
          in the problem are algebraic.

    `"use initial condition heuristics"'
          Set to a nonzero value to use the initial condition
          heuristics options described below.

    `"initial condition heuristics"'
          A vector of the following parameters that can be used to
          control the initial condition calculation.

         `MXNIT'
               Maximum number of Newton iterations (default is 5).

         `MXNJ'
               Maximum number of Jacobian evaluations (default is 6).

         `MXNH'
               Maximum number of values of the artificial stepsize
               parameter to be tried if the `"compute consistent
               initial condition"' option has been set to 1 (default is
               5).

               Note that the maximum total number of Newton iterations
               allowed is `MXNIT*MXNJ*MXNH' if the `"compute consistent
               initial condition"' option has been set to 1 and
               `MXNIT*MXNJ' if it is set to 2.

         `LSOFF'
               Set to a nonzero value to disable the linesearch
               algorithm (default is 0).

         `STPTOL'
               Minimum scaled step in linesearch algorithm (default is
               eps^(2/3)).

         `EPINIT'
               Swing factor in the Newton iteration convergence test.
               The test is applied to the residual vector,
               premultiplied by the approximate Jacobian.  For
               convergence, the weighted RMS norm of this vector
               (scaled by the error weights) must be less than
               `EPINIT*EPCON', where `EPCON' = 0.33 is the analogous
               test constant used in the time steps.  The default is
               `EPINIT' = 0.01.

    `"print initial condition info"'
          Set this option to a nonzero value to display detailed
          information about the initial condition calculation (default
          is 0).

    `"exclude algebraic variables from error test"'
          Set to a nonzero value to exclude algebraic variables from
          the error test.  You must also set the `"algebraic
          variables"' option to declare which variables in the problem
          are algebraic (default is 0).

    `"algebraic variables"'
          A vector of the same length as the state vector.  A nonzero
          element indicates that the corresponding element of the state
          vector is an algebraic variable (i.e., its derivative does
          not appear explicitly in the equation set.

          This option is required by the `compute consistent initial
          condition"' and `"exclude algebraic variables from error
          test"' options.

    `"enforce inequality constraints"'
          Set to one of the following values to enforce the inequality
          constraints specified by the `"inequality constraint types"'
          option (default is 0).

            1. To have constraint checking only in the initial
               condition calculation.

            2. To enforce constraint checking during the integration.

            3. To enforce both options 1 and 2.

    `"inequality constraint types"'
          A vector of the same length as the state specifying the type
          of inequality constraint.  Each element of the vector
          corresponds to an element of the state and should be assigned
          one of the following codes

         -2
               Less than zero.

         -1
               Less than or equal to zero.

         0
               Not constrained.

         1
               Greater than or equal to zero.

         2
               Greater than zero.

          This option only has an effect if the `"enforce inequality
          constraints"' option is nonzero.

    `"initial step size"'
          Differential-algebraic problems may occasionally suffer from
          severe scaling difficulties on the first step.  If you know a
          great deal about the scaling of your problem, you can help to
          alleviate this problem by specifying an initial stepsize
          (default is computed automatically).

    `"maximum order"'
          Restrict the maximum order of the solution method.  This
          option must be between 1 and 5, inclusive (default is 5).

    `"maximum step size"'
          Setting the maximum stepsize will avoid passing over very
          large regions (default is not specified).

   Octave also includes DASSL, an earlier version of DASPK, and DASRT,
which can be used to solve DAEs with constraints (stopping conditions).

 -- Loadable Function: [X, XDOT, ISTATE, MSG] = dassl (FCN, X_0,
          XDOT_0, T, T_CRIT)
     Solve the set of differential-algebraic equations

          0 = f (x, xdot, t)

     with

          x(t_0) = x_0, xdot(t_0) = xdot_0

     The solution is returned in the matrices X and XDOT, with each row
     in the result matrices corresponding to one of the elements in the
     vector T.  The first element of T should be t_0 and correspond to
     the initial state of the system X_0 and its derivative XDOT_0, so
     that the first row of the output X is X_0 and the first row of the
     output XDOT is XDOT_0.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of
     residuals for the set of equations.  It must have the form

          RES = f (X, XDOT, T)

     in which X, XDOT, and RES are vectors, and T is a scalar.

     If FCN is a two-element string array or a two-element cell array
     of strings, inline functions, or function handles, the first
     element names the function f described above, and the second
     element names a function to compute the modified Jacobian

                df       df
          jac = -- + c ------
                dx     d xdot

     The modified Jacobian function must have the form


          JAC = j (X, XDOT, T, C)

     The second and third arguments to `dassl' specify the initial
     condition of the states and their derivatives, and the fourth
     argument specifies a vector of output times at which the solution
     is desired, including the time corresponding to the initial
     condition.

     The set of initial states and derivatives are not strictly
     required to be consistent.  In practice, however, DASSL is not
     very good at determining a consistent set for you, so it is best
     if you ensure that the initial values result in the function
     evaluating to zero.

     The fifth argument is optional, and may be used to specify a set of
     times that the DAE solver should not integrate past.  It is useful
     for avoiding difficulties with singularities and points where
     there is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be
     greater than zero (consistent with the Fortran version of DASSL).

     If the computation is not successful, the value of ISTATE will be
     less than zero and MSG will contain additional information.

     You can use the function `dassl_options' to set optional
     parameters for `dassl'.

     *See also:* *note daspk: doc-daspk, *note dasrt: doc-dasrt, *note
     lsode: doc-lsode.

 -- Loadable Function:  dassl_options ()
 -- Loadable Function: val = dassl_options (OPT)
 -- Loadable Function:  dassl_options (OPT, VAL)
     Query or set options for the function `dassl'.  When called with
     no arguments, the names of all available options and their current
     values are displayed.  Given one argument, return the value of the
     corresponding option.  When called with two arguments,
     `dassl_options' set the option OPT to value VAL.

     Options include

    `"absolute tolerance"'
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the relative tolerance must also be a vector of the same
          length.

    `"relative tolerance"'
          Relative tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the absolute tolerance must also be a vector of the same
          length.

          The local error test applied at each integration step is

                 abs (local error in x(i))
                      <= rtol(i) * abs (Y(i)) + atol(i)

    `"compute consistent initial condition"'
          If nonzero, `dassl' will attempt to compute a consistent set
          of initial conditions.  This is generally not reliable, so it
          is best to provide a consistent set and leave this option set
          to zero.

    `"enforce nonnegativity constraints"'
          If you know that the solutions to your equations will always
          be non-negative, it may help to set this parameter to a
          nonzero value.  However, it is probably best to try leaving
          this option set to zero first, and only setting it to a
          nonzero value if that doesn't work very well.

    `"initial step size"'
          Differential-algebraic problems may occasionally suffer from
          severe scaling difficulties on the first step.  If you know a
          great deal about the scaling of your problem, you can help to
          alleviate this problem by specifying an initial stepsize.

    `"maximum order"'
          Restrict the maximum order of the solution method.  This
          option must be between 1 and 5, inclusive.

    `"maximum step size"'
          Setting the maximum stepsize will avoid passing over very
          large regions  (default is not specified).

    `"step limit"'
          Maximum number of integration steps to attempt on a single
          call to the underlying Fortran code.

 -- Loadable Function: [X, XDOT, T_OUT, ISTAT, MSG] = dasrt (FCN, [],
          X_0, XDOT_0, T)
 -- Loadable Function: ... = dasrt (FCN, G, X_0, XDOT_0, T)
 -- Loadable Function: ... = dasrt (FCN, [], X_0, XDOT_0, T, T_CRIT)
 -- Loadable Function: ... = dasrt (FCN, G, X_0, XDOT_0, T, T_CRIT)
     Solve the set of differential-algebraic equations

          0 = f (x, xdot, t)

     with

          x(t_0) = x_0, xdot(t_0) = xdot_0

     with functional stopping criteria (root solving).

     The solution is returned in the matrices X and XDOT, with each row
     in the result matrices corresponding to one of the elements in the
     vector T_OUT.  The first element of T should be t_0 and correspond
     to the initial state of the system X_0 and its derivative XDOT_0,
     so that the first row of the output X is X_0 and the first row of
     the output XDOT is XDOT_0.

     The vector T provides an upper limit on the length of the
     integration.  If the stopping condition is met, the vector T_OUT
     will be shorter than T, and the final element of T_OUT will be the
     point at which the stopping condition was met, and may not
     correspond to any element of the vector T.

     The first argument, FCN, is a string, inline, or function handle
     that names the function f to call to compute the vector of
     residuals for the set of equations.  It must have the form

          RES = f (X, XDOT, T)

     in which X, XDOT, and RES are vectors, and T is a scalar.

     If FCN is a two-element string array or a two-element cell array
     of strings, inline functions, or function handles, the first
     element names the function f described above, and the second
     element names a function to compute the modified Jacobian

                df       df
          jac = -- + c ------
                dx     d xdot

     The modified Jacobian function must have the form


          JAC = j (X, XDOT, T, C)

     The optional second argument names a function that defines the
     constraint functions whose roots are desired during the
     integration.  This function must have the form

          G_OUT = g (X, T)

     and return a vector of the constraint function values.  If the
     value of any of the constraint functions changes sign, DASRT will
     attempt to stop the integration at the point of the sign change.

     If the name of the constraint function is omitted, `dasrt' solves
     the same problem as `daspk' or `dassl'.

     Note that because of numerical errors in the constraint functions
     due to round-off and integration error, DASRT may return false
     roots, or return the same root at two or more nearly equal values
     of T.  If such false roots are suspected, the user should consider
     smaller error tolerances or higher precision in the evaluation of
     the constraint functions.

     If a root of some constraint function defines the end of the
     problem, the input to DASRT should nevertheless allow integration
     to a point slightly past that root, so that DASRT can locate the
     root by interpolation.

     The third and fourth arguments to `dasrt' specify the initial
     condition of the states and their derivatives, and the fourth
     argument specifies a vector of output times at which the solution
     is desired, including the time corresponding to the initial
     condition.

     The set of initial states and derivatives are not strictly
     required to be consistent.  In practice, however, DASSL is not
     very good at determining a consistent set for you, so it is best
     if you ensure that the initial values result in the function
     evaluating to zero.

     The sixth argument is optional, and may be used to specify a set of
     times that the DAE solver should not integrate past.  It is useful
     for avoiding difficulties with singularities and points where
     there is a discontinuity in the derivative.

     After a successful computation, the value of ISTATE will be
     greater than zero (consistent with the Fortran version of DASSL).

     If the computation is not successful, the value of ISTATE will be
     less than zero and MSG will contain additional information.

     You can use the function `dasrt_options' to set optional
     parameters for `dasrt'.

     *See also:* *note dasrt_options: doc-dasrt_options, *note daspk:
     doc-daspk, *note dasrt: doc-dasrt, *note lsode: doc-lsode.

 -- Loadable Function:  dasrt_options ()
 -- Loadable Function: val = dasrt_options (OPT)
 -- Loadable Function:  dasrt_options (OPT, VAL)
     Query or set options for the function `dasrt'.  When called with
     no arguments, the names of all available options and their current
     values are displayed.  Given one argument, return the value of the
     corresponding option.  When called with two arguments,
     `dasrt_options' set the option OPT to value VAL.

     Options include

    `"absolute tolerance"'
          Absolute tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the relative tolerance must also be a vector of the same
          length.

    `"relative tolerance"'
          Relative tolerance.  May be either vector or scalar.  If a
          vector, it must match the dimension of the state vector, and
          the absolute tolerance must also be a vector of the same
          length.

          The local error test applied at each integration step is

                 abs (local error in x(i)) <= ...
                     rtol(i) * abs (Y(i)) + atol(i)

    `"initial step size"'
          Differential-algebraic problems may occasionally suffer from
          severe scaling difficulties on the first step.  If you know a
          great deal about the scaling of your problem, you can help to
          alleviate this problem by specifying an initial stepsize.

    `"maximum order"'
          Restrict the maximum order of the solution method.  This
          option must be between 1 and 5, inclusive.

    `"maximum step size"'
          Setting the maximum stepsize will avoid passing over very
          large regions.

    `"step limit"'
          Maximum number of integration steps to attempt on a single
          call to the underlying Fortran code.

   See K. E. Brenan, et al., `Numerical Solution of Initial-Value
Problems in Differential-Algebraic Equations', North-Holland (1989) for
more information about the implementation of DASSL.


File: octave.info,  Node: Optimization,  Next: Statistics,  Prev: Differential Equations,  Up: Top

25 Optimization
***************

Octave comes with support for solving various kinds of optimization
problems.  Specifically Octave can solve problems in Linear Programming,
Quadratic Programming, Nonlinear Programming, and Linear Least Squares
Minimization.

* Menu:

* Linear Programming::
* Quadratic Programming::
* Nonlinear Programming::
* Linear Least Squares::


File: octave.info,  Node: Linear Programming,  Next: Quadratic Programming,  Up: Optimization

25.1 Linear Programming
=======================

Octave can solve Linear Programming problems using the `glpk' function.
That is, Octave can solve

     min C'*x

   subject to the linear constraints A*x = b where x >= 0.

The `glpk' function also supports variations of this problem.

 -- Function File: [XOPT, FMIN, STATUS, EXTRA] = glpk (C, A, B, LB, UB,
          CTYPE, VARTYPE, SENSE, PARAM)
     Solve a linear program using the GNU GLPK library.  Given three
     arguments, `glpk' solves the following standard LP:

          min C'*x

     subject to

          A*x  = b
            x >= 0

     but may also solve problems of the form

          [ min | max ] C'*x

     subject to

          A*x [ "=" | "<=" | ">=" ] b
            x >= LB
            x <= UB

     Input arguments:

    C
          A column array containing the objective function coefficients.

    A
          A matrix containing the constraints coefficients.

    B
          A column array containing the right-hand side value for each
          constraint in the constraint matrix.

    LB
          An array containing the lower bound on each of the variables.
          If LB is not supplied, the default lower bound for the
          variables is zero.

    UB
          An array containing the upper bound on each of the variables.
          If UB is not supplied, the default upper bound is assumed to
          be infinite.

    CTYPE
          An array of characters containing the sense of each
          constraint in the constraint matrix.  Each element of the
          array may be one of the following values
         "F"
               A free (unbounded) constraint (the constraint is
               ignored).

         "U"
               An inequality constraint with an upper bound (`A(i,:)*x
               <= b(i)').

         "S"
               An equality constraint (`A(i,:)*x = b(i)').

         "L"
               An inequality with a lower bound (`A(i,:)*x >= b(i)').

         "D"
               An inequality constraint with both upper and lower bounds
               (`A(i,:)*x >= -b(i)' _and_ (`A(i,:)*x <= b(i)').

    VARTYPE
          A column array containing the types of the variables.
         "C"
               A continuous variable.

         "I"
               An integer variable.

    SENSE
          If SENSE is 1, the problem is a minimization.  If SENSE is
          -1, the problem is a maximization.  The default value is 1.

    PARAM
          A structure containing the following parameters used to
          define the behavior of solver.  Missing elements in the
          structure take on default values, so you only need to set the
          elements that you wish to change from the default.

          Integer parameters:

         `msglev (`LPX_K_MSGLEV', default: 1)'
               Level of messages output by solver routines:
              0
                    No output.

              1
                    Error messages only.

              2
                    Normal output.

              3
                    Full output (includes informational messages).

         `scale (`LPX_K_SCALE', default: 1)'
               Scaling option:
              0
                    No scaling.

              1
                    Equilibration scaling.

              2
                    Geometric mean scaling, then equilibration scaling.

         `dual    (`LPX_K_DUAL', default: 0)'
               Dual simplex option:
              0
                    Do not use the dual simplex.

              1
                    If initial basic solution is dual feasible, use the
                    dual simplex.

         `price   (`LPX_K_PRICE', default: 1)'
               Pricing option (for both primal and dual simplex):
              0
                    Textbook pricing.

              1
                    Steepest edge pricing.

         `round   (`LPX_K_ROUND', default: 0)'
               Solution rounding option:
              0
                    Report all primal and dual values "as is".

              1
                    Replace tiny primal and dual values by exact zero.

         `itlim   (`LPX_K_ITLIM', default: -1)'
               Simplex iterations limit.  If this value is positive, it
               is decreased by one each time when one simplex iteration
               has been performed, and reaching zero value signals the
               solver to stop the search.  Negative value means no
               iterations limit.

         `itcnt (`LPX_K_OUTFRQ', default: 200)'
               Output frequency, in iterations.  This parameter
               specifies how frequently the solver sends information
               about the solution to the standard output.

         `branch (`LPX_K_BRANCH', default: 2)'
               Branching heuristic option (for MIP only):
              0
                    Branch on the first variable.

              1
                    Branch on the last variable.

              2
                    Branch using a heuristic by Driebeck and Tomlin.

         `btrack (`LPX_K_BTRACK', default: 2)'
               Backtracking heuristic option (for MIP only):
              0
                    Depth first search.

              1
                    Breadth first search.

              2
                    Backtrack using the best projection heuristic.

         `presol (`LPX_K_PRESOL', default: 1)'
               If this flag is set, the routine lpx_simplex solves the
               problem using the built-in LP presolver.  Otherwise the
               LP presolver is not used.

         `lpsolver (default: 1)'
               Select which solver to use.  If the problem is a MIP
               problem this flag will be ignored.
              1
                    Revised simplex method.

              2
                    Interior point method.

         `save (default: 0)'
               If this parameter is nonzero, save a copy of the problem
               in CPLEX LP format to the file `"outpb.lp"'.  There is
               currently no way to change the name of the output file.

          Real parameters:

         `relax (`LPX_K_RELAX', default: 0.07)'
               Relaxation parameter used in the ratio test.  If it is
               zero, the textbook ratio test is used.  If it is
               non-zero (should be positive), Harris' two-pass ratio
               test is used.  In the latter case on the first pass of
               the ratio test basic variables (in the case of primal
               simplex) or reduced costs of non-basic variables (in the
               case of dual simplex) are allowed to slightly violate
               their bounds, but not more than `relax*tolbnd' or
               `relax*toldj (thus, `relax' is a percentage of `tolbnd'
               or `toldj''.

         `tolbnd (`LPX_K_TOLBND', default: 10e-7)'
               Relative tolerance used to check if the current basic
               solution is primal feasible.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

         `toldj (`LPX_K_TOLDJ', default: 10e-7)'
               Absolute tolerance used to check if the current basic
               solution is dual feasible.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

         `tolpiv (`LPX_K_TOLPIV', default: 10e-9)'
               Relative tolerance used to choose eligible pivotal
               elements of the simplex table.  It is not recommended
               that you change this parameter unless you have a
               detailed understanding of its purpose.

         `objll (`LPX_K_OBJLL', default: -DBL_MAX)'
               Lower limit of the objective function.  If on the phase
               II the objective function reaches this limit and
               continues decreasing, the solver stops the search.  This
               parameter is used in the dual simplex method only.

         `objul (`LPX_K_OBJUL', default: +DBL_MAX)'
               Upper limit of the objective function.  If on the phase
               II the objective function reaches this limit and
               continues increasing, the solver stops the search.  This
               parameter is used in the dual simplex only.

         `tmlim (`LPX_K_TMLIM', default: -1.0)'
               Searching time limit, in seconds.  If this value is
               positive, it is decreased each time when one simplex
               iteration has been performed by the amount of time spent
               for the iteration, and reaching zero value signals the
               solver to stop the search.  Negative value means no time
               limit.

         `outdly (`LPX_K_OUTDLY', default: 0.0)'
               Output delay, in seconds.  This parameter specifies how
               long the solver should delay sending information about
               the solution to the standard output.  Non-positive value
               means no delay.

         `tolint (`LPX_K_TOLINT', default: 10e-5)'
               Relative tolerance used to check if the current basic
               solution is integer feasible.  It is not recommended
               that you change this parameter unless you have a
               detailed understanding of its purpose.

         `tolobj (`LPX_K_TOLOBJ', default: 10e-7)'
               Relative tolerance used to check if the value of the
               objective function is not better than in the best known
               integer feasible solution.  It is not recommended that
               you change this parameter unless you have a detailed
               understanding of its purpose.

     Output values:

    XOPT
          The optimizer (the value of the decision variables at the
          optimum).

    FOPT
          The optimum value of the objective function.

    STATUS
          Status of the optimization.

          Simplex Method:
         180 (`LPX_OPT')
               Solution is optimal.

         181 (`LPX_FEAS')
               Solution is feasible.

         182 (`LPX_INFEAS')
               Solution is infeasible.

         183 (`LPX_NOFEAS')
               Problem has no feasible solution.

         184 (`LPX_UNBND')
               Problem has no unbounded solution.

         185 (`LPX_UNDEF')
               Solution status is undefined.
          Interior Point Method:
         150 (`LPX_T_UNDEF')
               The interior point method is undefined.

         151 (`LPX_T_OPT')
               The interior point method is optimal.
          Mixed Integer Method:
         170 (`LPX_I_UNDEF')
               The status is undefined.

         171 (`LPX_I_OPT')
               The solution is integer optimal.

         172 (`LPX_I_FEAS')
               Solution integer feasible but its optimality has not
               been proven

         173 (`LPX_I_NOFEAS')
               No integer feasible solution.
          If an error occurs, STATUS will contain one of the following
          codes:

         204 (`LPX_E_FAULT')
               Unable to start the search.

         205 (`LPX_E_OBJLL')
               Objective function lower limit reached.

         206 (`LPX_E_OBJUL')
               Objective function upper limit reached.

         207 (`LPX_E_ITLIM')
               Iterations limit exhausted.

         208 (`LPX_E_TMLIM')
               Time limit exhausted.

         209 (`LPX_E_NOFEAS')
               No feasible solution.

         210 (`LPX_E_INSTAB')
               Numerical instability.

         211 (`LPX_E_SING')
               Problems with basis matrix.

         212 (`LPX_E_NOCONV')
               No convergence (interior).

         213 (`LPX_E_NOPFS')
               No primal feasible solution (LP presolver).

         214 (`LPX_E_NODFS')
               No dual feasible solution (LP presolver).

    EXTRA
          A data structure containing the following fields:
         `lambda'
               Dual variables.

         `redcosts'
               Reduced Costs.

         `time'
               Time (in seconds) used for solving LP/MIP problem.

         `mem'
               Memory (in bytes) used for solving LP/MIP problem (this
               is not available if the version of GLPK is 4.15 or
               later).

     Example:

          c = [10, 6, 4]';
          A = [ 1, 1, 1;
               10, 4, 5;
                2, 2, 6];
          b = [100, 600, 300]';
          lb = [0, 0, 0]';
          ub = [];
          ctype = "UUU";
          vartype = "CCC";
          s = -1;

          param.msglev = 1;
          param.itlim = 100;

          [xmin, fmin, status, extra] = ...
             glpk (c, A, b, lb, ub, ctype, vartype, s, param);


File: octave.info,  Node: Quadratic Programming,  Next: Nonlinear Programming,  Prev: Linear Programming,  Up: Optimization

25.2 Quadratic Programming
==========================

Octave can also solve Quadratic Programming problems, this is

     min 0.5 x'*H*x + x'*q

   subject to

          A*x = b
          lb <= x <= ub
          A_lb <= A_in*x <= A_ub

 -- Function File: [X, OBJ, INFO, LAMBDA] = qp (X0, H)
 -- Function File: [X, OBJ, INFO, LAMBDA] = qp (X0, H, Q)
 -- Function File: [X, OBJ, INFO, LAMBDA] = qp (X0, H, Q, A, B)
 -- Function File: [X, OBJ, INFO, LAMBDA] = qp (X0, H, Q, A, B, LB, UB)
 -- Function File: [X, OBJ, INFO, LAMBDA] = qp (X0, H, Q, A, B, LB, UB,
          A_LB, A_IN, A_UB)
 -- Function File: [X, OBJ, INFO, LAMBDA] = qp (..., OPTIONS)
     Solve the quadratic program

               min 0.5 x'*H*x + x'*q
                x

     subject to

               A*x = b
               lb <= x <= ub
               A_lb <= A_in*x <= A_ub

     using a null-space active-set method.

     Any bound (A, B, LB, UB, A_LB, A_UB) may be set to the empty
     matrix (`[]') if not present.  If the initial guess is feasible
     the algorithm is faster.

    OPTIONS
          An optional structure containing the following parameter(s)
          used to define the behavior of the solver.  Missing elements
          in the structure take on default values, so you only need to
          set the elements that you wish to change from the default.

         `MaxIter (default: 200)'
               Maximum number of iterations.

    INFO
          Structure containing run-time information about the
          algorithm.  The following fields are defined:

         `solveiter'
               The number of iterations required to find the solution.

         `info'
               An integer indicating the status of the solution.

              0
                    The problem is feasible and convex.  Global
                    solution found.

              1
                    The problem is not convex.  Local solution found.

              2
                    The problem is not convex and unbounded.

              3
                    Maximum number of iterations reached.

              6
                    The problem is infeasible.

 -- Function File: X = pqpnonneg (C, D)
 -- Function File: X = pqpnonneg (C, D, X0)
 -- Function File: [X, MINVAL] = pqpnonneg (...)
 -- Function File: [X, MINVAL, EXITFLAG] = pqpnonneg (...)
 -- Function File: [X, MINVAL, EXITFLAG, OUTPUT] = pqpnonneg (...)
 -- Function File: [X, MINVAL, EXITFLAG, OUTPUT, LAMBDA] = pqpnonneg
          (...)
     Minimize `1/2*x'*c*x + d'*x' subject to `X >= 0'.  C and D must be
     real, and C must be symmetric and positive definite.  X0 is an
     optional initial guess for X.

     Outputs:
        * minval

          The minimum attained model value, 1/2*xmin'*c*xmin + d'*xmin

        * exitflag

          An indicator of convergence.  0 indicates that the iteration
          count was exceeded, and therefore convergence was not
          reached; >0 indicates that the algorithm converged.  (The
          algorithm is stable and will converge given enough
          iterations.)

        * output

          A structure with two fields:
             * "algorithm": The algorithm used ("nnls")

             * "iterations": The number of iterations taken.

        * lambda

          Not implemented.

     *See also:* *note optimset: doc-optimset, *note lsqnonneg:
     doc-lsqnonneg, *note qp: doc-qp.


File: octave.info,  Node: Nonlinear Programming,  Next: Linear Least Squares,  Prev: Quadratic Programming,  Up: Optimization

25.3 Nonlinear Programming
==========================

Octave can also perform general nonlinear minimization using a
successive quadratic programming solver.

 -- Function File: [X, OBJ, INFO, ITER, NF, LAMBDA] = sqp (X0, PHI)
 -- Function File: [...] = sqp (X0, PHI, G)
 -- Function File: [...] = sqp (X0, PHI, G, H)
 -- Function File: [...] = sqp (X0, PHI, G, H, LB, UB)
 -- Function File: [...] = sqp (X0, PHI, G, H, LB, UB, MAXITER)
 -- Function File: [...] = sqp (X0, PHI, G, H, LB, UB, MAXITER, TOL)
     Solve the nonlinear program

               min phi (x)
                x

     subject to

               g(x)  = 0
               h(x) >= 0
               lb <= x <= ub

     using a sequential quadratic programming method.

     The first argument is the initial guess for the vector X0.

     The second argument is a function handle pointing to the objective
     function PHI.  The objective function must accept one vector
     argument and return a scalar.

     The second argument may also be a 2- or 3-element cell array of
     function handles.  The first element should point to the objective
     function, the second should point to a function that computes the
     gradient of the objective function, and the third should point to a
     function that computes the Hessian of the objective function.  If
     the gradient function is not supplied, the gradient is computed by
     finite differences.  If the Hessian function is not supplied, a
     BFGS update formula is used to approximate the Hessian.

     When supplied, the gradient function `PHI{2}' must accept one
     vector argument and return a vector.  When supplied, the Hessian
     function `PHI{3}' must accept one vector argument and return a
     matrix.

     The third and fourth arguments G and H are function handles
     pointing to functions that compute the equality constraints and
     the inequality constraints, respectively.  If the problem does not
     have equality (or inequality) constraints, then use an empty
     matrix ([]) for G (or H).  When supplied, these equality and
     inequality constraint functions must accept one vector argument
     and return a vector.

     The third and fourth arguments may also be 2-element cell arrays of
     function handles.  The first element should point to the constraint
     function and the second should point to a function that computes
     the gradient of the constraint function:

                          [ d f(x)   d f(x)        d f(x) ]
              transpose ( [ ------   -----   ...   ------ ] )
                          [  dx_1     dx_2          dx_N  ]

     The fifth and sixth arguments, LB and UB, contain lower and upper
     bounds on X.  These must be consistent with the equality and
     inequality constraints G and H.  If the arguments are vectors then
     X(i) is bound by LB(i) and UB(i).  A bound can also be a scalar in
     which case all elements of X will share the same bound.  If only
     one bound (lb, ub) is specified then the other will default to
     (-REALMAX, +REALMAX).

     The seventh argument MAXITER specifies the maximum number of
     iterations.  The default value is 100.

     The eighth argument TOL specifies the tolerance for the stopping
     criteria.  The default value is `sqrt(eps)'.

     The value returned in INFO may be one of the following:

    101
          The algorithm terminated normally.  Either all constraints
          meet the requested tolerance, or the stepsize, delta X, is
          less than `TOL * norm (x)'.

    102
          The BFGS update failed.

    103
          The maximum number of iterations was reached.

     An example of calling `sqp':

          function r = g (x)
            r = [ sumsq(x)-10;
                  x(2)*x(3)-5*x(4)*x(5);
                  x(1)^3+x(2)^3+1 ];
          endfunction

          function obj = phi (x)
            obj = exp(prod(x)) - 0.5*(x(1)^3+x(2)^3+1)^2;
          endfunction

          x0 = [-1.8; 1.7; 1.9; -0.8; -0.8];

          [x, obj, info, iter, nf, lambda] = sqp (x0, @phi, @g, [])

          x =

            -1.71714
             1.59571
             1.82725
            -0.76364
            -0.76364

          obj = 0.053950
          info = 101
          iter = 8
          nf = 10
          lambda =

            -0.0401627
             0.0379578
            -0.0052227

     *See also:* *note qp: doc-qp.


File: octave.info,  Node: Linear Least Squares,  Prev: Nonlinear Programming,  Up: Optimization

25.4 Linear Least Squares
=========================

Octave also supports linear least squares minimization.  That is,
Octave can find the parameter b such that the model y = x*b fits data
(x,y) as well as possible, assuming zero-mean Gaussian noise.  If the
noise is assumed to be isotropic the problem can be solved using the
`\' or `/' operators, or the `ols' function.  In the general case where
the noise is assumed to be anisotropic the `gls' is needed.

 -- Function File: [BETA, SIGMA, R] = ols (Y, X)
     Ordinary least squares estimation for the multivariate model
     y = x*b + e with mean (e) = 0 and cov (vec (e)) = kron (s, I).
     where y is a t by p matrix, x is a t by k matrix, b is a k by p
     matrix, and e is a t by p matrix.

     Each row of Y and X is an observation and each column a variable.

     The return values BETA, SIGMA, and R are defined as follows.

    BETA
          The OLS estimator for b.  BETA is calculated directly via
          `inv (x'*x) * x' * y' if the matrix `x'*x' is of full rank.
          Otherwise, `BETA = pinv (X) * Y' where `pinv (X)' denotes the
          pseudoinverse of X.

    SIGMA
          The OLS estimator for the matrix S,

               SIGMA = (Y-X*BETA)'
                 * (Y-X*BETA)
                 / (T-rank(X))

    R
          The matrix of OLS residuals, `R = Y - X*BETA'.

     *See also:* *note gls: doc-gls, *note pinv: doc-pinv.

 -- Function File: [BETA, V, R] = gls (Y, X, O)
     Generalized least squares estimation for the multivariate model
     y = x*b + e with mean (e) = 0 and cov (vec (e)) = (s^2) o,  where
     y is a t by p matrix, x is a t by k matrix, b is a k by p matrix, e
     is a t by p matrix, and o is a t*p by t*p matrix.

     Each row of Y and X is an observation and each column a variable.
     The return values BETA, V, and R are defined as follows.

    BETA
          The GLS estimator for b.

    V
          The GLS estimator for s^2.

    R
          The matrix of GLS residuals, r = y - x*beta.

     *See also:* *note ols: doc-ols.

 -- Function File: X = lsqnonneg (C, D)
 -- Function File: X = lsqnonneg (C, D, X0)
 -- Function File: [X, RESNORM] = lsqnonneg (...)
 -- Function File: [X, RESNORM, RESIDUAL] = lsqnonneg (...)
 -- Function File: [X, RESNORM, RESIDUAL, EXITFLAG] = lsqnonneg (...)
 -- Function File: [X, RESNORM, RESIDUAL, EXITFLAG, OUTPUT] = lsqnonneg
          (...)
 -- Function File: [X, RESNORM, RESIDUAL, EXITFLAG, OUTPUT, LAMBDA] =
lsqnonneg (...)
     Minimize `norm (C*X - d)' subject to `X >= 0'.  C and D must be
     real.  X0 is an optional initial guess for X.

     Outputs:
        * resnorm

          The squared 2-norm of the residual: norm(C*X-D)^2

        * residual

          The residual: D-C*X

        * exitflag

          An indicator of convergence.  0 indicates that the iteration
          count was exceeded, and therefore convergence was not
          reached; >0 indicates that the algorithm converged.  (The
          algorithm is stable and will converge given enough
          iterations.)

        * output

          A structure with two fields:
             * "algorithm": The algorithm used ("nnls")

             * "iterations": The number of iterations taken.

        * lambda

          Not implemented.

     *See also:* *note optimset: doc-optimset, *note pqpnonneg:
     doc-pqpnonneg.

 -- Function File:  optimset ()
 -- Function File:  optimset (PAR, VAL, ...)
 -- Function File:  optimset (OLD, PAR, VAL, ...)
 -- Function File:  optimset (OLD, NEW)
     Create options struct for optimization functions.

     Valid parameters are:
        * AutoScaling

        * ComplexEqn

        * FinDiffType

        * FunValCheck When enabled, display an error if the objective
          function returns a complex value or NaN.  Must be set to "on"
          or "off" [default].

        * GradObj When set to "on", the function to be minimized must
          return a second argument which is the gradient, or first
          derivative, of the function at the point X.  If set to "off"
          [default], the gradient is computed via finite differences.

        * Jacobian When set to "on", the function to be minimized must
          return a second argument which is the Jacobian, or first
          derivative, of the function at the point X.  If set to "off"
          [default], the Jacobian is computed via finite differences.

        * MaxFunEvals Maximum number of function evaluations before
          optimization stops.  Must be a positive integer.

        * MaxIter Maximum number of algorithm iterations before
          optimization stops.  Must be a positive integer.

        * OutputFcn A user-defined function executed once per algorithm
          iteration.

        * TolFun Termination criterion for the function output.  If the
          difference in the calculated objective function between one
          algorithm iteration and the next is less than `TolFun' the
          optimization stops.  Must be a positive scalar.

        * TolX Termination criterion for the function input.  If the
          difference in X, the current search point, between one
          algorithm iteration and the next is less than `TolX' the
          optimization stops.  Must be a positive scalar.

        * TypicalX

        * Updating

 -- Function File:  optimget (OPTIONS, PARNAME)
 -- Function File:  optimget (OPTIONS, PARNAME, DEFAULT)
     Return a specific option from a structure created by `optimset'.
     If PARNAME is not a field of the OPTIONS structure, return DEFAULT
     if supplied, otherwise return an empty matrix.


File: octave.info,  Node: Statistics,  Next: Sets,  Prev: Optimization,  Up: Top

26 Statistics
*************

Octave has support for various statistical methods.  This includes
basic descriptive statistics, probability distributions, statistical
tests, random number generation, and much more.

   The functions that analyze data all assume that multi-dimensional
data is arranged in a matrix where each row is an observation, and each
column is a variable.  Thus, the matrix defined by

     a = [ 0.9, 0.7;
           0.1, 0.1;
           0.5, 0.4 ];

contains three observations from a two-dimensional distribution.  While
this is the default data arrangement, most functions support different
arrangements.

   It should be noted that the statistics functions don't test for data
containing NaN, NA, or Inf.  These values need to be detected and dealt
with explicitly.  See *note isnan: doc-isnan, *note isna: doc-isna,
*note isinf: doc-isinf, *note isfinite: doc-isfinite.

* Menu:

* Descriptive Statistics::
* Basic Statistical Functions::
* Statistical Plots::
* Correlation and Regression Analysis::
* Distributions::
* Tests::
* Random Number Generation::


File: octave.info,  Node: Descriptive Statistics,  Next: Basic Statistical Functions,  Up: Statistics

26.1 Descriptive Statistics
===========================

One principal goal of descriptive statistics is to represent the
essence of a large data set concisely.  Octave provides the mean,
median, and mode functions which all summarize a data set with just a
single number corresponding to the central tendency of the data.

 -- Function File:  mean (X)
 -- Function File:  mean (X, DIM)
 -- Function File:  mean (X, OPT)
 -- Function File:  mean (X, DIM, OPT)
     Compute the mean of the elements of the vector X.

          mean (x) = SUM_i x(i) / N

     If X is a matrix, compute the mean for each column and return them
     in a row vector.

     The optional argument OPT selects the type of mean to compute.
     The following options are recognized:

    "a"
          Compute the (ordinary) arithmetic mean.  [default]

    "g"
          Compute the geometric mean.

    "h"
          Compute the harmonic mean.

     If the optional argument DIM is given, operate along this
     dimension.

     Both DIM and OPT are optional.  If both are supplied, either may
     appear first.

     *See also:* *note median: doc-median, *note mode: doc-mode.

 -- Function File:  median (X)
 -- Function File:  median (X, DIM)
     Compute the median value of the elements of the vector X.  If the
     elements of X are sorted, the median is defined as

                      x(ceil(N/2)),             N odd
          median(x) =
                      (x(N/2) + x((N/2)+1))/2,  N even

     If X is a matrix, compute the median value for each column and
     return them in a row vector.  If the optional DIM argument is
     given, operate along this dimension.

     *See also:* *note mean: doc-mean, *note mode: doc-mode.

 -- Function File:  mode (X)
 -- Function File:  mode (X, DIM)
 -- Function File: [M, F, C] = mode (...)
     Compute the most frequently occurring value in a dataset (mode).
     `mode' determines the frequency of values along the first
     non-singleton dimension and returns the value with the highest
     frequency.  If two, or more, values have the same frequency `mode'
     returns the smallest.

     If the optional argument DIM is given, operate along this
     dimension.

     The return variable F is the number of occurrences of the mode in
     in the dataset.  The cell array C contains all of the elements
     with the maximum frequency.

     *See also:* *note mean: doc-mean, *note median: doc-median.

   Using just one number, such as the mean, to represent an entire data
set may not give an accurate picture of the data.  One way to
characterize the fit is to measure the dispersion of the data.  Octave
provides several functions for measuring dispersion.

 -- Function File:  range (X)
 -- Function File:  range (X, DIM)
     Return the range, i.e., the difference between the maximum and the
     minimum of the input data.  If X is a vector, the range is
     calculated over the elements of X.  If X is a matrix, the range is
     calculated over each column of X.

     If the optional argument DIM is given, operate along this
     dimension.

     The range is a quickly computed measure of the dispersion of a
     data set, but is less accurate than `iqr' if there are outlying
     data points.

     *See also:* *note iqr: doc-iqr, *note std: doc-std.

 -- Function File:  iqr (X)
 -- Function File:  iqr (X, DIM)
     Return the interquartile range, i.e., the difference between the
     upper and lower quartile of the input data.  If X is a matrix, do
     the above for first non-singleton dimension of X.

     If the optional argument DIM is given, operate along this
     dimension.

     As a measure of dispersion, the interquartile range is less
     affected by outliers than either `range' or `std'.

     *See also:* *note range: doc-range, *note std: doc-std.

 -- Function File:  meansq (X)
 -- Function File:  meansq (X, DIM)
     Compute the mean square of the elements of the vector X.

          std (x) = 1/N SUM_i x(i)^2

     For matrix arguments, return a row vector containing the mean
     square of each column.

     If the optional argument DIM is given, operate along this
     dimension.

     *See also:* *note var: doc-var, *note std: doc-std, *note moment:
     doc-moment.

 -- Function File:  std (X)
 -- Function File:  std (X, OPT)
 -- Function File:  std (X, OPT, DIM)
     Compute the standard deviation of the elements of the vector X.

          std (x) = sqrt ( 1/(N-1) SUM_i (x(i) - mean(x))^2 )

     where N is the number of elements.

     If X is a matrix, compute the standard deviation for each column
     and return them in a row vector.

     The argument OPT determines the type of normalization to use.
     Valid values are

    0:
          normalize with N-1, provides the square root of the best
          unbiased estimator of the variance [default]

    1:
          normalize with N, this provides the square root of the second
          moment around the mean

     If the optional argument DIM is given, operate along this
     dimension.

     *See also:* *note var: doc-var, *note range: doc-range, *note iqr:
     doc-iqr, *note mean: doc-mean, *note median: doc-median.

   In addition to knowing the size of a dispersion it is useful to know
the shape of the data set.  For example, are data points massed to the
left or right of the mean?  Octave provides several common measures to
describe the shape of the data set.  Octave can also calculate moments
allowing arbitrary shape measures to be developed.

 -- Function File:  var (X)
 -- Function File:  var (X, OPT)
 -- Function File:  var (X, OPT, DIM)
     Compute the variance of the elements of the vector X.

          var (x) = 1/(N-1) SUM_i (x(i) - mean(x))^2

     If X is a matrix, compute the variance for each column and return
     them in a row vector.

     The argument OPT determines the type of normalization to use.
     Valid values are

    0:
          normalize with N-1, provides the best unbiased estimator of
          the variance [default]

    1:
          normalizes with N, this provides the second moment around the
          mean

     If the optional argument DIM is given, operate along this
     dimension.

     *See also:* *note cov: doc-cov, *note std: doc-std, *note
     skewness: doc-skewness, *note kurtosis: doc-kurtosis, *note
     moment: doc-moment.

 -- Function File:  skewness (X)
 -- Function File:  skewness (X, DIM)
     Compute the skewness of the elements of the vector X.

          skewness (x) = N^(-1) std(x)^(-3) sum ((x - mean(x)).^3)

     If X is a matrix, return the skewness along the first
     non-singleton dimension of the matrix.  If the optional DIM
     argument is given, operate along this dimension.

     *See also:* *note var: doc-var, *note kurtosis: doc-kurtosis,
     *note moment: doc-moment.

 -- Function File:  kurtosis (X)
 -- Function File:  kurtosis (X, DIM)
     Compute the kurtosis of the elements of the vector X.

          kurtosis (x) = N^(-1) std(x)^(-4) sum ((x - mean(x)).^4) - 3

     If X is a matrix, return the kurtosis over the first non-singleton
     dimension of the matrix.  If the optional DIM argument is given,
     operate along this dimension.

     Note: The definition of kurtosis above yields a kurtosis of zero
     for the stdnormal distribution and is sometimes referred to as
     "excess kurtosis".  To calculate kurtosis without the
     normalization factor of -3 use `moment (X, 4, 'c') / std (X)^4'.

     *See also:* *note var: doc-var, *note skewness: doc-skewness,
     *note moment: doc-moment.

 -- Function File:  moment (X, P)
 -- Function File:  moment (X, P, TYPE)
 -- Function File:  moment (X, P, DIM)
 -- Function File:  moment (X, P, TYPE, DIM)
 -- Function File:  moment (X, P, DIM, TYPE)
     Compute the P-th moment of the vector X about zero.

          moment (x) = 1/N SUM_i x(i)^p

     If X is a matrix, return the row vector containing the P-th moment
     of each column.

     The optional string TYPE specifies the type of moment to be
     computed.  Valid options are:
    "c"
          Central Moment.  The moment about the mean defined as

               1/N SUM_i (x(i) - mean(x))^p

    "a"
          Absolute Moment.  The moment about zero ignoring sign defined
          as

               1/N SUM_i ( abs(x(i)) )^p

    "ac"
          Absolute Central Moment.  Defined as

               1/N SUM_i ( abs(x(i) - mean(x)) )^p


     If the optional argument DIM is given, operate along this
     dimension.

     If both TYPE and DIM are given they may appear in any order.

     *See also:* *note var: doc-var, *note skewness: doc-skewness,
     *note kurtosis: doc-kurtosis.

 -- Function File: Q = quantile (X, P)
 -- Function File: Q = quantile (X, P, DIM)
 -- Function File: Q = quantile (X, P, DIM, METHOD)
     For a sample, X, calculate the quantiles, Q, corresponding to the
     cumulative probability values in P.  All non-numeric values (NaNs)
     of X are ignored.

     If X is a matrix, compute the quantiles for each column and return
     them in a matrix, such that the i-th row of Q contains the P(i)th
     quantiles of each column of X.

     The optional argument DIM determines the dimension along which the
     quantiles are calculated.  If DIM is omitted, and X is a vector or
     matrix, it defaults to 1 (column-wise quantiles).  If X is an N-D
     array, DIM defaults to the first non-singleton dimension.

     The methods available to calculate sample quantiles are the nine
     methods used by R (http://www.r-project.org/).  The default value
     is METHOD = 5.

     Discontinuous sample quantile methods 1, 2, and 3

       1. Method 1: Inverse of empirical distribution function.

       2. Method 2: Similar to method 1 but with averaging at
          discontinuities.

       3. Method 3: SAS definition: nearest even order statistic.

     Continuous sample quantile methods 4 through 9, where p(k) is the
     linear interpolation function respecting each methods'
     representative cdf.

       4. Method 4: p(k) = k / n. That is, linear interpolation of the
          empirical cdf.

       5. Method 5: p(k) = (k - 0.5) / n. That is a piecewise linear
          function where the knots are the values midway through the
          steps of the empirical cdf.

       6. Method 6: p(k) = k / (n + 1).

       7. Method 7: p(k) = (k - 1) / (n - 1).

       8. Method 8: p(k) = (k - 1/3) / (n + 1/3).  The resulting
          quantile estimates are approximately median-unbiased
          regardless of the distribution of X.

       9. Method 9: p(k) = (k - 3/8) / (n + 1/4).  The resulting
          quantile estimates are approximately unbiased for the
          expected order statistics if X is normally distributed.

     Hyndman and Fan (1996) recommend method 8.  Maxima, S, and R
     (versions prior to 2.0.0) use 7 as their default.  Minitab and SPSS
     use method 6.  MATLAB uses method 5.

     References:

        * Becker, R. A., Chambers, J. M. and Wilks, A. R. (1988) The New
          S Language.  Wadsworth & Brooks/Cole.

        * Hyndman, R. J. and Fan, Y. (1996) Sample quantiles in
          statistical packages, American Statistician, 50, 361-365.

        * R: A Language and Environment for Statistical Computing;
          `http://cran.r-project.org/doc/manuals/fullrefman.pdf'.

     Examples:

          x = randi (1000, [10, 1]);  # Create empirical data in range 1-1000
          q = quantile (x, [0, 1]);   # Return minimum, maximum of distribution
          q = quantile (x, [0.25 0.5 0.75]); # Return quartiles of distribution

     *See also:* *note prctile: doc-prctile.

 -- Function File: Q = prctile (X)
 -- Function File: Q = prctile (X, P)
 -- Function File: Q = prctile (X, P, DIM)
     For a sample X, compute the quantiles, Q, corresponding to the
     cumulative probability values, P, in percent.  All non-numeric
     values (NaNs) of X are ignored.

     If X is a matrix, compute the percentiles for each column and
     return them in a matrix, such that the i-th row of Y contains the
     P(i)th percentiles of each column of X.

     If P is unspecified, return the quantiles for `[0 25 50 75 100]'.
     The optional argument DIM determines the dimension along which the
     percentiles are calculated.  If DIM is omitted, and X is a vector
     or matrix, it defaults to 1 (column-wise quantiles).  When X is an
     N-D array, DIM defaults to the first non-singleton dimension.

     *See also:* *note quantile: doc-quantile.

   A summary view of a data set can be generated quickly with the
`statistics' function.

 -- Function File:  statistics (X)
 -- Function File:  statistics (X, DIM)
     Return a vector with the minimum, first quartile, median, third
     quartile, maximum, mean, standard deviation, skewness, and
     kurtosis of the elements of the vector X.

     If X is a matrix, calculate statistics over the first
     non-singleton dimension.  If the optional argument DIM is given,
     operate along this dimension.

     *See also:* *note min: doc-min, *note max: doc-max, *note median:
     doc-median, *note mean: doc-mean, *note std: doc-std, *note
     skewness: doc-skewness, *note kurtosis: doc-kurtosis.


File: octave.info,  Node: Basic Statistical Functions,  Next: Statistical Plots,  Prev: Descriptive Statistics,  Up: Statistics

26.2 Basic Statistical Functions
================================

Octave supports various helpful statistical functions.  Many are useful
as initial steps to prepare a data set for further analysis.  Others
provide different measures from those of the basic descriptive
statistics.

 -- Function File:  center (X)
 -- Function File:  center (X, DIM)
     If X is a vector, subtract its mean.  If X is a matrix, do the
     above for each column.  If the optional argument DIM is given,
     operate along this dimension.

     *See also:* *note zscore: doc-zscore.

 -- Function File:  zscore (X)
 -- Function File:  zscore (X, DIM)
     If X is a vector, subtract its mean and divide by its standard
     deviation.

     If X is a matrix, do the above along the first non-singleton
     dimension.  If the optional argument DIM is given, operate along
     this dimension.

     *See also:* *note center: doc-center.

 -- Function File: N = histc (X, EDGES)
 -- Function File: N = histc (X, EDGES, DIM)
 -- Function File: [N, IDX] = histc (...)
     Produce histogram counts.

     When X is a vector, the function counts the number of elements of
     X that fall in the histogram bins defined by EDGES.  This must be
     a vector of monotonically increasing values that define the edges
     of the histogram bins.  `N(k)' contains the number of elements in
     X for which `EDGES(k) <= X < EDGES(k+1)'.  The final element of N
     contains the number of elements of X exactly equal to the last
     element of EDGES.

     When X is an N-dimensional array, the computation is carried out
     along dimension DIM.  If not specified DIM defaults to the first
     non-singleton dimension.

     When a second output argument is requested an index matrix is also
     returned.  The IDX matrix has the same size as X.  Each element of
     IDX contains the index of the histogram bin in which the
     corresponding element of X was counted.

     *See also:* *note hist: doc-hist.

 -- Function File: C = nchoosek (N, K)
 -- Function File: C = nchoosek (SET, K)
     Compute the binomial coefficient or all combinations of a set of
     items.

     If N is a scalar then calculate the binomial coefficient of N and
     K which is defined as

           /   \
           | n |    n (n-1) (n-2) ... (n-k+1)       n!
           |   |  = ------------------------- =  ---------
           | k |               k!                k! (n-k)!
           \   /

     This is the number of combinations of N items taken in groups of
     size K.

     If the first argument is a vector, SET, then generate all
     combinations of the elements of SET, taken K at a time, with one
     row per combination.  The result C has K columns and
     `nchoosek (length (SET), K)' rows.

     For example:

     How many ways can three items be grouped into pairs?

          nchoosek (3, 2)
             => 3

     What are the possible pairs?

          nchoosek (1:3, 2)
             =>  1   2
                 1   3
                 2   3

     `nchoosek' works only for non-negative, integer arguments.  Use
     `bincoeff' for non-integer and negative scalar arguments, or for
     computing many binomial coefficients at once with vector inputs
     for N or K.

     *See also:* *note bincoeff: doc-bincoeff, *note perms: doc-perms.

 -- Function File:  perms (V)
     Generate all permutations of V, one row per permutation.  The
     result has size `factorial (N) * N', where N is the length of V.

     As an example, `perms([1, 2, 3])' returns the matrix

            1   2   3
            2   1   3
            1   3   2
            2   3   1
            3   1   2
            3   2   1

 -- Function File:  ranks (X, DIM)
     Return the ranks of X along the first non-singleton dimension
     adjusted for ties.  If the optional argument DIM is given, operate
     along this dimension.

     *See also:* *note spearman: doc-spearman, *note kendall:
     doc-kendall.

 -- Function File:  run_count (X, N)
 -- Function File:  run_count (X, N, DIM)
     Count the upward runs along the first non-singleton dimension of X
     of length 1, 2, ..., N-1 and greater than or equal to N.

     If the optional argument DIM is given then operate along this
     dimension.

 -- Function File: [count, value] = runlength (X)
     Find the lengths of all sequences of common values.  Return the
     vector of lengths and the value that was repeated.

          runlength ([2, 2, 0, 4, 4, 4, 0, 1, 1, 1, 1])
          =>  [2, 1, 3, 1, 4]

 -- Function File:  probit (P)
     For each component of P, return the probit (the quantile of the
     standard normal distribution) of P.

 -- Function File:  logit (P)
     For each component of P, return the logit of P defined as

          logit(P) = log (P / (1-P))

     *See also:* *note logistic_cdf: doc-logistic_cdf.

 -- Function File:  cloglog (X)
     Return the complementary log-log function of X, defined as

          cloglog(x) = - log (- log (X))


 -- Function File:  mahalanobis (X, Y)
     Return the Mahalanobis' D-square distance between the multivariate
     samples X and Y, which must have the same number of components
     (columns), but may have a different number of observations (rows).

 -- Function File: [T, L_X] = table (X)
 -- Function File: [T, L_X, L_Y] = table (X, Y)
     Create a contingency table T from data vectors.  The L_X and L_Y
     vectors are the corresponding levels.

     Currently, only 1- and 2-dimensional tables are supported.


File: octave.info,  Node: Statistical Plots,  Next: Correlation and Regression Analysis,  Prev: Basic Statistical Functions,  Up: Statistics

26.3 Statistical Plots
======================

Octave can create Quantile Plots (QQ-Plots), and Probability Plots
(PP-Plots).  These are simple graphical tests for determining if a data
set comes from a certain distribution.

   Note that Octave can also show histograms of data using the `hist'
function as described in *note Two-Dimensional Plots::.

 -- Function File: [Q, S] = qqplot (X)
 -- Function File: [Q, S] = qqplot (X, DIST)
 -- Function File: [Q, S] = qqplot (X, DIST, PARAMS)
 -- Function File:  qqplot (...)
     Perform a QQ-plot (quantile plot).

     If F is the CDF of the distribution DIST with parameters PARAMS
     and G its inverse, and X a sample vector of length N, the QQ-plot
     graphs ordinate S(I) = I-th largest element of x versus abscissa
     Q(If) = G((I - 0.5)/N).

     If the sample comes from F, except for a transformation of location
     and scale, the pairs will approximately follow a straight line.

     The default for DIST is the standard normal distribution.  The
     optional argument PARAMS contains a list of parameters of DIST.
     For example, for a quantile plot of the uniform distribution on
     [2,4] and X, use

          qqplot (x, "unif", 2, 4)

     DIST can be any string for which a function DISTINV or DIST_INV
     exists that calculates the inverse CDF of distribution DIST.

     If no output arguments are given, the data are plotted directly.

 -- Function File: [P, Y] = ppplot (X, DIST, PARAMS)
     Perform a PP-plot (probability plot).

     If F is the CDF of the distribution DIST with parameters PARAMS
     and X a sample vector of length N, the PP-plot graphs ordinate
     Y(I) = F (I-th largest element of X) versus abscissa P(I) = (I -
     0.5)/N.  If the sample comes from F, the pairs will approximately
     follow a straight line.

     The default for DIST is the standard normal distribution.  The
     optional argument PARAMS contains a list of parameters of DIST.
     For example, for a probability plot of the uniform distribution on
     [2,4] and X, use

          ppplot (x, "uniform", 2, 4)

     DIST can be any string for which a function DIST_CDF that
     calculates the CDF of distribution DIST exists.

     If no output arguments are given, the data are plotted directly.


File: octave.info,  Node: Correlation and Regression Analysis,  Next: Distributions,  Prev: Statistical Plots,  Up: Statistics

26.4 Correlation and Regression Analysis
========================================

 -- Function File:  cov (X)
 -- Function File:  cov (X, OPT)
 -- Function File:  cov (X, Y)
 -- Function File:  cov (X, Y, OPT)
     Compute the covariance matrix.

     If each row of X and Y is an observation, and each column is a
     variable, then the (I, J)-th entry of `cov (X, Y)' is the
     covariance between the I-th variable in X and the J-th variable in
     Y.

          cov (x) = 1/N-1 * SUM_i (x(i) - mean(x)) * (y(i) - mean(y))

     If called with one argument, compute `cov (X, X)', the covariance
     between the columns of X.

     The argument OPT determines the type of normalization to use.
     Valid values are

    0:
          normalize with N-1, provides the best unbiased estimator of
          the covariance [default]

    1:
          normalize with N, this provides the second moment around the
          mean

     *See also:* *note corr: doc-corr.

 -- Function File:  corr (X)
 -- Function File:  corr (X, Y)
     Compute matrix of correlation coefficients.

     If each row of X and Y is an observation and each column is a
     variable, then the (I, J)-th entry of `corr (X, Y)' is the
     correlation between the I-th variable in X and the J-th variable
     in Y.

          corr(x,y) = cov(x,y)/(std(x)*std(y))

     If called with one argument, compute `corr (X, X)', the
     correlation between the columns of X.

     *See also:* *note cov: doc-cov.

 -- Function File:  spearman (X)
 -- Function File:  spearman (X, Y)
     Compute Spearman's rank correlation coefficient RHO.

     For two data vectors X and Y, Spearman's RHO is the correlation
     coefficient of the ranks of X and Y.

     If X and Y are drawn from independent distributions, RHO has zero
     mean and variance `1 / (n - 1)', and is asymptotically normally
     distributed.

     `spearman (X)' is equivalent to `spearman (X, X)'.

     *See also:* *note ranks: doc-ranks, *note kendall: doc-kendall.

 -- Function File:  kendall (X)
 -- Function File:  kendall (X, Y)
     Compute Kendall's TAU.

     For two data vectors X, Y of common length N, Kendall's TAU is the
     correlation of the signs of all rank differences of X and Y; i.e.,
     if both X and Y have distinct entries, then

                   1
          tau = -------   SUM sign (q(i) - q(j)) * sign (r(i) - r(j))
                n (n-1)   i,j

     in which the Q(I) and R(I) are the ranks of X and Y, respectively.

     If X and Y are drawn from independent distributions, Kendall's TAU
     is asymptotically normal with mean 0 and variance `(2 * (2N+5)) /
     (9 * N * (N-1))'.

     `kendall (X)' is equivalent to `kendall (X, X)'.

     *See also:* *note ranks: doc-ranks, *note spearman: doc-spearman.

 -- Function File: [THETA, BETA, DEV, DL, D2L, P] = logistic_regression
          (Y, X, PRINT, THETA, BETA)
     Perform ordinal logistic regression.

     Suppose Y takes values in K ordered categories, and let `gamma_i
     (X)' be the cumulative probability that Y falls in one of the
     first I categories given the covariate X.  Then

          [theta, beta] = logistic_regression (y, x)

     fits the model

          logit (gamma_i (x)) = theta_i - beta' * x,   i = 1 ... k-1

     The number of ordinal categories, K, is taken to be the number of
     distinct values of `round (Y)'.  If K equals 2, Y is binary and
     the model is ordinary logistic regression.  The matrix X is
     assumed to have full column rank.

     Given Y only, `theta = logistic_regression (y)' fits the model
     with baseline logit odds only.

     The full form is

          [theta, beta, dev, dl, d2l, gamma]
             = logistic_regression (y, x, print, theta, beta)

     in which all output arguments and all input arguments except Y are
     optional.

     Setting PRINT to 1 requests summary information about the fitted
     model to be displayed.  Setting PRINT to 2 requests information
     about convergence at each iteration.  Other values request no
     information to be displayed.  The input arguments THETA and BETA
     give initial estimates for THETA and BETA.

     The returned value DEV holds minus twice the log-likelihood.

     The returned values DL and D2L are the vector of first and the
     matrix of second derivatives of the log-likelihood with respect to
     THETA and BETA.

     P holds estimates for the conditional distribution of Y given X.


File: octave.info,  Node: Distributions,  Next: Tests,  Prev: Correlation and Regression Analysis,  Up: Statistics

26.5 Distributions
==================

Octave has functions for computing the Probability Density Function
(PDF), the Cumulative Distribution function (CDF), and the quantile
(the inverse of the CDF) for a large number of distributions.

   The following table summarizes the supported distributions (in
alphabetical order).

Distribution           PDF               CDF               Quantile
----------------------------------------------------------------------------- 
Beta Distribution      `betapdf'         `betacdf'         `betainv'
Binomial Distribution  `binopdf'         `binocdf'         `binoinv'
Cauchy Distribution    `cauchy_pdf'      `cauchy_cdf'      `cauchy_inv'
Chi-Square             `chi2pdf'         `chi2cdf'         `chi2inv'
Distribution                                               
Univariate Discrete    `discrete_pdf'    `discrete_cdf'    `discrete_inv'
Distribution                                               
Empirical Distribution `empirical_pdf'   `empirical_cdf'   `empirical_inv'
Exponential            `exppdf'          `expcdf'          `expinv'
Distribution                                               
F Distribution         `fpdf'            `fcdf'            `finv'
Gamma Distribution     `gampdf'          `gamcdf'          `gaminv'
Geometric Distribution `geopdf'          `geocdf'          `geoinv'
Hypergeometric         `hygepdf'         `hygecdf'         `hygeinv'
Distribution                                               
Kolmogorov Smirnov     _Not Available_   `kolmogorov_smirnov_cdf'_Not Available_
Distribution                                               
Laplace Distribution   `laplace_pdf'     `laplace_cdf'     `laplace_inv'
Logistic Distribution  `logistic_pdf'    `logistic_cdf'    `logistic_inv'
Log-Normal             `lognpdf'         `logncdf'         `logninv'
Distribution                                               
Univariate Normal      `normpdf'         `normcdf'         `norminv'
Distribution                                               
Pascal Distribution    `nbinpdf'         `nbincdf'         `nbininv'
Poisson Distribution   `poisspdf'        `poisscdf'        `poissinv'
Standard Normal        `stdnormal_pdf'   `stdnormal_cdf'   `stdnormal_inv'
Distribution                                               
t (Student)            `tpdf'            `tcdf'            `tinv'
Distribution                                               
Univariate Discrete    `unidpdf'         `unidcdf'         `unidinv'
Distribution                                               
Uniform Distribution   `unifpdf'         `unifcdf'         `unifinv'
Weibull Distribution   `wblpdf'          `wblcdf'          `wblinv'

 -- Function File:  betapdf (X, A, B)
     For each element of X, compute the probability density function
     (PDF) at X of the Beta distribution with parameters A and B.

 -- Function File:  betacdf (X, A, B)
     For each element of X, compute the cumulative distribution function
     (CDF) at X of the Beta distribution with parameters A and B.

 -- Function File:  betainv (X, A, B)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the Beta distribution with parameters A and B.

 -- Function File:  binopdf (X, N, P)
     For each element of X, compute the probability density function
     (PDF) at X of the binomial distribution with parameters N and P,
     where N is the number of trials and P is the probability of
     success.

 -- Function File:  binocdf (X, N, P)
     For each element of X, compute the cumulative distribution function
     (CDF) at X of the binomial distribution with parameters N and P,
     where N is the number of trials and P is the probability of
     success.

 -- Function File:  binoinv (X, N, P)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the binomial distribution with parameters N and P,
     where N is the number of trials and P is the probability of
     success.

 -- Function File:  cauchy_pdf (X)
 -- Function File:  cauchy_pdf (X, LOCATION, SCALE)
     For each element of X, compute the probability density function
     (PDF) at X of the Cauchy distribution with location parameter
     LOCATION and scale parameter SCALE > 0.  Default values are
     LOCATION = 0, SCALE = 1.

 -- Function File:  cauchy_cdf (X)
 -- Function File:  cauchy_cdf (X, LOCATION, SCALE)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the Cauchy distribution with location
     parameter LOCATION and scale parameter SCALE.  Default values are
     LOCATION = 0, SCALE = 1.

 -- Function File:  cauchy_inv (X)
 -- Function File:  cauchy_inv (X, LOCATION, SCALE)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the Cauchy distribution with location parameter
     LOCATION and scale parameter SCALE.  Default values are LOCATION =
     0, SCALE = 1.

 -- Function File:  chi2pdf (X, N)
     For each element of X, compute the probability density function
     (PDF) at X of the chi-square distribution with N degrees of
     freedom.

 -- Function File:  chi2cdf (X, N)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the chi-square distribution with N degrees
     of freedom.

 -- Function File:  chi2inv (X, N)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the chi-square distribution with N degrees of freedom.

 -- Function File:  discrete_pdf (X, V, P)
     For each element of X, compute the probability density function
     (PDF) at X of a univariate discrete distribution which assumes the
     values in V with probabilities P.

 -- Function File:  discrete_cdf (X, V, P)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of a univariate discrete distribution which
     assumes the values in V with probabilities P.

 -- Function File:  discrete_inv (X, V, P)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the univariate distribution which assumes the values
     in V with probabilities P.

 -- Function File:  empirical_pdf (X, DATA)
     For each element of X, compute the probability density function
     (PDF) at X of the empirical distribution obtained from the
     univariate sample DATA.

 -- Function File:  empirical_cdf (X, DATA)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the empirical distribution obtained from
     the univariate sample DATA.

 -- Function File:  empirical_inv (X, DATA)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the empirical distribution obtained from the
     univariate sample DATA.

 -- Function File:  exppdf (X, LAMBDA)
     For each element of X, compute the probability density function
     (PDF) at X of the exponential distribution with mean LAMBDA.

 -- Function File:  expcdf (X, LAMBDA)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the exponential distribution with mean
     LAMBDA.

     The arguments can be of common size or scalars.

 -- Function File:  expinv (X, LAMBDA)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the exponential distribution with mean LAMBDA.

 -- Function File:  fpdf (X, M, N)
     For each element of X, compute the probability density function
     (PDF) at X of the F distribution with M and N degrees of freedom.

 -- Function File:  fcdf (X, M, N)
     For each element of X, compute the cumulative distribution function
     (CDF) at X of the F distribution with M and N degrees of freedom.

 -- Function File:  finv (X, M, N)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the F distribution with M and N degrees of freedom.

 -- Function File:  gampdf (X, A, B)
     For each element of X, return the probability density function
     (PDF) at X of the Gamma distribution with shape parameter A and
     scale B.

 -- Function File:  gamcdf (X, A, B)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the Gamma distribution with shape parameter
     A and scale B.

 -- Function File:  gaminv (X, A, B)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the Gamma distribution with shape parameter A and
     scale B.

 -- Function File:  geopdf (X, P)
     For each element of X, compute the probability density function
     (PDF) at X of the geometric distribution with parameter P.

 -- Function File:  geocdf (X, P)
     For each element of X, compute the cumulative distribution function
     (CDF) at X of the geometric distribution with parameter P.

 -- Function File:  geoinv (X, P)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the geometric distribution with parameter P.

 -- Function File:  hygepdf (X, T, M, N)
     Compute the probability density function (PDF) at X of the
     hypergeometric distribution with parameters T, M, and N.  This is
     the probability of obtaining X marked items when randomly drawing
     a sample of size N without replacement from a population of total
     size T containing M marked items.

     The parameters T, M, and N must be positive integers with M and N
     not greater than T.

 -- Function File:  hygecdf (X, T, M, N)
     Compute the cumulative distribution function (CDF) at X of the
     hypergeometric distribution with parameters T, M, and N.  This is
     the probability of obtaining not more than X marked items when
     randomly drawing a sample of size N without replacement from a
     population of total size T containing M marked items.

     The parameters T, M, and N must be positive integers with M and N
     not greater than T.

 -- Function File:  hygeinv (X, T, M, N)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the hypergeometric distribution with parameters T, M,
     and N.  This is the probability of obtaining X marked items when
     randomly drawing a sample of size N without replacement from a
     population of total size T containing M marked items.

     The parameters T, M, and N must be positive integers with M and N
     not greater than T.

 -- Function File:  kolmogorov_smirnov_cdf (X, TOL)
     Return the cumulative distribution function (CDF) at X of the
     Kolmogorov-Smirnov distribution,

                   Inf
          Q(x) =   SUM    (-1)^k exp (-2 k^2 x^2)
                 k = -Inf

     for X > 0.

     The optional parameter TOL specifies the precision up to which the
     series should be evaluated; the default is TOL = `eps'.

 -- Function File:  laplace_pdf (X)
     For each element of X, compute the probability density function
     (PDF) at X of the Laplace distribution.

 -- Function File:  laplace_cdf (X)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the Laplace distribution.

 -- Function File:  laplace_inv (X)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the Laplace distribution.

 -- Function File:  logistic_pdf (X)
     For each element of X, compute the PDF at X of the logistic
     distribution.

 -- Function File:  logistic_cdf (X)
     For each element of X, compute the cumulative distribution function
     (CDF) at X of the logistic distribution.

 -- Function File:  logistic_inv (X)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the logistic distribution.

 -- Function File:  lognpdf (X)
 -- Function File:  lognpdf (X, MU, SIGMA)
     For each element of X, compute the probability density function
     (PDF) at X of the lognormal distribution with parameters MU and
     SIGMA.  If a random variable follows this distribution, its
     logarithm is normally distributed with mean MU and standard
     deviation SIGMA.

     Default values are MU = 1, SIGMA = 1.

 -- Function File:  logncdf (X)
 -- Function File:  logncdf (X, MU, SIGMA)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the lognormal distribution with parameters
     MU and SIGMA.  If a random variable follows this distribution, its
     logarithm is normally distributed with mean MU and standard
     deviation SIGMA.

     Default values are MU = 1, SIGMA = 1.

 -- Function File:  logninv (X)
 -- Function File:  logninv (X, MU, SIGMA)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the lognormal distribution with parameters MU and
     SIGMA.  If a random variable follows this distribution, its
     logarithm is normally distributed with mean `log (MU)' and
     variance SIGMA.

     Default values are MU = 1, SIGMA = 1.

 -- Function File:  nbinpdf (X, N, P)
     For each element of X, compute the probability density function
     (PDF) at X of the negative binomial distribution with parameters N
     and P.

     When N is integer this is the Pascal distribution.  When N is
     extended to real numbers this is the Polya distribution.

     The number of failures in a Bernoulli experiment with success
     probability P before the N-th success follows this distribution.

 -- Function File:  nbincdf (X, N, P)
     For each element of X, compute the cumulative distribution function
     (CDF) at X of the negative binomial distribution with parameters N
     and P.

     When N is integer this is the Pascal distribution.  When N is
     extended to real numbers this is the Polya distribution.

     The number of failures in a Bernoulli experiment with success
     probability P before the N-th success follows this distribution.

 -- Function File:  nbininv (X, N, P)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the negative binomial distribution with parameters N
     and P.

     When N is integer this is the Pascal distribution.  When N is
     extended to real numbers this is the Polya distribution.

     The number of failures in a Bernoulli experiment with success
     probability P before the N-th success follows this distribution.

 -- Function File:  normpdf (X)
 -- Function File:  normpdf (X, MU, SIGMA)
     For each element of X, compute the probability density function
     (PDF) at X of the normal distribution with mean MU and standard
     deviation SIGMA.

     Default values are MU = 0, SIGMA = 1.

 -- Function File:  normcdf (X)
 -- Function File:  normcdf (X, MU, SIGMA)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the normal distribution with mean MU and
     standard deviation SIGMA.

     Default values are MU = 0, SIGMA = 1.

 -- Function File:  norminv (X)
 -- Function File:  norminv (X, MU, SIGMA)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the normal distribution with mean MU and standard
     deviation SIGMA.

     Default values are MU = 0, SIGMA = 1.

 -- Function File:  poisspdf (X, LAMBDA)
     For each element of X, compute the probability density function
     (PDF) at X of the Poisson distribution with parameter LAMBDA.

 -- Function File:  poisscdf (X, LAMBDA)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the Poisson distribution with parameter
     lambda.

 -- Function File:  poissinv (X, LAMBDA)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the Poisson distribution with parameter LAMBDA.

 -- Function File:  stdnormal_pdf (X)
     For each element of X, compute the probability density function
     (PDF) at X of the standard normal distribution (mean = 0, standard
     deviation = 1).

 -- Function File:  stdnormal_cdf (X)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the standard normal distribution (mean = 0,
     standard deviation = 1).

 -- Function File:  stdnormal_inv (X)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the standard normal distribution (mean = 0, standard
     deviation = 1).

 -- Function File:  tpdf (X, N)
     For each element of X, compute the probability density function
     (PDF) at X of the T (Student) distribution with N degrees of
     freedom.

 -- Function File:  tcdf (X, N)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the t (Student) distribution with N degrees
     of freedom, i.e., PROB (t(N) <= X).

 -- Function File:  tinv (X, N)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the t (Student) distribution with N degrees of
     freedom.  This function is analogous to looking in a table for the
     t-value of a single-tailed distribution.

 -- Function File:  unidpdf (X, N)
     For each element of X, compute the probability density function
     (PDF) at X of a discrete uniform distribution which assumes the
     integer values 1-N with equal probability.

     Warning: The underlying implementation uses the double class and
     will only be accurate for N <= `bitmax' (2^53 - 1 on IEEE-754
     compatible systems).

 -- Function File:  unidcdf (X, N)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of a discrete uniform distribution which
     assumes the integer values 1-N with equal probability.

 -- Function File:  unidinv (X, N)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the discrete uniform distribution which assumes the
     integer values 1-N with equal probability.

 -- Function File:  unifpdf (X)
 -- Function File:  unifpdf (X, A, B)
     For each element of X, compute the probability density function
     (PDF) at X of the uniform distribution on the interval [A, B].

     Default values are A = 0, B = 1.

 -- Function File:  unifcdf (X)
 -- Function File:  unifcdf (X, A, B)
     For each element of X, compute the cumulative distribution
     function (CDF) at X of the uniform distribution on the interval
     [A, B].

     Default values are A = 0, B = 1.

 -- Function File:  unifinv (X)
 -- Function File:  unifinv (X, A, B)
     For each element of X, compute the quantile (the inverse of the
     CDF) at X of the uniform distribution on the interval [A, B].

     Default values are A = 0, B = 1.

 -- Function File:  wblpdf (X)
 -- Function File:  wblpdf (X, SCALE)
 -- Function File:  wblpdf (X, SCALE, SHAPE)
     Compute the probability density function (PDF) at X of the Weibull
     distribution with scale parameter SCALE and shape parameter SHAPE
     which is given by

             shape * scale^(-shape) * x^(shape-1) * exp (-(x/scale)^shape)

     for X >= 0.

     Default values are SCALE = 1, SHAPE = 1.

 -- Function File:  wblcdf (X)
 -- Function File:  wblcdf (X, SCALE)
 -- Function File:  wblcdf (X, SCALE, SHAPE)
     Compute the cumulative distribution function (CDF) at X of the
     Weibull distribution with scale parameter SCALE and shape
     parameter SHAPE, which is

          1 - exp (-(x/scale)^shape)

     for X >= 0.

     Default values are SCALE = 1, SHAPE = 1.

 -- Function File:  wblinv (X)
 -- Function File:  wblinv (X, SCALE)
 -- Function File:  wblinv (X, SCALE, SHAPE)
     Compute the quantile (the inverse of the CDF) at X of the Weibull
     distribution with scale parameter SCALE and shape parameter SHAPE.

     Default values are SCALE = 1, SHAPE = 1.

