Orthogonalization

In linear algebra, orthogonalization is the process of finding a set of orthogonal vectors that span a particular subspace. Formally, starting with a linearly independent set of vectors {v₁, ... , v_k} in an inner product space (most commonly the Euclidean space Rⁿ), orthogonalization results in a set of orthogonal vectors {u₁, ... , u_k} that generate the same subspace as the vectors v₁, ... , v_k. Every vector in the new set is orthogonal to every other vector in the new set; and the new set and the old set have the same linear span.

In addition, if we want the resulting vectors to all be unit vectors, then we normalize each vector and the procedure is called orthonormalization.

Orthogonalization is also possible with respect to any symmetric bilinear form (not necessarily an inner product, not necessarily over real numbers), but standard algorithms may encounter division by zero in this more general setting.

Orthogonalization algorithms

Methods for performing orthogonalization include:

Gram–Schmidt process, which uses projection
Householder transformation, which uses reflection
Givens rotation
Symmetric orthogonalization, which uses the Singular value decomposition

When performing orthogonalization on a computer, the Householder transformation is usually preferred over the Gram–Schmidt process since it is more numerically stable, i.e. rounding errors tend to have less serious effects.

On the other hand, the Gram–Schmidt process produces the jth orthogonalized vector after the jth iteration, while orthogonalization using Householder reflections produces all the vectors only at the end. This makes only the Gram–Schmidt process applicable for iterative methods like the Arnoldi iteration.

The Givens rotation is more easily parallelized than Householder transformations.

Symmetric orthogonalization was formulated by Per-Olov Löwdin.^[1]

Local orthogonalization

To compensate for the loss of useful signal in traditional noise attenuation approaches because of incorrect parameter selection or inadequacy of denoising assumptions, a weighting operator can be applied on the initially denoised section for the retrieval of useful signal from the initial noise section. The new denoising process is referred to as the local orthogonalization of signal and noise.^[2] It has a wide range of applications in many signals processing and seismic exploration fields.

References

^ Löwdin, Per-Olov (1970). "On the nonorthogonality problem". Advances in quantum chemistry. Vol. 5. Elsevier. pp. 185–199. doi:10.1016/S0065-3276(08)60339-1. ISBN 9780120348053.
^ Chen, Yangkang; Fomel, Sergey (2015). "Random noise attenuation using local signal-and-noise orthogonalization". Geophysics. 80 (6): WD1–WD9. Bibcode:2015Geop...80D...1C. doi:10.1190/GEO2014-0227.1.

[1] Löwdin, Per-Olov (1970). "On the nonorthogonality problem". Advances in quantum chemistry. Vol. 5. Elsevier. pp. 185–199. doi:10.1016/S0065-3276(08)60339-1. ISBN 9780120348053.

[ortho-2] Chen, Yangkang; Fomel, Sergey (2015). "Random noise attenuation using local signal-and-noise orthogonalization". Geophysics. 80 (6): WD1–WD9. Bibcode:2015Geop...80D...1C. doi:10.1190/GEO2014-0227.1.

[1]

[2]