Singular Value Decomposition

Overview


Any matrix {% A %} has the following decomposition

{% A = U \Sigma V^* %}
where {% A %} is an {% m %} by {% n %} matrix.

  • {% U %} is a unitary matrix of dim {% m %}. The columns of {% U %} are eigenvectors of {% AA^* %}.
  • {% V %} is a unitary matrix of dim {% n %}. The columns of {% V %} are eigenvectors of {% A^*A %}.
  • {% \Sigma %} is a {% m %} by {% n %} diagonal matrix with non negative entries. The diagonal entries are the non negative square roots of the eigenvalues of {% A^*A %}

* is the conjugate transpose. When dealing with real matrices, * becomes the transpose. In such a case, instead of being unitary matrices, {% U %} and {% V %} are orthogonal.

SVD Observations


The following observations are useful in some computations.
{% A A^* = U \Sigma V^* V \Sigma^* U^* %}
because {% V %} is unitary, then {% V^* V = I %}
{% A A^* = U \Sigma^2 U^* %}
conversely
{% A^* A = U \Sigma V^* V \Sigma^* U^* = V \Sigma^2 V^* %}

Vector Representation


{% A = \sum \sigma_k \vec{u}_k \vec{v}_k^T %}
where {% \vec{u}+k %} is the {% k^{th} %} row of {% U %} and {% \vec{v}_k %} is the {% k^{th} %} row of {% V %}. {% \sigma_i %} is the {% i^{th} %} diagonal value of {% \Sigma %}.

Implementation


The following code utilizes the linear algebra library to compute the svd decomposition.


let la = await import('/lib/liniear-albgebra/v1.0.0/linear-algebra.mjs');
let ans = la.SVD(matrix1);
					
Try it!

Topics


  • Truncated SVD