Overview
A vectorized matrix takes a matrix, such as
{%
\begin{bmatrix}
a & b & c \\
d & e & f\\
g & h & i \\
\end{bmatrix}
%}
and returns a vector, such as
{%
\begin{bmatrix}
a \\
d \\
g \\
b \\
e \\
h \\
c \\
f \\
i \\
\end{bmatrix}
%}
Vectorized matrices can make some computations easier. For example, in a neural network, it is necessary to calculate
the
derivative
of
{% \frac{M \vec{x}}{d M} %}
with respect to the Matrix {% M %}. In order to keep the answer as a matrix, and not a tensor, the matrix is vectorized first.
(see below)
Example Vectorized Gradient
This example demonstrates computing the vectorized gradient of the following equation.
{% \frac{M \vec{x}}{d M} %}
A sample {% 2 \times 2 %} matrix is provided
{%
M \vec{x} =
\begin{bmatrix}
a & b \\
c & d \\
\end{bmatrix}
\begin{bmatrix}
v_1 \\
v_2 \\
\end{bmatrix} =
\begin{bmatrix}
z_1 \\
z_2 \\
\end{bmatrix}
=
\begin{bmatrix}
a v_1 + b v_2 \\
c v_1 + d v_2 \\
\end{bmatrix}
%}
The vectorized gradient can then be seen to be
{% \frac{M \vec{x}}{d \vec{x}} =
\begin{bmatrix}
\frac{\partial z_1}{\partial a} & \frac{\partial z_2}{\partial a}\\
\frac{\partial z_1}{\partial c} & \frac{\partial z_2}{\partial c}\\
\frac{\partial z_1}{\partial b} & \frac{\partial z_2}{\partial b}\\
\frac{\partial z_1}{\partial d} & \frac{\partial z_2}{\partial d}\\
\end{bmatrix}
=
\begin{bmatrix}
v_1 & 0 \\
0 & v_1\\
v_2 & 0\\
0 & v_2\\
\end{bmatrix}
%}