Overview
Auto encoding is the process of using a neural network to encode a vector into another vector of a smaller size which still retains most of the information of the original vector. That is, the original vector can be mostly recovered from the compressed version.
The mathematics of the nature and limits of compression is given in Information Theory
Architecture
Given a set of vectors, {% \vec{v}_i %}, of lenth {% n %}, the autoencoder is a neural network that is designed to encode each vector into a vector of length {% m %} where {% m < n %}.
The network is designed to be a neural network with {% n %} inputs. It has one hidden layer with {% m %} nodes, and an output layer with {% n %} outputs. Then the network is trained with a loss function that penalizes the network when the output does not match the input.
After the network has been trained, the compressed (encoded) vector is the value of the m hidden layer nodes when the network is fed the given input. This is a vector of length {% m %}. To recover the original vector, this encoded vector is fed to the output layer of the network.