Overview
The bag of words model is a simple model that is used to train a computer to recognize the topic of a document. This can be used when compiling a document search algorithm that utilizes document topic in the search criteria.
Feature Extraction
The bag of words model converts a document into a vector that can be input into a neural network. First, the model identifies a set of words (a dictionary) and then arranges them into a vector. Next, it parses out the words in a document, counting the number of occurences of each word in the dictionary in the document. It then constructs a vector where each element indicates the number of occurences of the relevant word.
Neural Network Model
A simple neural network that can be used to classify a bag of words model is multinomial regression. This can be solved using a neural network library. In this case, a single layer neural network is created. The number of inputes equals the size of the chosen dictionary. The number of outputs equals the number of different topics that are being considered. Next, the softmax function is used as the activation function.
Lastly, the cross entropy loss function is used as the loss function to be minimized.