Overview
Logistic Regression can be seen to be a fairly simple example of a neural network.
The forecast produced by the logistic regression for a data point {% \vec{x} %} is given by
{% \hat{y} = \sigma(b + \vec{w}^T \vec{x}) %}
This can be seen as a 1-layer neural network with the sigmoid activation function and a single output.
Optimization
The typical way that a logistic regression determines its weights is through the use of Maximum Likelihood using Newtons Method . (see Fitting a Logistic Regression)
Mulinomial
The logistic regression can be extended from identifying a single category (or probability) to predicting probabilities for a set of category labels. That is, the output becomes a vector instead of a single variable. (see multi-nomial regression)