Logistic Regression and Machine Learning

Overview

Logistic Regression can be seen to be a fairly simple example of a neural network.

The forecast produced by the logistic regression for a data point {% \vec{x} %} is given by

{% \hat{y} = \sigma(b + \vec{w}^T \vec{x}) %}

This can be seen as a 1-layer neural network with the sigmoid activation function and a single output.

Optimization

The typical way that a logistic regression determines its weights is through the use of Maximum Likelihood using Newtons Method . (see Fitting a Logistic Regression)

Mulinomial

The logistic regression can be extended from identifying a single category (or probability) to predicting probabilities for a set of category labels. That is, the output becomes a vector instead of a single variable. (see multi-nomial regression)