# 逻辑回归回顾

这篇分享，我仅从频率学学派派去推导逻辑回归。后期会加上从贝叶斯学派进行的逻辑回归推导
This post, I’m gonna introduce logistic regression from frequentist perspective, and I will add conduction of logistic regression from bayes perspective.

## Logistic Regression Summary

Logistic regression is the appropriate regression analysis to conduct when the dependent variable is dichotomous (binary). Like all regression analyses, the logistic regression is a predictive analysis. Logistic regression is used to describe data and to explain the relationship between one dependent binary variable and one or more nominal, ordinal, interval or ratio-level independent variables.

## Mathmatical function of LR:

### Base function( hypothesis fucntion of linear regression )

### Sigmoid Function of LR:

Sigmoid function is implemented to compress predict value into {0,1}, which can help us to do classification.

### Hypothesis function of LR:

And different from linear regression, the graph of this function is not a bowl-shaped funciton . Therefore, it would be hard to get a global minimum. So loss function of logistic regression is different from linear regression.

### Cost function of LR:

If we use least square as our loss function, the cost function would be a non-convex function that we can not get global minimum.

In order to overcome this issue, we use logistic loss as our loss function and cross-entry as our cost function. Therefore, it will penalize learning algorithm by a large cost if y is not equal to $h_\theta(x)$

### Regularization

Same as regularization in linear regression, here we always use L2-norm as our penality item.

## Overfitting

There are several options to deal with overfitting:

- Reduce number of features
- Regularization
- Enlarger the data set