The cause of poor performance in machine learning model is either overfitting or underfitting the data.
Gaining a proper understanding of underfitting and overfitting would help us to develop the most accurate model.
So let’s start discussing about underfit ,overfit of model and see how they make impacted.
What is Underfitting of model?
- Underfitting happens , when a model unable to capture the underlying pattern of the data.
- Underfitted Models usually have high bias and low variance.
- It happens when we have very less amount of data to build an accurate model or when we try to build a linear model with a nonlinear data.
- It performs bad with the training data and test data.
What is Overfitting of model?
- Overfitting happens when our model captures the noise along with the underlying pattern in data.
- It happens when we train our model a lot over noisy dataset.
- Overfitted models have low bias and high variance.
- It performs very good with the training dataset but don’t work well with the test dataset.
Example of Underfit and Overfit:
By looking at the graph on the middle, we can predict that the line does not cover all the points shown in the graph. Such model tend to cause underfitting of data .It also called High Bias.
Where as the graph on left side, shows the predicted line covers all the points. Such model tend to cause overfitting of data. It also called High variance.
Where as the graph on right, shows pretty good predicted line. It covers majority of the point in graph and also maintains the balance between bias and variance.
Bias Variance TradeOff:
If our model is too simple and has very few parameters then it may have high bias and low variance and we call it underfitted model. Model does not get trained properly and won’t be used in production environment.
On the other hand if our model has large number of parameters then it’s going to have high variance and low bias and called it overfitted model.Model works very well for the training dataset but failed to get correct output on test dataset.
So we need to find the right/good balance without overfitting and underfitting the data.
How to detect model underfit?
If model accuracy is not well for the training and test dataset, model is underfit.
Techniques to prevent model underfitting:
- More data provided to train the model.
- Parameters change in the training the model, increase the number of iteration to give more chances to get trained.
- Train the linear model against non linear data.
- Modify the number of feature while training the model.
How to detect model overfit?
If model accuracy is very good for the training dataset but not performing well on the test data, model is overfit.
Techniques to prevent model overfitting:
- Train with more data.
- Remove features.
- Early stopping.