Web logs

Bias refers to the deviation of the predicted values from the correct value. The error occurs when you make wrong assumptions about data. In other words, it is the error that is created when you represent a real-life complex problem using a simpler model while it might be making them easier to understand. For instance, when you build a linear model to solve for a non-linear problem. It results in under fitting and makes them less flexible. Parametric algos like Linear Regression can produce high bias while non-parametric algos like Decision Trees make good assumptions about the training data and target function and hence do not have high bias. Variance refers to the change that occurs when the model is applied on a different training data. It occurs when the model captures not just the underlying pattern but noise as well. It results in overfitting. In other words, it is memorising the data. It is often observed in Decision Trees. When the observations are limited but the...

In his book 'An Enquiry concerning Human Understanding'  David Hume posited that inherently fallible evidence is insufficient proof against natural laws - eyewitness testimony can’t prove a miracle. Bayes, a Presbytarian minister, motivated to rebut him was interested in understanding how much evidence would we need to be persuaded that something is a probability regardless of how improbable it is. He developed an equation that focuses on updating our beliefs with new evidence. His work, An Essay towards solving a Problem in the Doctrine of Chances, was reviewed by Richard Price after Bayes' death who believed that Bayes’ Theorem helped prove the existence of God. When we are presented with new information, we can use Bayes’ Theorem to refine our pre-existing belief.  It is fairly easy to determine the probability of the effect given a cause. Since Bayes’ Theorem works in the reverse direction, many find it complicated.

The green line has a negative Beta and the black one has a positive beta for logistic regression.