\({\hat y=f(z)}\)

\(z=\sum_{i=1}^{n}w_ix_i+b\)

\(\hat y\) = Predicted output

\({f(z)}\) = Activation function

\({w}\) = Weight (It also means slope)

\({b}\) = Bias (It also means intercept)

We want to understand what exactly is bias(b) in the linear equation and why bias?

Lets take a linear equation (By the way, the linear equation is our model, more specifically linear model, which needs learning to properly fit in the data):

\(y=3x +2\)

Here the weight is 3 and bias is 2. Let us represent the linear equation (with and without bias) in 2D graph:

You can notice that both the lines have the same slope (3), but the line with bias is 2 units above the origin.

So with this we understand that without bias, the linear equation passes through origin. With bias, the line intersects the y-axis.

This choice for the linear equation to not let pass through the origin is the fundamental reason why bias is mandatory for a model to predict correctly.

Let us understand this with an example on why bias is required:

Consider a model which wants to predict marks obtained based on hours of study. Suppose our model without bias is :

\(y=3x\)

Then as we know, the model (linear equation) passes through origin, hence the relation between hours of study and marks obtained would be:

So we see that the model assumes that the student who studies 0 hours obtains 0 marks, which is completely wrong to begin with for a model. Now let us introduce a bias on the same model.

\(y=3x +2\)

So now we notice that with bias, even if a student studies 0 hours, model predicts a sensible output, in this case marks obtained is 2 for 0 hours of study.

Conclusion on bias: We can clearly see in the graph that bias controls the vertical position. It allows the line to move up or down without affecting the slope. Without bias, the linear model is forced to pass through the origin thus allowing the model to predict incorrect values. With bias, the model is mature enough to fit the model in real world data.

If you've made it this far, thank you so much for reading! 😊 If you found this article helpful, please consider bookmarking my blog. Your support would mean a lot and motivate me to keep sharing more! 🚀

Biological (Single Building Block) : Neuron
Artificial (Single Building Block) : Perceptron

By now you must be thinking what does a "Perceptron" do?

Perceptron's primary purpose is to learn patterns from input data and make predictions or classification decisions. It takes multiple inputs, assigns weights to them, adds a bias, and applies an activation function to produce an output.

Let us understand it in Mathematical terms, its all about school days Linear Equation:

\(y = mx + c\)

\(y = m_1x_1 + m_2x_2 + \cdots + m_nx_n + c\)

\(y = \sum_{i=1}^{n} m_i x_i + c\)

I have represented Linear Equation in three different format for the readers to understand the equation easily. Similarly below is the predicted output equation for Perceptron Model (Single Neuron).

\(z=\sum_{i=1}^{n}w_ix_i+b\)

\({\hat y=f(w_1x_1+w_2x_2+\cdots+w_nx_n+b)}\)

So by now you know mathematically how output is predicted via a Perceptron Model (Single Neuron)

Let us solve an example to further understand how output is predicted using Perceptron Model.

Example: Email Spam Detection

Suppose we want to classify whether an email is spam or not. We assume that the perceptron receives three inputs:

\(x_1\): Does the email contain the word "Money"?

\(x_2\): Does the email contain a suspicious link?

\(x_3\): Is the sender unknown?

Assume the perceptron has learned these weights:

\(w1= 0.5, w2= 0.8, w3= 0.3, b= -1\)

For an email where all the three inputs are true x1, x2, x3 = 1 (True):

\(z=(0.5\times1)+(0.8\times1)+(0.3\times1)-1\)

\(z=0.6\)

Applying the step activation function( 1 or 0 means prediction can most likely be true or false based on z value) :

\(\hat{y} = \begin{cases} 1, & \text{if } z \geq 0 \\ 0, & \text{if } z < 0 \end{cases}\)

Since z=0.6 hence the prediction is True (Spam most likely):

\({\hat y=1}\) (Spam)

Hence the perceptron predicted that the input email is a Spam.

After going through the mathematical example, you might be having two questions in your mind:

1. What exactly is a bias?
2. What is the meaning of assumption that perceptron has learned weights?

For now lets assume that Bias is important because it allows the perceptron to make predictions more flexibly, rather than depending only on the weighted inputs. I will cover these two questions in my next blog.

If you've made it this far, thank you so much for reading! 😊 If you found this article helpful, please consider bookmarking my blog. Your support would mean a lot and motivate me to keep sharing more! 🚀