Linear Regression(With Gradient Descent ) Fully Explained| Machine Learning

Linear Regression (Gradient Descent Method)

If you don't have any idea of Gradient Descent Algorithm, please check our previous post, there I have explained Gradient Descent Algorithm very well explained in brief.

Now moving toward our current topic Linear Regression . In the previous post , we have just discussed the theory behind the Gradient Descent. Today we will learn Linear Regression where we will use Gradient Descent to minimize the cost function.

WHAT IS LINEAR REGRESSION:

Suppose you are given a equation: y=2x1+3x2+4x3+1 and you are said to find the value at any point (1,1,1) corresponds to x1, x2, x3 respectively.
You'll simply put the value of x1, x2, x3 into equation and tell me the answer :10,Right?
But What if you are given different set of (x1, x2, x3,y) and you are said to find the equation.
Here's what,Linear Regression Comes into picture.It helps us to find out or fit a Linear equation to datasets  given.

Above equation can be easily transformed to following form.If we suppose:

θ=[2,3,4]

X=[x1, x2, x3]

c=1

Above equation can be written as : Y=θ.X+C

Linear Regression help us to find the value of θ and c

For this post we will be limited to a single line equation, multidimensional linear equation can also be done in similar way.

X      Y

2      7

3    10

4    13

5    16

After Seeing above data set, we can clearly say ,there is a relation ,

Y=2*X +1

is there.But how will we get above equation using linear regression , will try to understand in this post.

•  Assume a line y_p=mx+c
• Cost Function : Our cost function is Measured Squared Error(MSE) i.e Average squared difference of observed value and predicted value.

Cost Function=Mean Squared Error(MSE) = (1/n)Σ(y - y_p)²

J(m,c) = (1/n)Σ(y-(mx+c))²

J(m,c) is our cost function now which depends on m and c, as z was depending upon x and y in post of gradient descent.

Now,
For Gradient Descent we need:

∂j/∂m=(-2/n)Σ(x*(y-(mx+c)))

∂j/∂c=(-2/n)Σ(y-(mx+c))

Assume,

learning_rate(η) =0.001

Iterations:

(m)new=(m)old - η*(∂j/∂m)

(c)new=(c)old - η*(∂j/∂c)

How many times you want to iterate? It depends upon you or you can fix a condition like cost<=0.000001

Try Below given Code:

Python Code to implement this

	import numpy as np
	import matplotlib.pyplot as plt
	def plot(x,y,m,b):
	plt.scatter(x,y)
	# datapoints
	y_pred=[i*m+b for i in x]
	plt.plot(x,y) #classifier line
	plt.show()
	def grad_desc(x,y):
	m_curr=b_curr=0 #starting value m_curr,b_curr
	iterations=1000 #Total iteration count
	learning_rate=0.001 #Assumed learning rate
	n=len(x) #length of x vector
	for i in range(iterations):
	y_pred = m_curr*x + b_curr #predicted value of y
	md=(-2/n)*sum(x*(y-y_pred)) #value of derivative of cost w.r.t m
	cost=sum((y-y_pred)**2)/n
	bd=(-2/n)*sum(y-y_pred) #value of derivative of cost w.r.t b
	m_curr=m_curr-learning_rate*md
	b_curr=b_curr-learning_rate*bd
	print('m {} b {} cost {}'.format(m_curr,b_curr,cost))
	plot(x,y,m_curr,b_curr)
	if __name__=="__main__":
	x=np.array([2,3,4,5,6,7,8,9,11,10,12])
	y=np.array([7,10,13,16,19,22,25,28,34,31,37])
	grad_desc(x,y)

view raw gradient_descent.py hosted with ❤ by GitHub