why gradient descent when we can solve linear regression analytically

Question

what is the benefit of using Gradient Descent in the linear regression space? looks like the we can solve the problem (finding theta0-n that minimum the cost func) with analytic

Answer 1

You should provide more details about yout problem - what exactly are you asking about - are we talking about linear regression in one or many dimensions? Simple or generalized ones?

In general, why do people use the GD?

• it is easy to implement
• it is very generic optimization technique - even if you change your model to the more general one, you can stil use it

So what about analytical solutions? Well, we do use them, your claim is simply false here (if we are talking in general), for example the OLS method is a closed form, analytical solution, which is widely used. If you can use the analytical solution, it is affordable computationaly (as sometimes GD is simply cheapier or faster) then you can, and even should - use it.

Neverlethles this is always a matter of some pros and cons - analytical solutions are strongly connected to the model, so implementing them can be inefficient if you plan to generalize/change your models in the future. They are sometimes less efficient then their numerical approximations, and sometimes there are simply harder to implement. If none of above is true - you should use the analytical solution, and people do it, really.

To sum up, you rather use GD over analytical solution if:

• you are considering changes in the model, generalizations, adding some more complex terms/regularization/modifications
• you need a generic method because you do not know much about the future of the code and the model (you are only one of the developers)
• analytical solution is more expensive computationaly, and you need efficiency
• analytical solution requires more memory, which you do not have
• analytical solution is hard to implement and you need easy, simple code