How to find multivariable regression equation in javascript

Question

I have searched stack overflow and have not found any question that really is the same as mine because none really have more than one independent variable. Basically I have an array of datapoints and I want to be able to find a regression equation for those data points. The code I have so far looks like this: (w,x,z are the independent variables and y is the dependent variable)

var dataPoints = [{
 "w" : 1, "x" : 2, "z" : 1, "y" : 7
}, {
 "w" : 2, "x" : 1, "z" : 4, "y" : 5
}, {
 "w" : 1, "x" : 5, "z" : 3, "y" : 2
}, {
 "w" : 4, "x" : 3, "z" : 5, "y" : 15
}];

I would like a function that would return a formula object like this:

var regressionEquation = [{
 "var" : "w", "power" : 1, "coeff" : "1.5"
}, {
 "var" : "x", "power" : 1, "coeff" : "2"
}, {
 "var" : "z", "power" : 1, "coeff" : "1"
}];

Is there a way to come up with a regression equation like this without using a loop to step and plug in the values? Is there a way to come up with the regression equation for powers that are more than 1? Thanks in advance.

EDIT

Many people have suggested solving a system of equations made by plugging in the powers. The problem I have with this is when there is more than enough data points to solve for a system of equations. In the examples in the question, I have 3 variables in order to solve the system of equations that people are suggesting, I would need 3 datapoints but I have 4. This leads to a problem because there is more than one solution. There are 4 possible solutions because there are 4 ways to combine the 4 equations into different groups of 3. This would leave me with 4 answers with possibly none of them the best fit to all 4 points.

Answer 1

The problem as you've stated it is equivalent, under transformation, to a linear regression problem. You said in comments that you have fixed exponents k_1, k_2, and k_3. The transformation takes a tuple {w, x, z ,y} to the tuple {w^k_1, x^k_2, z^k_2, y} = {w', x', z' ,y}. Use linear regression on the primed variables to get your coefficients.

For example, if k_1 = 2, k_2 = 3, and k_3 = 1, then here's a single example of the transform:

{"w" : 4, "x" : 3, "z" : 5, "y" : 15} 
==> {"w*" : 16, "x*" : 27, "z*" : 5, "y" : 15}

This is just a special case of how you convert a polynomial regression problem into a linear regression one. In your case the polynomial forms you are considering are particularly simple.

Use any JavaScript library you like to solve the linear regression problem; there are a number of them.

Answer 2

I think if it is the case that there are four equations and only 3 variables (As you already determined the powers, plugin and make it a linear equation), the linear equation is over complete, and there does not exist an exact answer that will satisfy all four equations.

What you can do is to minimize the residual error and get a best approximation.

Assume you have coefficients a b and c for the w x and z,

define matrix

M=[w1,x1,z1;w2,x2,z2;w3,x3,z3;w4,x4,z4]. 

and define the vector

v=[a;b;c], 

define vector

r=[y1;y2;y3;y4]. 

Then the problem is

M*v=r solve v. 

1. If rank(M)>variable number, you have to minimize the residual error

||M*v-r||_2. 

Since this is convex, take derivative on it and make it zero:

M^T*M*v-M^T*r=0 => v=(M^T*M)\M^T*r. 

(M^T*M)\M^T is MP-inverse of M, if rank(M)>variable number, then (M^T*M) is inversible.

2. If the rank(M)<=variable number, you can get infinitely many exact solution to the equation.

M*v=r. 

Let singular value decomposition of M:

M=U*S*V^T, 

then

v=V*S^-1*U^T*r 

is one of the solutions.

V*S^-1*U^T is pseudo inverse of M.

If you use a linear algebra library, it is very easy to get closed form solution without iterating. http://sylvester.jcoglan.com/

Answer 3

I would suggest using least squares for getting a linear equation. Further, you could use non-linear least squares, given that you know in advance the function you want to fit.

(http://en.wikipedia.org/wiki/Least_squares)

There are several links for linear LS in javascript and you can probably adapt these to 3 dimensions (e.g. http://dracoblue.net/dev/linear-least-squares-in-javascript/159/ from a quick Google search). For the non-linear case it would need some more work though.