Neural network sine approximation

Question

After spending days failing to use neural network for Q learning, I decided to go back to the basics and do a simple function approximation to see if everything was working

Answer 1

I managed to get a good approximation by changing the architecture and the training as in the following code. It's a bit of an overkill but at least I know where the problem was coming from.

from keras.models import Sequential
from keras.layers import Dense
import matplotlib.pyplot as plt
import random
import numpy
from sklearn.preprocessing import MinMaxScaler
from sklearn.ensemble import ExtraTreesRegressor
from keras import optimizers

regressor = Sequential()
regressor.add(Dense(units=500, activation='sigmoid', kernel_initializer='uniform', input_dim=1))
regressor.add(Dense(units=500, activation='sigmoid', kernel_initializer='uniform'))
regressor.add(Dense(units=1, activation='sigmoid'))
regressor.compile(loss='mean_squared_error', optimizer='adam')
#regressor = ExtraTreesRegressor()

N = 5000

X = numpy.empty((N,))
Y = numpy.empty((N,))

for i in range(N):
    X[i] = random.uniform(-10, 10)

X = numpy.sort(X).reshape(-1, 1)

for i in range(N):
    Y[i] = numpy.sin(X[i])

Y = Y.reshape(-1, 1)

X_scaler = MinMaxScaler()
Y_scaler = MinMaxScaler()
X = X_scaler.fit_transform(X)
Y = Y_scaler.fit_transform(Y)

regressor.fit(X, Y, epochs=50, verbose=1, batch_size=2)
#regressor.fit(X, Y.reshape(5000,))

x = numpy.mgrid[-10:10:100*1j]
x = x.reshape(-1, 1)
y = numpy.mgrid[-10:10:100*1j]
y = y.reshape(-1, 1)
x = X_scaler.fit_transform(x)
for i in range(len(x)):
    y[i] = regressor.predict(numpy.array([x[i]]))


plt.figure()
plt.plot(X_scaler.inverse_transform(x), Y_scaler.inverse_transform(y))
plt.plot(X_scaler.inverse_transform(X), Y_scaler.inverse_transform(Y))

However I'm still baffled that I found papers saying that they were using only two hidden layers of five neurons to approximate the Q function of the mountain car problem and training their network for only a few minutes and get good results. I will try changing my batch size in my original problem to see what results I can get but I'm not very optimistic

Answer 2

With these changes:

• Activations to relu
• Remove kernel_initializer (i.e. leave the default 'glorot_uniform')
• Adam optimizer
• 100 epochs

i.e.

regressor = Sequential()
regressor.add(Dense(units=20, activation='relu', input_dim=1)) 
regressor.add(Dense(units=20, activation='relu')) 
regressor.add(Dense(units=20, activation='relu')) 
regressor.add(Dense(units=1))
regressor.compile(loss='mean_squared_error', optimizer='adam')

regressor.fit(X, Y, epochs=100, verbose=1, batch_size=32)

and the rest of your code unchanged, here is the result:

Tinker, again and again...

Answer 3

A more concise version of your code that works:

def data_gen():
    while True:
        x = (np.random.random([1024])-0.5) * 10 
        y = np.sin(x)
        yield (x,y)

regressor = Sequential()
regressor.add(Dense(units=20, activation='tanh', input_dim=1))
regressor.add(Dense(units=20, activation='tanh'))
regressor.add(Dense(units=20, activation='tanh'))
regressor.add(Dense(units=1, activation='linear'))
regressor.compile(loss='mse', optimizer='adam')

regressor.fit_generator(data_gen(), epochs=3, steps_per_epoch=128)

x = (np.random.random([1024])-0.5)*10
x = np.sort(x)
y = np.sin(x)

plt.plot(x, y)
plt.plot(x, regressor.predict(x))
plt.show()

Changes made: replacing low layer activations with hyperbolic tangents, replacing the static dataset with a random generator, replacing sgd with adam. That said, there still are problems with other parts of your code that I haven't been able to locate yet (most likely your scaler and random process).