Neural network for adding two integer numbers

问题

I am beginner in NeuralNets . I want to create a neural network which can add two integer numbers. I have designed it as follows : question I have really low accuracy of 0.002% . what can i do to increase it? 1) For creating data:

import numpy as np
import random 
a=[]
b=[]
c=[]

for i in range(1, 1001):
    a.append(random.randint(1,999))
    b.append(random.randint(1,999))
    c.append(a[i-1] + b[i-1])

X = np.array([a,b]).transpose()
y = np.array(c).transpose().reshape(-1, 1)

2) scaling my data :

from sklearn.preprocessing import MinMaxScaler
minmax = MinMaxScaler()
minmax2 = MinMaxScaler()
X = minmax.fit_transform(X)
y = minmax2.fit_transform(y)

3) The network :


from keras import Sequential
from keras.layers import Dense
from keras.optimizers import SGD

clfa = Sequential()
clfa.add(Dense(input_dim=2, output_dim=2, activation='sigmoid', kernel_initializer='he_uniform'))
clfa.add(Dense(output_dim=2, activation='sigmoid', kernel_initializer='uniform'))
clfa.add(Dense(output_dim=2, activation='sigmoid', kernel_initializer='uniform'))
clfa.add(Dense(output_dim=2, activation='sigmoid', kernel_initializer='uniform'))
clfa.add(Dense(output_dim=1, activation='relu'))

opt = SGD(lr=0.01)
clfa.compile(opt, loss='mean_squared_error', metrics=['acc'])
clfa.fit(X, y, epochs=140)

outputs :

Epoch 133/140
1000/1000 [==============================] - 0s 39us/step - loss: 0.0012 - acc: 0.0020
Epoch 134/140
1000/1000 [==============================] - 0s 40us/step - loss: 0.0012 - acc: 0.0020   
Epoch 135/140
1000/1000 [==============================] - 0s 41us/step - loss: 0.0012 - acc: 0.0020
Epoch 136/140
1000/1000 [==============================] - 0s 40us/step - loss: 0.0012 - acc: 0.0020
Epoch 137/140
1000/1000 [==============================] - 0s 41us/step - loss: 0.0012 - acc: 0.0020
Epoch 138/140
1000/1000 [==============================] - 0s 42us/step - loss: 0.0012 - acc: 0.0020   
Epoch 139/140
1000/1000 [==============================] - 0s 40us/step - loss: 0.0012 - acc: 0.0020   
Epoch 140/140
1000/1000 [==============================] - 0s 42us/step - loss: 0.0012 - acc: 0.0020

That is my code with console outputs..

I have tried every different combinations of optimizers, losses, and activations, plus this data fits perfectly a Linear Regression.

回答1:

Two mistakes, several issues.

The mistakes:

This is a regression problem, so the activation of the last layer should be linear, not relu (leaving it without specifying anything will work, since linear is the default activation in a Keras layer).
Accuracy is meaningless in regression; remove metrics=['acc'] from your model compilation - you should judge the performance of your model only with your loss.

The issues:

We don't use sigmoid activations for the intermediate layers; change all of them to relu.
Remove the kernel_initializer argument, thus leaving the default glorot_uniform, which is the recommended one.
A number of Dense layers each one only with two nodes is not a good idea; try reducing the number of layers and increasing the number of nodes. See here for a simple example network for the iris data.

回答2:

You are trying to fit a linear function, but internally use sigmoid nodes, which map values to a range (0,1). Sigmoid is very useful for classification, but not really for regression if the values are outside (0,1). It could MAYBE work if you restricted your random number to floating point in the interval [0,1]. OR input into your nodes all the bits seperately, and have it learn an adder.

来源：https://stackoverflow.com/questions/61497523/neural-network-for-adding-two-integer-numbers

标签

python

machine-learning

keras

neural-network