neural network cost doesn't decrease

问题

When i run my neural network the cost always stays around its initial value. sometime it increases and sometimes it decreases but never by much.

I tried some different seeds but this didn't work. I also tried with different input but this also didn't work.

I looked over the math for the training but I couldn't find anything wrong with it.

class NeuralNet():
    def __init__(self):
        np.random.seed(1)

        self.w0 = 2*np.random.random((3, 4))-1
        self.w1 = 2*np.random.random((4, 1))-1

    def sigmoid(self, x):
        return 1/(1+np.exp(-x))

    def sigmoid_d(self, x):
        x = self.sigmoid(x)
        return x * (1-x)

    def train(self, training_length, x, y, print_cost_cycle=1000):
        for i in range(training_length):
            l0 = x
            l1 = self.sigmoid(np.dot(l0, self.w0))
            l2 = self.sigmoid(np.dot(l1, self.w1))

            l2_error = (l2 - y)**2
            l2_d = 2*(y - l2) * self.sigmoid_d(l2)
            l1_cost = np.dot(l2_d, self.w1.T)
            l1_d = l1_cost * self.sigmoid_d(l1)

            self.w0 += np.dot(l0.T, l1_d)
            self.w1 += np.dot(l1.T, l2_d)

            if i % print_cost_cycle == 0:
                print(f"cost = {np.mean(l2_error)}")


x_train = np.array([
    [0, 0, 1],
    [0, 1, 1],
    [1, 0, 1],
    [1, 1, 1]
])
y_train = np.array([
    [0],
    [1],
    [1],
    [0]
])

neural_net = NeuralNet()
neural_net.train(10000, x_train, y_train)

the output:

cost = 0.2475932677010128
cost = 0.24844153430705612
cost = 0.21300873346681945
cost = 0.21497568086911634
cost = 0.22868625594402198
cost = 0.23076981080756687
cost = 0.23242035817817885
cost = 0.23560061594143974
cost = 0.23578477135747
cost = 0.2356102896524483

来源：https://stackoverflow.com/questions/61232327/neural-network-cost-doesnt-decrease

标签

python

neural-network