问题
So I am currently trying to implement my first NN with a genetic algorithm for training and a sigmoid activation function. It's all good but I'm not quite sure in what ranges the weights must be. I've searched some about the question but with no luck. How does one choose the ranges of the weights in a NN? What does it depend on?
回答1:
The weights can be seen as an intrinsic property of the problem you're trying to solve using the GA/NN approach; there's no general best value fo these, so you're best off studying different weight spans (w.r.t. training sets) with other parameters fixed.
E.g., study different settings for parameter weightSpan in
weights \in [-weightSpan/2, weightSpan/2],
and let your initial chromosomes describe weights with randomized values in this range. Your squashing function (sigmoid) is used to grade the NN response to the range [0, 1].
Finding an appropriate weight span is, much like setting a value of number of hidden layer, a process if problem-specific testing. ("There is no free lunch").
Edit:
I thought I'd add that the easiest way to study different weight spans is probably to set a fixed weight span, say [-1, 1], and study the squashing constant in your squashing function (sigmoid). I.e., study different (non-negative) values of constant c in your sigmoid
σ(s) = 1 / (1 + e^(-c*s))
来源：https://stackoverflow.com/questions/34120076/calculating-weights-in-a-nn