How to visualize a neural network

Question

I want to draw a dynamic picture for a neural network to watch the weights changed and the activation of neurons during learning. How could I simulate the process in Python?

Answer 1

This solution involves both Python and LaTeX. Might be an overkill for your case, but the results are really aesthetic and suit more complicated, modern architectures (deep learning etc.), so I guess it is worth mentioning here. You first need to define your network in Python, such as this one:

import sys
sys.path.append('../')
from pycore.tikzeng import *

# defined your arch
arch = [
    to_head( '..' ),
    to_cor(),
    to_begin(),
    to_Conv("conv1", 512, 64, offset="(0,0,0)", to="(0,0,0)", height=64, depth=64, width=2 ),
    to_Pool("pool1", offset="(0,0,0)", to="(conv1-east)"),
    to_Conv("conv2", 128, 64, offset="(1,0,0)", to="(pool1-east)", height=32, depth=32, width=2 ),
    to_connection( "pool1", "conv2"), 
    to_Pool("pool2", offset="(0,0,0)", to="(conv2-east)", height=28, depth=28, width=1),
    to_SoftMax("soft1", 10 ,"(3,0,0)", "(pool1-east)", caption="SOFT"  ),
    to_connection("pool2", "soft1"),    
    to_end()
    ]

def main():
    namefile = str(sys.argv[0]).split('.')[0]
    to_generate(arch, namefile + '.tex' )

if __name__ == '__main__':
    main()

After that, you generate a TikZ image...

bash ../tikzmake.sh my_arch

...which will yield you a PDF with your network:

Examples are provided in the repo, below one of the them. I've tested it on OS X, should work on Linux as well. Not sure how about Windows. Naturally, you'll need a LaTeX distribution installed.