How to build a Neural Network for Reinforcement learning with look up table?

Question

I am self-learning Reinforcement Learning with Q-learning using online resources like blog posts, youtube videos and books and at this point, I have learned the underpinning