Is it possible to add buttons inside the JTable cell along with data? What I am trying to do is to create a table with columns which display data(number) from the database,
I think you need to create a custom cell renderer if you want to show anything else than text (or numbers) in the cell. The cell renderer's job is to paint whatever you need to show in the cell.
See Table Renderer documentation.
So in this case you could create a small JPane which contains the text field and the tiny + and - buttons - or a just a JSpinner component, if does what you need. A bit tricky, for sure, but should be possible.