I am new to matplotlib, and I don\'t know what the name of the title bar arrowed in following image is.
Title bar
I googled and dug from matplotlib official docu
What you refer to as "title bar", is actually part of the notebook backend - essentially the code that translates what you write in python to something that can be visualised as a figure. The notebook backend is made to be interactive within Jupyter notebooks, and as such has that bar, plus buttons you can use to pan, zoom, etc. When you save the figure via plt.savefig() or by clicking the save button, the bar will not appear.
If you want to get rid of the interactivity all together, try using the %matplotlib inline magic instead - that will select the inline backend, and remove the interactivity.