As you mentioned, matplotlib is not installed on the EMR cluster, therefore such error will occur:
However, it is actually available in the managed Jupyter notebook instance (the docker container). Using the %%local magic will allow you to run the cell locally: