plotting a 2D matrix in python, code and most useful visualization

Question

I have a very large matrix(10x55678) in \"numpy\" matrix format. the rows of this matrix correspond to some \"topics\" and the columns correspond to words(unique words from

Answer 1

The key thing to consider is whether you have important structure along both dimensions in the matrix. If you do then it's worth trying a colored matrix plot (e.g., imshow), but if your ten topics are basically independent, you're probably better off doing ten individual line or histogram plots. Both plots have advantages and disadvantages.

In particular, in full matrix plots, the z-axis color values are not very precise or quantitative, so its difficult to see, for example, small ripples on a trend, or quantitative assessments of rates of change, etc, so there's a significant cost to these. And they are also more difficult to pan and zoom since one can get lost and therefore not examine the entire plot, whereas panning along a 1D plot is trivial.

Also, of course, as others have mentioned, 50K points is too many to actually visualize, so you'll need to sort them, or something, to reduce the number of values that you'll actually need to visually assess.

In practice though, finding a good visualizing technique for a given data set is not always trivial, and for large and complex data sets, people try everything that has a chance of being helpful, and then choose what actually helps.