I need to be able to rotate a video on screen, so I created a custom TextureView which provides a convenience layer over a MediaPlayer similar to how the current implementat
Yes, it is expected with TextureView. TextureView causes the video to go through normal view-compositing for rendering, unlike SurfaceView which is composited directly in the GPU (the decoding pipeline is rendering directly to the area of the screen where you place the SurfaceView). While the TextureView rendering is hardware-accelerated, it still is going through more steps for the additional flexibility, and there is a definite performance hit. Additionally, any code running on the UI-thread may affect TextureView unlike SurfaceView.
Additional information: