Why is a CoreML image segmentation model with significantly reduced input image dimensions no faster than the original model?

Question

I am building an iPhone app with a vision processing component that could use image segmantation, especially if it can run at 30fps. I am starting the exploration process w