Multiple gateways but traffic is dropped on non-primary interface (ubuntu 18.04)

Question

I have a multi-vlan setup (single phys nic) and a gateway on each vlan. I need multiple gateways in my setup. I thought I set things up right, but something is still broken.