I tried a bunch of different answers but all had their shortcomings or didn't work with the material FAB.
When I realized this is not what it is meant for I simply replaced it with an ImageView. This works great and looks identical.
fab_gradient.xml:
-
-