I ran in to this same issue today. Roll back to Django 3.0.x (I went back to 3.0.6) and then run your migrations.
I realize that's not a fix, but at least it's a work around to get you going. Once you migrate, you should be able to update back to 3.1 and it will run fine.