I want to create an automata type with a type like this:
newtype Auto i o = Auto {runAuto :: i -> (o, Auto i o)}
I know this is the type
That type is a Mealy machine. See https://hackage.haskell.org/package/machines-0.5.1/docs/Data-Machine-Mealy.html for a bunch of instances - but note that the Monad instance is always going to be slow, because it's required to diagonalize by the monad laws.
It sounds like what you really want is the auto package anyway.