I went with a variant of the suggested solution; keeping all ICommandHandlers and IQueryHandlers potentially aynchrono
Async and await don't mix perfectly with traditional OOP. I have a blog series on the subject; you may find the post on async interfaces helpful in particular (though I don't cover anything you haven't already discovered).
The design problems around async are extremely similar to the ones around IDisposable; it's a breaking change to add IDisposable to an interface, so you need to know whether any possible implementation may ever be disposable (an implementation detail). A parallel problem exists with async; you need to know whether any possible implementation may ever be asynchronous (an implementation detail).
For these reasons, I view Task-returning methods on an interface as "possibly asynchronous" methods, just like an interface inheriting from IDisposable means it "possibly owns resources."
The best approach I know of is:
Task/Task).Task.FromResult(...) for synchronous implementations. This is more proper than an async without an await.This approach is almost exactly what you're already doing. A more ideal solution may exist for a purely functional language, but I don't see one for C#.