I have a interface which exposes some async methods. More specifically it has methods defined which return either Task or Task
Michael Liu answered well your question about how you can avoid the warning: by returning Task.FromResult.
I'm going to answer the "Should I worry about the warning" part of your question.
The answer is Yes!
The reason for this is that the warning frequently results when you call a method that returns Task inside of an async method without the await operator. I just fixed a concurrency bug that happened because I invoked an operation in Entity Framework without awaiting the previous operation.
If you can meticulously write your code to avoid compiler warnings, then when there is a warning, it will stand out like a sore thumb. I could have avoided several hours of debugging.