I am building a class library to interact with an API. I need to call the API and process the XML response. I can see the benefits of using HttpClient for Async
If you're building a class library, then perhaps the users of your library would like to use your library asynchronously. I think that's the biggest reason right there.
You also don't know how your library is going to be used. Perhaps the users will be processing lots and lots of requests, and doing so asynchronously will help it perform faster and more efficient.
If you can do so simply, try not to put the burden on the users of your library trying to make the flow asynchronous when you can take care of it for them.
The only reason I wouldn't use the async version is if I were trying to support an older version of .NET that does not already have built in async support.