I have been looking into setting up a login for a web app that lets clients view data hosted in S3 and found that AWS Cognito has a hosted web UI [link] that handles most of
That use case, putting Cognito in front of S3, using the hosted UI, is covered in this AWS blog: https://aws.amazon.com/blogs/networking-and-content-delivery/authorizationedge-using-cookies-protect-your-amazon-cloudfront-content-from-being-downloaded-by-unauthenticated-users/
The blog contains sample code you can steal from. And you can deploy the sample solution easily using the serverless application repository.
Disclaimer: I authored that blog. Hope it can be of use to you!