Fast forward to 2020, and using aws-okta as our 2fa, the following command, while slow as hell to iterate through all of the objects and folders in this particular bucket (+270,000) worked fine.
aws-okta exec dev -- aws s3 ls my-cool-bucket --recursive | grep needle-in-haystax.txt