aws cli s3 bucket remove object with date condition

Question

how do i remove aws cli s3 bucket remove object with date condition recursively

i am using this command for listing

aws s3 ls --recursive s3://uat-files-transfer-storage/ | awk '$1 < "2018-02-01 11:13:29" {print $0}' | sort -n

its run perfectly but when i use this command with rm its delete all files

aws s3 rm --recursive s3://uat-files-transfer-storage/ | awk '$1 < "2018-02-01 11:13:29" {print $0}' | sort -n

any solution

Answer 1

You're on the right track. To understand what's going on, let's look at what your commands are doing step by step.

aws s3 ls --recursive s3://uat-files-transfer-storage/ | awk '$1 < "2018-02-01 11:13:29" {print $0}' | sort -n

This command lists all files in the bucket recursively, checks the output for a specific condition, and then sorts the resultant output lines.

aws s3 rm --recursive s3://uat-files-transfer-storage/ | awk '$1 < "2018-02-01 11:13:29" {print $0}' | sort -n

This command deletes all the files in your bucket recursively, checks the output for a specific condition, and then sorts the resultant output lines. So the second command deletes all your files first!

What you want to do is list all the files in your bucket, check that they meet a certain criteria, and then delete them. This should work:

aws s3 ls --recursive s3://uat-files-transfer-storage/ | awk '$1 < "2018-02-01 11:13:29" {print $4}' | xargs -n1 -t -I 'KEY' aws s3 rm s3://uat-files-transfer-storage/'KEY'

BE CAREFUL! This won't prompt you before deleting, and it's an easy way to clean out your whole bucket by mistake.