Fastest way to get Google Storage bucket size?

Question

I\'m currently doing this, but it\'s VERY slow since I have several terabytes of data in the bucket:

gsutil du -sh gs://my-bucket-1/

And th

Answer 1

If the daily storage log you get from enabling bucket logging (per Brandon's suggestion) won't work for you, one thing you could do to speed things up is to shard the du request. For example, you could do something like:

gsutil du -s gs://my-bucket-1/a* > a.size &
gsutil du -s gs://my-bucket-1/b* > b.size &
...
gsutil du -s gs://my-bucket-1/z* > z.size &
wait
awk '{sum+=$1} END {print sum}' *.size

(assuming your subfolders are named starting with letters of the English alphabet; if not; you'd need to adjust how you ran the above commands).