Listing all public links for all objects in a bucket using gsutil

爱⌒轻易说出口 提交于 2021-02-08 15:03:56

问题


Is there a way to list all public links for all the objects stored into a Google Cloud Storage bucket (or a directory in a bucket) using Cloud SDK's gsutil or gcloud?

Something like:

$ gsutil ls --public-link gs://my-bucket/a-directory


回答1:


Public links for publicly visible objects are predictable. They just match this pattern: https://storage.googleapis.com/BUCKET_NAME/OBJECT_NAME.

gsutil doesn't have a command to print URLs for objects in a bucket, but it can just list objects. You could pipe that to a program like sed to replace those listings with object names. For example:

gsutil ls gs://pub/** | sed 's|gs://|https://storage.googleapis.com/|'

The downside here is that this would produce links to all resources, not just those that are publicly visible. So you'd need to either know which resources are publicly visible, or you'd need to write a more elaborate filter based on gsutil ls -L.




回答2:


Even though the question is about a possible flag passed to gsutil to achieve the desired result and since there isn't one at the moment, I'd like to post another programmatic approach using a Cloud Storage Client Library that could be extended and/or adapted to Python modules.

Is as follows (the only third party dependency is google-cloud-storage):

python3 -c """
from operator import attrgetter
from pathlib import Path
import sys

from google.cloud import storage

url = Path(sys.argv[1]) #a blob with the objects we want...

bucket = storage.Client().bucket(url.parent.name)

urls = tuple(map(attrgetter('public_url'), filter(lambda blob:not blob.name.endswith('/'), bucket.list_blobs(prefix=url.name)))) # TODO improve this as not only excludes self blob as homologous 'folder' abstraction blobs inside

print('\n'.join(urls))
""" gs://my-bucket/a-directory


来源:https://stackoverflow.com/questions/39609497/listing-all-public-links-for-all-objects-in-a-bucket-using-gsutil

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!