Differences between Amazon S3 and S3n in Hadoop

后端 未结 3 1816
隐瞒了意图╮
隐瞒了意图╮ 2020-12-12 12:48

When I connected my Hadoop cluster to Amazon storage and downloaded files to HDFS, I found s3:// did not work. When looking for some help on the Internet I foun

3条回答
  •  [愿得一人]
    2020-12-12 13:17

    I think your main problem was related with having S3 and S3n as two separate connection points for Hadoop. s3n:// means "A regular file, readable from the outside world, at this S3 url". s3:// refers to an HDFS file system mapped into an S3 bucket which is sitting on AWS storage cluster. So when you were using a file from Amazon storage bucket you must be using S3N and that's why your problem is resolved. The information added by @Steffen is also great!!

提交回复
热议问题