Error opening a parquet file on Amazon S3 using pyarrow

房东的猫 提交于 2019-12-11 01:23:56

问题


I have this code, which is supposed to read a single column data from a parquet file stored on S3: fs = s3fs.S3FileSystem() data_set = pq.ParquetDataset(f"s3://{bucket}/{key}", filesystem=fs) column_data = data_set.read(columns=[col_name])

and I get this excption: validate_schemas self.schema = self.pieces[0].get_metadata(open_file).schema IndexError: list index out of range

I upgraded to the latest version of pyarrow but it did not help

来源:https://stackoverflow.com/questions/52057964/error-opening-a-parquet-file-on-amazon-s3-using-pyarrow

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!