Reading from compressed files in Dataflow

后端 未结 4 1773
半阙折子戏
半阙折子戏 2021-01-14 14:07

Is there a way (or any kind of hack) to read input data from compressed files? My input consists of a few hundreds of files, which are produced as compressed with gzip and u

4条回答
  •  谎友^
    谎友^ (楼主)
    2021-01-14 14:43

    I just noticed that specifying the compression type is now available in the latest version of the SDK (v0.3.150210). I've tested it, and was able to load my GZ files directly from GCS to BQ without any problems.

提交回复
热议问题