Reading parquet files from multiple directories in Pyspark

Question

I need to read parquet files from multiple paths that are not parent or child directories.

for example,

dir1 ---
       |
       ------- dir1_1
            


        

Answer 1

For ORC

spark.read.orc("/dir1/*","/dir2/*")

spark goes inside dir1/ and dir2/ folder and load all the ORC files.

For Parquet,

spark.read.parquet("/dir1/*","/dir2/*")