how to read hdfs file with wildcard character used by pyspark

问题

there are some parquet file paths are:

/a/b/c='str1'/d='str'

/a/b/c='str2'/d='str'

/a/b/c='str3'/d='str'

I want to read the parquet files like this:

df = spark.read.parquet('/a/b/c='*'/d='str')

but it doesn't work by using "*" wildcard character.How can I do that? thank you for helping

回答1:

You need to escape single quotes:

df = spark.read.parquet('/a/b/c=\'*\'/d=\'str\'')

... or just use double quotes:

df = spark.read.parquet("/a/b/c='*'/d='str'")

来源：https://stackoverflow.com/questions/50312789/how-to-read-hdfs-file-with-wildcard-character-used-by-pyspark