Pyspark - identical dataframe filter operation gives different output

Question

I’m facing a particularly bizarre issue while firing filter queries on a spark dataframe. Here\'s a screenshot of the filter command I\'m trying to run:

As yo