Fetching data with snowflake connector throws EmptyPyArrowIterator error

问题

I use python snowflake connector in my python script (plotly dash app) and today the app stopped working without me changing the code. I tried a couple of things to find out what might be the issue and I even tried to run the example code from Snowflake documentation and I got the same error:

code:

cur = conn.cursor()
try:
    cur.execute("SELECT col1, col2 FROM test_table ORDER BY col1")
    for (col1, col2) in cur:
        print('{0}, {1}'.format(col1, col2))
finally:
    cur.close()

error:

Traceback (most recent call last):
  File "db_connection.py", line 48, in <module>
    cur.execute("SELECT col1, col2 FROM test_table ORDER BY col1")
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/snowflake/connector/cursor.py", line 580, in execute
    self._init_result_and_meta(data, _use_ijson)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/snowflake/connector/cursor.py", line 630, in _init_result_and_meta
    self._result = ArrowResult(data, self, use_dict_result=self._use_dict_result)
  File "arrow_result.pyx", line 42, in snowflake.connector.arrow_result.ArrowResult.__init__
  File "arrow_result.pyx", line 156, in snowflake.connector.arrow_result.ArrowResult._reset
NameError: name 'EmptyPyArrowIterator' is not defined

The connection is established, I am capable of creating a table in my database but I cannot seem to query and iterate the data.

I am on macOS Catalina 10.15.1, snowflake-connector-python==2.1.0, Python 3.7.0.

回答1:

Please use the Python connector version 2.1.1

pip install snowflake-connector-python==2.1.1

pip3 install snowflake-connector-python==2.1.1

回答2:

You have to install pyarrow module via

pip3 install pyarrow

Installation of the snowflake connector does not automatically install it.

回答3:

Short version: Roll back your snowflake-connector-python to version 2.0.4 by running: pipenv install "snowflake-connector-python~=2.0.4"

Long version: I checked the commits to the github for the connector, and I saw that they're adding ARROW support. Not sure exactly why arrow_result.pyx is unable to include EmptyPyArrowIterator, but it's wrapped in a try, so it still executes. However, when it gets to the reset function, it can't find the reference and throws an exception.

回答4:

I believe you might have used pip install snowflake -- for installing snowflake connector

try with, pip install snowflake-connector-python

回答5:

I ran into this a few hours ago in the middle of a long batch job. Interestingly the same job worked jut fine, until all of a sudden it started throwing the "EmptyPyArrowIterator" exception and wouldn't restart. Downgrading snowflake-connector-python to 2.0.4 seems to have fixed the problem for now.

回答6:

For More details what else impact , refer the release document below.

https://docs.snowflake.net/manuals/release-notes/2019-12.html#python-connector-2-1-0-jdbc-driver-3-11-0-support-for-internal-arrow-format

To fix I have install pyarrow at user level

pip3 install pyarrow --user

来源：https://stackoverflow.com/questions/59307076/fetching-data-with-snowflake-connector-throws-emptypyarrowiterator-error

标签

python

snowflake-cloud-data-platform