Executing MySQL SELECT * query in parallel

那年仲夏 提交于 2019-12-10 22:18:31

问题


I have a multithreaded application that periodically fetches the whole content of the MySQL table (with SELECT * FROM query) The application is written in python, uses threading module to multithreading and uses mysql-python (mysqldb) as MySQL driver (using mysqlalchemy as a wrapper produces similar results). I use InnoDB engine for my MySQL database.

I wrote a simple test to check the performance of SELECT * query in parallel and discovered that all of those queries are implemented sequentially.

I explicitly set the ISOLATION LEVEL to READ UNCOMMITTED, although it does not seem to help with performance.

The code snipper making the DB call is below:


@performance.profile()
def test_select_all_raw_sql(conn_pool, queue):
    ''' 
    conn_pool - connection pool to get mysql connection from
    queue - task queue
    '''
    query = '''SELECT * FROM table'''
    try:
        conn = conn_pool.connect()
        cursor = conn.cursor()
        cursor.execute("SET SESSION TRANSACTION ISOLATION LEVEL READ UNCOMMITTED")
        # execute until the queue is empty (Queue.Empty is thrown)
        while True:
            id = queue.get_nowait()
            cursor.execute(query)
            result = cursor.fetchall()
    except Queue.Empty:
            pass
    finally:
        cursor.execute("SET SESSION TRANSACTION ISOLATION LEVEL REPEATABLE READ")
        conn.close()

Am I right expecting this query to be executed in parallel? If yes, how can I implement that in python?

来源:https://stackoverflow.com/questions/21619856/executing-mysql-select-query-in-parallel

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!