Google Cloud dataflow : How to initialize Hikari connection pool only once per worker (singleton)?

有些话、适合烂在心里 提交于 2021-02-11 17:40:14

问题


Hibernate Utils is creating the session factory along with Hikari configuration. Currently we are doing inside @Setup method of ParDo, but it opens way too many connections. So is there any good example to initialize connection pool per worker ?


回答1:


If you are using @Setup method inside DoFn to create a database connection keep in mind that Apache Beam would create connection pool per worker instance thread. This might result in a lot of database connections depending on the number of workers that spin up.

You can use a Singleton class inside a ParDo to create your connection pool and control the number of connections per worker. You can take a look here for a sample implementation

http://mail-archives.apache.org/mod_mbox/beam-issues/201905.mbox/%3CJIRA.13231789.1557156230000.210182.1557222420198@Atlassian.JIRA%3E



来源:https://stackoverflow.com/questions/59311710/google-cloud-dataflow-how-to-initialize-hikari-connection-pool-only-once-per-w

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!