Camel-SQL Why using StreamList seems to load all ResultSet?

无人久伴 提交于 2019-12-21 20:44:16

问题


Hi,

What am I trying to do ?

I am currently working on an ESB project (apache-camel + spring boot 2) where i read a MySQL table with more than 100 000 000 rows. I empty this table 1 row at a time, transform the row and send it to another database.

How am I doing this ?

Currently I use camel-sql to read the data

//edited .from(sql:SELECT * FROM mytable?outputType=StreamList&outpuClass=MyClass) .split(body()).streaming() .bean(mybean, "transform") .end()

Problem :

As I can't make a select * and get all 100M rows in my RAM because it's not possibly big enough, I thougth about using streams.

Therefore: It seems that using the StreamList as outputType still gets all rows at first and only then returns it as a "stream" (ResultSet).

Question

Can't we just use the property of PreparedStatement to really stream data from my database "one row at a time" rather than getting all of it at once and destroy my VM memory ?

Thanks.

来源:https://stackoverflow.com/questions/52200896/camel-sql-why-using-streamlist-seems-to-load-all-resultset

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!