问题
In the spring batch program, I am reading the records from a file and comparing with the DB if the data say column1 from file is already exists in table1.
Table1 is fairly small and static. Is there a way I can get all the data from table1 and store it in memory in the spring batch code? Right now for every record in the file, the select query is hitting the DB.
The file is having 3 columns delimited with "|".
The file I am reading is having on an average 12 million records and it is taking around 5 hours to complete the job.
回答1:
Preload in memory using a StepExecutionListener.beforeStep (or @BeforeStep).
Using this trick data will be loaded once before step execution.
This also works for step restarting.
回答2:
I'd use caching like a standard web app. Add service caching using Spring's caching abstractions and that should take care of it IMHO.
回答3:
Load static table in JobExecutionListener.beforeJob(-) and keep this in jobContext and you can access through multiple steps using 'Late Binding of Job and Step Attributes'. You may refer 5.4 section of this link http://docs.spring.io/spring-batch/reference/html/configureStep.html
来源：https://stackoverflow.com/questions/33633977/getting-data-from-db-in-spring-batch-and-store-in-memory