Spring Batch - Process lines in huge file, start new job for each line

帅比萌擦擦* 提交于 2020-01-06 04:13:11

问题


Addition to this question: Spring Batch - where does the process run

Once every hour i get a huge file with Usage Data for a set of services. A Usage Data line has a UserId and a ServiceId and how long/much the service has been used.

For each Usage Data i need to check which Subscription the User has, and how much the service cost per. unit for this particular Subscription. This will result in a line i can bill to the user.

For each line:

  1. Database: Find Subscription by User
  2. Database: Find Service cost by Subscription
  3. Task: Calculate price
  4. Database: Write billing line

It is important to notice that Subscriptions will not always have a cost for a specific service. Because of this each line can fail untill a user adds a cost for the given Subscription and service.

There is alot of logic for each line, and each of these can fail, with a need to rerun.

I have also been thinking of fetching all UserSubscriptions, Subscriptions and SubscriptionServiceCost into memory and then itteration through these, so i will only do a select in the beginning of the flow. Instead of having to do a select for each line. Furthermore i could batch insert the billing line at the end of the flow.

How do i go down the correct path for my design using Spring Batch?

来源:https://stackoverflow.com/questions/31558148/spring-batch-process-lines-in-huge-file-start-new-job-for-each-line

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!