Need a way to prevent unwanted job param from propagating to next execution of spring boot batch job

问题

I am running a batch app using spring boot 2.1.2 and spring batch 4.1.1. The app uses a MySQL database for the spring batch metadata data source.

First, I run the job with this command:

java -jar target/batchdemo-0.0.1-SNAPSHOT.jar -Dspring.batch.job.names=echo com.paypal.batch.batchdemo.BatchdemoApplication myparam1=value1 myparam2=value2

Notice I am passing two params:

myparam1=value1 myparam2=value2

Since the job uses RunIdIncrementer, the actual params used by the app are logged as:

Job: [SimpleJob: [name=echo]] completed with the following parameters: [{myparam2=value2, run.id=1, myparam1=value1}]

Next I run the job again, this time dropping myparam2:

java -jar target/batchdemo-0.0.1-SNAPSHOT.jar -Dspring.batch.job.names=echo com.paypal.batch.batchdemo.BatchdemoApplication myparam1=value1

This time the job again runs with param2 still included:

Job: [SimpleJob: [name=echo]] completed with the following parameters: [{myparam2=value2, run.id=2, myparam1=value1}]

This causes business logic to be invoked as if I had again passed myparam2 to the app.

Is there a way to drop the job parameter and have it not be passed to the next instance?

App code:

package com.paypal.batch.batchdemo;

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;

@SpringBootApplication
@EnableBatchProcessing
public class BatchdemoApplication {

    public static void main(String[] args) {
        SpringApplication.run(BatchdemoApplication.class, args);
    }

    @Autowired
    JobBuilderFactory jobBuilder;

    @Autowired
    StepBuilderFactory stepBuilder;

    @Autowired
    ParamEchoTasklet paramEchoTasklet;

    @Bean
    public RunIdIncrementer incrementer() {
        return new RunIdIncrementer();
    }

    @Bean
    public Job job() {
        return jobBuilder.get("echo").incrementer(incrementer()).start(echoParamsStep()).build();
    }

    @Bean
    public Step echoParamsStep() {
        return stepBuilder.get("echoParams").tasklet(paramEchoTasklet).build();
    }
}

package com.paypal.batch.batchdemo;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.batch.core.StepContribution;
import org.springframework.batch.core.scope.context.ChunkContext;
import org.springframework.batch.core.step.tasklet.Tasklet;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.stereotype.Component;

@Component
public class ParamEchoTasklet implements Tasklet {

    @Override
    public RepeatStatus execute(StepContribution contribution, ChunkContext chunkContext) throws Exception {
        LOGGER.info("ParamEchoTasklet BEGIN");
        chunkContext.getStepContext().getJobParameters().entrySet().stream().forEachOrdered((entry) -> {
            String key = entry.getKey();
            Object value = entry.getValue();
            LOGGER.info("Param {} = {}", key, value);
        });
        LOGGER.info("ParamEchoTasklet END");
        return RepeatStatus.FINISHED;
    }

    private Logger LOGGER = LoggerFactory.getLogger(ParamEchoTasklet.class);
}

I debugged the spring batch and spring boot code, and here is what is happening. JobParametersBuilder line 273 adds the params from the most recent prior job instance to the nextParameters map along with any params added by the JobParametersIncrementer:

List<JobExecution> previousExecutions = this.jobExplorer.getJobExecutions(lastInstances.get(0));
if (previousExecutions.isEmpty()) {
    // Normally this will not happen - an instance exists with no executions
    nextParameters = incrementer.getNext(new JobParameters());
}
else {
    JobExecution previousExecution = previousExecutions.get(0);
    nextParameters = incrementer.getNext(previousExecution.getJobParameters());
}

Then since I am using spring boot, JobLauncherCommandLineRunner line 213 merges the prior params with the new params passed for the new execution, which results in the old param being passed to the new execution:

return merge(nextParameters, jobParameters);

It appears to be impossible to run the job ever again without the param unless I am missing something. Could it be a bug in spring batch?

回答1:

The normal behavior for RunIdIncrementer appears to increment the run id for the JobExecution and pass along the remaining prior JobParameters. I would not call this a bug.

Keep in mind that the idea behind the RunIdIncrementer is simply to change one identifying parameter to allow a job to be run again, even if a prior run with the same (other) parameters completed successfully and restart has not been configured.

You could always create a customized incrementer by implementing JobParametersIncrementer.

Another alternative is to use the JobParametersBuilder to build a JobParameters object and then use the JobLauncher to run your job with those parameters. I often use the current system time in milliseconds to create uniqueness if I'm running jobs that will otherwise have the same JobParameters. You will obviously have to figure out the logic for pulling your specific parameters from the command line (or wherever else) and iterating over them to populate the JobParameters object.

Example:

public JobExecution executeJob(Job job) {
    JobExecution jobExecution = null;
    try {
        JobParameters jobParameters =
            new JobParametersBuilder()
                .addLong( "time.millis", System.currentTimeMillis(), true)
                .addString( "param1", "value1", true)
                .toJobParameters();
        jobExecution = jobLauncher.run( job, jobParameters );
    } catch ( JobInstanceAlreadyCompleteException | JobRestartException | JobParametersInvalidException | JobExecutionAlreadyRunningException e ) {
        e.printStackTrace();
    }
    return jobExecution;
}

来源：https://stackoverflow.com/questions/55750109/need-a-way-to-prevent-unwanted-job-param-from-propagating-to-next-execution-of-s