REST Api with Multithreading for handling Files in Spring Boot

问题

Is their any way so that i can make use of Multithreading concept to call the execution in parallel and make execution faster for created @RestController which will accepts a String and List<MultipartFile> as request parameters, and the code is working fine.Problem here is if i'm parsing one file after other via a for loop. Time taken for execution is more.

Below is Controller

@RequestMapping(value = "/csvUpload", method = RequestMethod.POST)
    public List<String> csvUpload(@RequestParam String parentPkId, @RequestParam List<MultipartFile> file)
            throws IOException {
        log.info("Entered method csvUpload() of DaoController.class");
        List<String> response = new ArrayList<String>();
        String temp = parentPkId.replaceAll("[-+.^:,]", "");
        for (MultipartFile f : file) {
            String resp = uploadService.csvUpload(temp, f);
            response.add(resp);
        }
        return response;
    }

from controller, i'm calling uploadService.csvUpload() method where i'm parsing the files one after the other as i'm using For loop.

Below is my UploadService Class

public String csvUpload(String parentPkId, MultipartFile file) {
        try {
            BufferedReader br = new BufferedReader(new InputStreamReader(file.getInputStream()));
            String line = "";
            int header = 0;
            while ((line = br.readLine()) != null) {
                // TO SKIP HEADER
                if(header == 0) {
                    header++;  
                    continue;
                }
                header++;
                //Use Comma As Separator
                String[] csvDataSet = line.split(",");
                //Saving it to DB

        }catch(IOException ex) {
            ex.printStackTrace();
        }
        return "Successfully Uploaded "+ file.getOriginalFilename();
    }

How to make this Controller as a Multithreaded so that processing is parallel and fast. I'm new to Multithreading and I tried by making use of Callable interface but the Call() method will not take parameters.

any leads and suggestion are welcomed, thanks in advance.

回答1:

you need to create a Class which will implement callable as below and store futures in a list and finally process the futures as below

public class ProcessMutlipartFile implements Callable<String>
{
   private Mutlipartfile file; 
   private String temp;
   private UploadService uploadService;
   public ProcessMutlipartFile(Mutlipartfile file,String temp, UploadService uploadService )
   {
       this.file=file;
       this.temp=temp,
       this.uploadService=uploadService;
   }

   public String call() throws Exception 
   {

    return   uploadService.csvUpload(temp, file);
    }

 }

in your controller create a list of future object

 ExecutorService executor = Executors.newFixedThreadPool(10)
List< Future<String> > futureList = new ArrayList<Future<String>>();
     .
     .
     .
         for (MultipartFile f : file) {
                    futureList.add(executor.submit(new ProcessMutlipartFile(file ,temp,uploadService));
                }

finally in your controller

 for (Future f :futureList)
 {
  response.add(f.get());
  }

//shuttingdown the Executor
executor.shutdown();

hope this helps

回答2:

You can execute the uploading code using parallel stream

    List<String> response = file.parallelStream().map(f -> uploadService.csvUpload(temp, f))
    .collect(Collectors.toList());

You can execute streams in serial or in parallel. When a stream executes in parallel, the Java runtime partitions the stream into multiple substreams. Aggregate operations iterate over and process these substreams in parallel and then combine the results.

来源：https://stackoverflow.com/questions/54899271/rest-api-with-multithreading-for-handling-files-in-spring-boot

标签

java

multithreading

spring-boot

threadpool