Powershell Throttle Multi thread jobs via job completion

Question

All the tuts I have found use a pre defined sleep time to throttle jobs. I need the throttle to wait until a job is completed before starting a new one. Only 4 jobs can be r

Answer 1

Instead of sleep 10 you could also just wait on a job (-any job):

Get-Job | Wait-Job -Any | Out-Null

When there are no more jobs to kick off, start printing the output. You can also do this within the loop immediately after the above command. The script will receive jobs as they finish instead of waiting until the end.

Get-Job -State Completed | % {
   Receive-Job $_ -AutoRemoveJob -Wait
}

So your script would look like this:

$servers = Get-Content "C:\temp\flashfilestore\serverlist.txt"

$scriptBlock = { #DO STUFF }

$MaxThreads = 4

foreach ($server in $servers) {
   Start-Job -ScriptBlock $scriptBlock -argumentlist $server 
   While($(Get-Job -State Running).Count -ge $MaxThreads) {
      Get-Job | Wait-Job -Any | Out-Null
   }
   Get-Job -State Completed | % {
      Receive-Job $_ -AutoRemoveJob -Wait
   }
}
While ($(Get-Job -State Running).Count -gt 0) {
   Get-Job | Wait-Job -Any | Out-Null
}
Get-Job -State Completed | % {
   Receive-Job $_ -AutoRemoveJob -Wait
}

---

Having said all that, I prefer runspaces (similar to Ryans post) or even workflows if you can use them. These are far less resource intensive than starting multiple powershell processes.