PowerShell, Start-Job, -ScriptBlock = sad panda face

I am working on a project where I am using PowerShell to collect a lot of performance counters from a lot of servers.  More on that later.  For now I wanted to highlight an important lesson I learned when trying to use Start-Job to call a PS script using -ScriptBlock and passing in parameters.  This could be a comedy of errors if you haven't come across it before, so I thought it might be useful to throw up a quick post about it.

To keep things simple, let's say I am calling a script with two parameters, am exporting a CSV file, and want those parameters embedded in the name of the CSV file.  Something nice and easy, like this:

 param ([string]$foo, [string]$bar)                          function JobTest {                 param ([string]$foo, [string]$bar)                              Get-Counter | Export-Csv "C:\csv\$foo-$bar.csv";             }                          JobTest -foo $foo -bar $bar;

Calling this from the command line is straightforward, and works as expected:

C:\csv\JobTest.ps1 "test" "1";

Result: success!  The file "test-1.csv" is created and we are happy.

Now, let's make things a little more complicated. Let's say with one script I want to call multiple jobs that will call this script asynchronously for a set of values for $foo and $bar.  I might try creating a second script file like this:

param ([string]$foo, [string]$bar)                          function StartJobs {                 param ([string]$foo, [string]$bar)                              Start-Job -ScriptBlock { C:\csv\JobTest.ps1 $foo $bar }             }                          StartJobs -foo $foo -bar $bar

When I call this from the command line:

C:\csv\StartJobs.ps1 "test" "2";

Result: not so good. A file is created, but it is named "-.csv" (so "test" and "2" got lost somehow).

Next, I tried embedding the variables in quotes in the Start-Job call (in the remaining code samples, I am just showing the changes to line 6 of StartJobs.ps1):

Start-Job -ScriptBlock { C:\csv\JobTest.ps1 "$foo" "$bar" }

Result: still not good. The "-.csv" file is over-written by a new version.

After that, I tried using -ArgumentList to pass the values in:

Start-Job -ScriptBlock { C:\csv\JobTest.ps1 "$foo" "$bar" } -ArgumentList $foo $bar

Result: error message as follows:

Start-Job : Cannot bind parameter 'InitializationScript'. Cannot convert the "5" value of type "System.String" to type "System.Management.Automation.ScriptBlock".
At C:\csv\StartJobs.ps1:6 char:11
+     Start-Job <<<<  -ScriptBlock { C:\csv\JobTest.ps1 "$foo" "$bar" } -ArgumentList $foo $bar
    + CategoryInfo          : InvalidArgument: (:) [Start-Job], ParameterBindingException
    + FullyQualifiedErrorId : CannotConvertArgumentNoMessage,Microsoft.PowerShell.Commands.StartJobCommand

Well, of course… silly me, I forgot the comma between $foo and $bar.  Let's try this:

Start-Job -ScriptBlock { C:\csv\JobTest.ps1 "$foo" "$bar" } -ArgumentList $foo, $bar

Result: no error this time, but we have created yet another file named "-.csv" so our arguments are still not getting passed in correctly. 

A bit of searching around and I see that -ArgumentList should be used with $args[i] as opposed to the explicitly-named argument.  Aha!  I am lulled into the belief that I have finally figured out the problem.  So I try this:

Start-Job -ScriptBlock { C:\csv\JobTest.ps1 "$args[0]" "$args[1]" } -ArgumentList $foo, $bar

Result: once again no error, but once again we have created another "-.csv" file. Bummer.

I search exhaustively for alternate syntax examples – surely someone out there is passing parameters into a Start-Job call?  I find the @(arg, arg) syntax somewhere (all apologies, I don't recall where I spotted this).  So I try it this way:

Start-Job -ScriptBlock { C:\csv\JobTest.ps1 "$args[0]" "$args[1]" } -ArgumentList @($foo, $bar)

Result: nada. Still a poorly-named "-.csv" file.  I try one more stab, and remove the quotes around the arguments within the -ScriptBlock:

Start-Job -ScriptBlock { C:\csv\JobTest.ps1 $args[0] $args[1] } -ArgumentList @($foo, $bar)

Result: success! Finally, I have a file named "test-9.csv" – it only took 9 tries (and lots of cursing) to get the syntax right!  For the PowerShell veterans and gurus out there, you are probably saying, "DUH!"  But I am neither, and I spent a lot of time experimenting with this and trying to figure out what the problem was, so I hope this helps prevent the same frustration for someone else at some point.

 

Aaron Bertrand

I am a passionate technologist with industry experience dating back to Classic ASP and SQL Server 6.5. I am a long-time Microsoft MVP, speak frequently at conferences, and write at SQLPerformance and MSSQLTips. I recently joined the team at Wayfair and hope to blog about some of the scale challenges I've already started helping to solve.

29 Responses

  1. AllenMWhite says:

    Aaron, the @() construct creates an empty collection, so the @($foo, $bar) creates a collection with $foo in the [0] member and $bar in the [1] member, which of course feeds your $args collection.  I learned about the @() construct in Dr. Tobias Weltner's PowerShell book, which you can download free at his site at http://www.powershell.com/

  2. BartekB says:

    Yes, working with jobs can be real pain in the… back. :/ BTW: for more complicated jobs you can use param statement inside scriptblock and make things more obvious if you read them later. Unfortunately -ArguemtnList does not supports named parameters, so all param () elements need to come in the right order.
    Also, if all you need is expanded parameter just expand them when you create scriptblock:
    Start-Job -ScriptBlock ([scriptblock]::create("C:\csv\JobTest.ps1 $foo $bar"))

  3. Scott R. says:

    Aaron,
    Interesting reading about your PowerShell / PerfMon exploits, especially as another PowerShell newbie.
    As an alternative, have you considered the option of using a PerfMon collection to a text file (csv or tsv) created and managed by the PerfMon tools built in to the OS?  The collection definition can be created and scheduled using either the interactive GUI approach or using the LogMan command line utility.  I don't know if this approach is useful for your situation.
    I am getting more into using PowerShell in different ways, and am in favor of using it where it best fits, and in combination with other tools where they best fit.
    Just some more options to consider in the buy / build / reuse spectrum of solutions.
    Thanks for sharing your efforts.
    Scott R.

  4. AaronBertrand says:

    Hi Scott,
    Yes, I have used LogMan in a previous iteration of this project – I'll talk a bit about this in a post I'll be ready to publish in the morning.  The reason I went with PowerShell is simply to consolidate the technologies used to preform various aspects of the project (building VMs, simulating load, running queries, and collecting counters).  If LogMan had a significant advantage over PowerShell I probably would have stuck with it; the only advantage I saw, though, was that I already had a few batch files written.
    Cheers,
    Aaron

  5. Brett R says:

    Dude, you saved me hours!!!!!  I was literally just starting on the frustrating road you walked and found this page….TG for Google and you my friend!

  6. James J says:

    You saved me many hours of hair pulling, me having learnt Powershell only yesterday and was happily creating scripts before I stumbled across this not so intuitive syntax.

  7. Aaron Bertrand says:

    Brett & James, great to hear, glad it helped!

  8. Mike says:

    I also spent frustrating hours about a year ago with this, but I ended up with a different solution where I didn't need the empty array "@()" to get the argument list to work. Instead I used the -filepath parameter. I don't know if it helps for what you were doing, but here's what I use:
    $MyJob = start-job -name $Index -initializationscript {Add-PSSnapin Microsoft.Exchange.Transporter} -filepath "c:\scripts\migrate.ps1" -ArgumentList $User,$Index
    Anyway, your post is great info, thanks for the post!
    -Mike

  9. diver says:

    Thanks for your sharing.
    I tried and tried and tried and then I started to search the internet… And found your really helpful tip!
    Lots of wasted time for a simple job to do.

  10. Jeffery Hicks says:

    Scriptblocks can use the Param statement just like functions. I find this a little easier to follow and use
    Start-Job -ScriptBlock {param([string]$foo,[string]$bar) C:\csv\JobTest.ps1 $foo $bar }  -ArgumentList $foo,$bar
    The arguments are still passed in sequential order. Here's a post with more examples of this technique. http://jdhitsolutions.com/blog/2010/08/friday-the-13-script-blocks/

  11. GLOOX says:

    THANKS A LOT !

  12. Chris says:

    You can also do the following, which I think is a bit simpler:
    Invoke-Expresion "Start-Job -ScriptBlock { C:\csv\JobTest.ps1 $foo $bar }"
    Invoke-Expression will expand your variables prior to calling Start-Job.  I just found this tonight on some other website (sorry, don't remember where it was).

  13. kiquenet says:

    Can I use named parameters ??
    Can I use variable for ps1 script like
    $myScript = "C:\csv\JobTest.ps1"
    Which is the code for script ??
    -ScriptBlock { $myScript… } -ArgumentList ….

  14. jack says:

    Starting background jobs doesn't seem to work under Windows 2003 R2 SP2 32bit.  Jobs just run forever.  At least not with parameters.  Anyone else found a way around this?

  15. Hi Jack says:

    I found the same problem.  'Invoke-Command -AsJob ….' or 'Start-Job …' with params being passed in just doesn't fly on Windows 2003.

  16. brendan62269 says:

    start-job -filepath c:\script.ps1 -args $string,"arg2","$etc"
    $dosomethingvariable = [string] "dosomethingthisway"
    $arguments = $somevariable,"dontdoitthatway","haveacoke","andasmile"
    start-job -filepath c:\script.ps1 -args $arguments
    *
    -args or -argumentlist should come last
    **
    your script needs to specify the 'Posistion' for each $arg (in your scripts param() declarations)and the $args need to be in the right order
    There are a few ways to create an array, one method may suit a need better than the other.  Here are the methods I use most often:
    PS C:\> remove-variable a
    PS C:\> $a
    PS C:\> $a = @()
    PS C:\> $a += "one"
    PS C:\> $a += "two"
    PS C:\> $a
    one
    two
    PS C:\> remove-variable a
    PS C:\> $a
    PS C:\> $a = "one","two" #auto-creates $a as an array
    PS C:\> $a
    one
    two
    PS C:\> remove-variable a
    PS C:\> $a
    PS C:\> $a = [array]"one"
    PS C:\> $a += "two"
    PS C:\> $a
    one
    two
    #inside of a one-liner
    PS C:\> remove-variable a
    PS C:\> $a
    PS C:\> [email protected]();$a+="one";$a+="two"
    PS C:\> $a
    one
    two
    If you don't declare $a to be a variable (somehow), then added data will concat to the string:
    PS C:\> remove-variable a
    PS C:\> $a
    PS C:\> $a += "one"
    PS C:\> $a += "two"
    PS C:\> $a
    onetwo

  17. brendan62269 says:

    @Jack
    I've been having problems with start-job or invoke-command when something in the script has a progress indicator or do{}until() loops.  I'll post something when I figure it out…

  18. Dylan says:

    @Chris Great tip, thanks!!

  19. Peter says:

    The last couple tries that failed failed only because you forgot the rules about expanding variables in strings.
    $a = "foo"
    #expands the variable
    "$a" == "foo"
    $a = 1,2,3
    "$a[1]" == "1 2 3[1]"
    #this happens because the only thing expanded is the reference to a
    #not the expression following it
    "$($a[1])" is what you are expecting
    Similarly, in your exmaples "$($args[1])" "$($args[2])" I bet would have worked.

  20. James says:

    All I can say is thank you!!!!!!!!!!! Been banging my head against this exact problem all morning!

  21. Nathan says:

    Thank you! Many tips out there for this same thing but none of them go through the kind of walk through you provide here that gives a real good picture of what you need to do for passing multiple values into the "job-start scriptblock". This lesson will sure save time for me the next time I run into a script block within PowerShell. Thanks for posting this.

  22. Ori Gil says:

    2 and a half years for this post and it is so helpful and relevant.
    I can't believe Microsoft doesn't bother to update the command help description with more relevant examples…
    Aaron you are the greatest!

  23. Mendel says:

    Thanks!
    Just needed this! 🙂
    Damn argumentlist as an array did the trick!

  24. Matt C says:

    You just saved me from tearing out any further hair on this problem! Thanks for the post, man 🙂

  25. Patrick S says:

    Thanks for the post. I went through more than 9 iterations before your post sorted me out!

  26. Xeleema says:

    Thanks! I spent most the day yesterday trying to figure this out!
    (Had to 'cd' to a directory first, *then* run a script)
    [email protected]("svr1","svr2","svr3")
    foreach($x in $a){Start-Job -ScriptBlock{cd $args[0];E:\PS\Script.ps1 $args[1]} -ArgumentList @("E:\PS",$x)}

  27. Tahir Hassan says:

    This is simples. You do not use a script block.  Intead you pass the (string) path of the script to the Start-Job command together with the Argument list:
    Start-Job C:\csv\JobTest.ps1 -ArgumentList @($foo, $bar)

  28. Gordon Freeman says:

    In Powershell 3, you'd be able to write it like this:
    Start-Job -ScriptBlock { C:\csv\JobTest.ps1 $using:foo $using:bar }