S h o r t S t o r i e s

// Tales from software development

MSBuild: Behaviour of the CreateItem task

leave a comment »

The CreateItem task allows you to dynamically create item lists and is a very useful and powerful feature but it does have some quirks that are worth understanding.

A quick introduction

Just to refresh your memory, the CreateItem task is typically coded like this:

  <CreateItem Include="@(MyItems)" >
   <Output TaskParameter="Include" ItemName="MyOutputItems" />
  </CreateItem>

The items in the MyItems list are copied to the MyOutputItems item list.

CreateItem does not replace items in a list

The first thing to note is that CreateItem always adds items to a list and does not overwrite any existing items if the item list being written to already exists.

Executing CreateItem in a target

When the current target is exited all items added will now be in the item list. This is obvious in non-batched targets but in a target that is batched on the same item list as the one used as the input to the CreateItem task, the visibility of added items within the target is constrained by the batching process.

You might expect that at the end of the execution of the target the item list contains only the item that was added  in the last batched execution of the target. This is what you’d expect in a procedural language that was setting a variable on each execution of a loop but MSBuild is more of a declaritive language than a procedural one.

This behaviour is not always obvious and is of significance in some batching scenarios as explained next.

Batching considerations

In a non-batched target, or a target that is batched on something other than the input to the CreateItem task, the result of CreateItem is an item list with all input items added.

In a the target that is batched on the same item list as the input to the CreateItem task, the item list created will only ever appear to have one item added to it within the scope of the target. If you’ve used batching then hopefully this will make sense – it’s the only logical way that this could work. When the target is exited the item list will contain all the added items. Again, this makes sense but it is a little bit counterintuitive in some situations.

But here’s the catch… While batching, the item list being added to will appear to only have one item added from the item list that is being batched on but it will have all items that were already in the list when the target was called. The examples in the next section illustrate this.

Examples

The first example demonstrates the behaviour in non-batched targets:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Full" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <ItemGroup> 
  <ItemList1 Include="Item 1" />
  <ItemList1 Include="Item 2" />
  <ItemList1 Include="Item 3" /> 
 </ItemGroup>
 <Target Name="Full" DependsOnTargets="List1;UnbatchedTarget;List2">
 </Target>
 
 <Target Name="List1">
  <Message Text="List1: @(item)" Importance="High" />
 </Target>
 
 <Target Name="List2">
  <Message Text="List2: @(item)" Importance="High" />
 </Target>
 <Target Name="UnbatchedTarget" >
  <CreateItem Include="@(ItemList1)">
   <Output TaskParameter="Include" ItemName="item" />
  </CreateItem>
  
  <Message Text="UnbatchedTarget: @(item)" Importance="High" />
 </Target>
</Project>

The output from this is:

Target List1:
    List1:
Target UnbatchedTarget:
    UnbatchedTarget: Item 1;Item 2;Item 3
Target List2:
    List2: Item 1;Item 2;Item 3
Build succeeded.
    0 Warning(s)
    0 Error(s)

The output from the List1 task shows that the item list is empty at this point.

The output from the Message task in the UnbatchedTarget shows that all three items defined in ItemList1 have now been added to the item list ‘item’ by the CreateItem task.

The output from the List2 target is the same as the UnbatchedTarget output.

The second example demonstrates the behaviour in a batched scenario:

<?xml version="1.0" encoding="utf-8"?>
<Project DefaultTargets="Full" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
 <ItemGroup>
 
  <ItemList1 Include="Item 1" />
  <ItemList1 Include="Item 2" />
  <ItemList1 Include="Item 3" />
 
  <ItemList2 Include="Item A" />
  <ItemList2 Include="Item B" />
  <ItemList2 Include="Item C" /> 
 
 </ItemGroup>
 <Target Name="Full" DependsOnTargets="List1;BatchedTarget1;List2;BatchedTarget2;List3">
 </Target>
 
 <Target Name="List1">
  <Message Text="List1: @(item)" Importance="High" />
 </Target>
 
 <Target Name="List2">
  <Message Text="List2: @(item)" Importance="High" />
 </Target>
 <Target Name="List3">
  <Message Text="List3: @(item)" Importance="High" />
 </Target>
 <Target Name="BatchedTarget1" Inputs="%(ItemList1.Identity)Dummy" Outputs="%(ItemList1.Identity)Dummy" >
  <CreateItem Include="@(ItemList1)">
   <Output TaskParameter="Include" ItemName="item" />
  </CreateItem>
  
  <Message Text="BatchedTarget1: @(item)" Importance="High" />
 </Target>
 <Target Name="BatchedTarget2" Inputs="%(ItemList2.Identity)Dummy" Outputs="%(ItemList2.Identity)Dummy" >
  <CreateItem Include="@(ItemList2)">
   <Output TaskParameter="Include" ItemName="item" />
  </CreateItem>
  
  <Message Text="BatchedTarget2: @(item)" Importance="High" />
 </Target>
</Project>

The output from this project is:

Target List1:
    List1:
Target BatchedTarget1:
    BatchedTarget1: Item 1
Target BatchedTarget1:
    BatchedTarget1: Item 2
Target BatchedTarget1:
    BatchedTarget1: Item 3
Target List2:
    List2: Item 1;Item 2;Item 3
Target BatchedTarget2:
    BatchedTarget2: Item 1;Item 2;Item 3;Item A
Target BatchedTarget2:
    BatchedTarget2: Item 1;Item 2;Item 3;Item B
Target BatchedTarget2:
    BatchedTarget2: Item 1;Item 2;Item 3;Item C
Target List3:
    List3: Item 1;Item 2;Item 3;Item A;Item B;Item C
Build succeeded.
    0 Warning(s)
    0 Error(s)

As before, the List1 target shows that the item list being created is empty at this point.

BatchedTarget1 displays a single item value for as many items in the list. This is what you would expect as the target is batched on the item list that is being used as the input to the CreateItem task.

The output from the List2 target shows that the item list now contains all three items from the ItemList1 item list.

Now it gets interesting… What happens if you use the item list created in another CreateItem task ? The output from BatchedTarget2 shows exactly what happens – the item list contains the items that were already in it when the target was called and each item that the target is batched on, ItemList2 in this case.

Again, when the target is exited, all the items that were added during the execution of the target are added so that the item list now contains all of the items in ItemList1 (added by BatchedTarget1) and all of the items added by BatchedTarget2.

Does it matter ?

Do you need to understand this behaviour ? Maybe, maybe not… It’s useful to remember the basic concept because it will explain behaviour that you may regard as odd in some circumstances.

To give an example, I recently wrote a project to collect together some source files for delivery to a client. Some of the files had to be retrieved from a source control system while others were on a file server.

I wrote a couple of targets. The first accesses the source control system, syncs the required files, and copies them to the target folder. The second simply copies specific files from the file server to the target folder.

The files for each target are defined using separate item lists. Each list contains several items, each of which describes a set of files to be copied. Each item includes metadata for files to be excluded, the location under the target folder where the files are to be copied, etc.

The two targets use the CreateItem task to dynamically create an item list that is be passed to the MSBuild Copy task. The first is in the target that copies from the source control system:

  <CreateItem Include="%(SourceDepotCopy.DepotPath)\**\*.*" Exclude="%(SourceDepotCopy.Exclude)" >
   <Output TaskParameter="Include" ItemName="SourceFiles" />
  </CreateItem>

The second is in the target that copies the files from the file server:

  <CreateItem Include="%(FileSystemCopy.SourceFiles)" Exclude="%(FileSystemCopy.Exclude)" >
   <Output TaskParameter="Include" ItemName="SourceFiles" />
  </CreateItem>

Note that both CreateItem tasks output to the SourceFiles item list.

The result isn’t what I intended. The first target executes correctly but the second doesn’t. As explained above, when the second batched target is executed the output item list for the CreateItem task will already have all the items that were added during the execution of the first target and batched items will be added for each time the task is executed as determined by the batching on the task.

In short, when the second target is executing, the SourceFiles item list is full of the items added in the first target and so copies hundreds of files from the source control system to the locations specified in the metadata in the items in the item list used to control the copying of files from the file server.

The solution is to simply use a different name for the item list used in the second target:

  <CreateItem Include="%(FileSystemCopy.SourceFiles)" Exclude="%(FileSystemCopy.Exclude)" >
   <Output TaskParameter="Include" ItemName="FilesToCopy" />
  </CreateItem>

The project now does what was intended.

I deliberately used the same name on both CreateItem tasks and I should have realised what would happen but I overlooked it. More worryingly, it would be easy to accidentally use the same item list name in different places in a project and not realise it because MSBuild gives no warnings about the reuse of an item list such as this.

Advertisements

Written by Sea Monkey

June 6, 2008 at 6:00 pm

Posted in Development

Tagged with

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: