What would you want in an MSBuild integration?

Feb 23, 2010 at 5:31 AM
Edited Feb 23, 2010 at 3:21 PM

I'm integrating StatLight with MSBuild and TFBuild at my day job and I will contribute my solution. My integration includes a dll with the MSBuild task and a targets file to simplify using it.

This is how I'll use it in MSBuild:

 

<Target Name="RunStatLight">
	<Error Condition="'$(StatLightToolPath)' == ''" Text="The required property 'StatLightToolPath' is missing and must declare the path to the StatLight executable" />
	<!-- Get Assemblies to test -->
	<ItemGroup>
		<TestAssemblies Include="$(OutDir)$(AssemblyName).xap" />
	</ItemGroup>
	<!-- Run StatLight-->
	<StatLight ContinueOnError="true" Assemblies="@(TestAssemblies)" OutputXmlFile="$(StatLightResultsFile)" ToolPath="$(StatLightToolPath)">
		<Output TaskParameter="TestRunSucceeded" PropertyName="StatLightSucceeded" />
		<Output TaskParameter="FormattedSummary" PropertyName="StatLightResultSummary" />
	</StatLight>
	<!-- Report error message according to the result -->
	<Message Importance="High" Text="$(StatLightResultSummary)" />
	<Error Condition="'$(StatLightSucceeded)' != 'True'" Text="Unit Tests Failed" />
</Target>

 

The FormattedSummary would look something like:

 

StatLight complete -- Of 7 tests: 6 passed, 1 failed, 0 skipped, took 0.461 seconds

Test Failed: Some.Tests.ThisTestWillFail
		Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assert.Fail failed. I wanted this test to FAIL!!
		   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.HandleFail(String assertionName, String message, Object[] parameters)
		   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail(String message, Object[] parameters)
		   at Microsoft.VisualStudio.TestTools.UnitTesting.Assert.Fail(String message)
		   at Some.Tests.ThisTestWillFail()

 

I want to add the names of the passed tests to the FormattedSummary.

Is this useful? What would you do different?

Coordinator
Mar 10, 2010 at 1:40 AM

Ugg - sorry I forgot to respond to this. After I wrote a whole reply a week or more ago - it failed to post. I guess I forgot to re-respond...

I've done a ton of work in the trunk to make working against the XmlReport a little cleaner and easier. However, I don't know if this is the best way to integrate. Ideally it would be great for the MSBuild task to work directly with StatLight.Core.

Basically all you need to do is call the StatLightRunnerFactory.CreateOnetimeConsoleRunner(.***NOTE A..) - this will return an IRunner.

 

    public interface IRunner : IDisposable
	{
		TestReport Run();
	}
***NOTE A: There's a whole slue of parameters to give the StatLightRunnerFactory - You could take a look at how the program.cs in the console project gets thing setup to call this factory. (NOTE don't get scared by Program.cs - it's way over complicated as it support many different scenarios).

Then you can work with directly with the TestReport to get all the necessary results.

If you need "realtime" results you can look how the TeamCityTestResultHandler is implemented (this gives realtime results when running under teamcity). If this is a must, let me know. I know of at least one change I would have to make to StatLight.Core to allow you to do this.

Hope this helps.