StatLight running tests in multiple .xap files

Nov 24, 2010 at 8:03 PM

I use StatLight to run unit tests in RIA Services solutions.

I have a VS solution containing 2 .xap files with unit tests.

In TeamCity, if I follow the article "Running Silverlight unit tests in TeamCity using StatLight", I have to create one build configuration for each .xap file.

Is there a way to run StatLight in a unique build configuration that runs tests contained in more that one .xap file?

For example, by adding multiple -x options, or by searching for all xap files in a solution/folder, or by creating a build script designed to run StatLight on multiple .xap file at once in order to merge unit tests statistics.

If this is not possible, please add it as a must-have feature.

Any suggestion? Thanks

Coordinator
Nov 24, 2010 at 11:48 PM

As of V1.3 you can pass multiple -x={xap file} parameters. Let me know how it goes for you.

Nov 25, 2010 at 1:53 PM

Hello,

I just tried the new release 1.3.

Yes it is possible to pass multiple -x="xap file" parameters now.

However, I noticed a strange problem, that seems a bug.

My command parameters in the Teamcity build runner are:

-x="%system.teamcity.build.checkoutDir%\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Shell.Test.xap" -x="%system.teamcity.build.checkoutDir%\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap" --teamcity

I have 69 tests passed and 3 failed in the x.Shell.Test.xap (first one) and 8 tests passed in the x.Data.Test.xap (the second one).

The result with release 1.3 is: 85 tests passed and 3 failed, instead of 77 passed and 3 failed. It seems the second xap is counted twice (8 passed x2= 16 + 69 passed = 85)

To be sure I changed the order, and I could reproduce the same problem (85 passed x2 = 170 + 8 passed = 178).

 

Looking at the log file in TeamCity, 8 tests passed are reported, but each test is displayed twice.

Here the end of the log:

[14:36:29]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Shell.Test.xap] Test run results: Total 72, Successful 69, Failed 3,
[14:36:29]: C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap (1s)
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckBirthdayCantBeInFutureTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckBirthdayCantBeInFutureTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckBirthdayCorrectTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckBirthdayCorrectTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckMaxCharTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckMaxCharTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckValueCantBeNullOrEmptyEmptyTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckValueCantBeNullOrEmptyEmptyTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckValueCantBeNullOrEmptyNullTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckValueCantBeNullOrEmptyNullTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckValueCantBeNullOrEmptyTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.CheckValueCantBeNullOrEmptyTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.IsEMailBadFormatTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.IsEMailBadFormatTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.IsEMailGoodFormatTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Polyright.PersonManager.Client.Data.Test.ValidationHelperTest.IsEMailGoodFormatTest.
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] --- Completed Test Run at: 25.11.2010 14:36:30. Total Run Time: 00:00:01.1093750
[14:36:30]: [C:\Phoenix\trunk\Polyright.PersonManager\PersonManager\Polyright.PersonManager.Server.Web\ClientBin\Polyright.PersonManager.Client.Data.Test.xap] Test run results: Total 8, Successful 8, Failed 0, 

Hope this info will help.

 

 

Coordinator
Nov 25, 2010 at 6:18 PM
Edited Nov 25, 2010 at 6:35 PM

Thanks for the report. I've reproduced the issue and will get it fixed. Re-download the v1.3 release - I've updated with a patch. Let me know how that does for you.