XAML binding tests fail when run via StatLight

Jun 23, 2011 at 11:37 AM

I have the following test:

    [TestMethod]
    [Description("Tests that the active CheckBox is databound correctly")]
    [Tag("Location")]
    [Asynchronous]
    public void TestActiveCheckBox()
    {
        var activeCheckBox = GetUIElement<CheckBox>("chkActive");
 
        EnqueueCallback(() => TestPanel.Children.Add(TargetView));
        EnqueueCallback(() => { _viewModel.Active = true; });
        EnqueueCallback(() => Assert.AreEqual(activeCheckBox.IsChecked, _viewModel.Active, "Failed to data-bind active flag."));
        EnqueueCallback(() => { activeCheckBox.IsChecked = false; });
        EnqueueCallback(() => Assert.AreEqual(activeCheckBox.IsChecked, _viewModel.Active, "Failed to update active flag."));
        EnqueueTestComplete();
    }

that's testing the binding between the check box and the variable in the view model. The view's DataContext is set to the ViewModel and the view code is:

    <CheckBox Grid.Column="4" x:Name="chkActive" IsChecked="{Binding Active, Mode=TwoWay}"
              Content="Active" />

and there is a boolean property on the ViewModel that implements the "INotifyPropertyChanged" interface correctly.

This works fine when the tests are run interactively, but when I checked the code in and the tests are run on the build machine via StatLight they fail at the first "Assert.AreEqual". It appears that the binding hasn't actually been set up.

I originally had my tests organised as per the examples here on Jeremy Likness's blog:

http://csharperimage.jeremylikness.com/2010/08/unit-testing-xaml-data-bindings-in.html

But I was finding that not all the tests were running all the time. The tests would all pass green, but when I checked via the debugger I could see that on one run tests 1, 2 and 4 fired and the next time tests 1, 3 and 4 will fire. Therefore we couldn't rely on them to verify that the code was correct.

Further investigation led me to Sergey Barskity's blog:

http://dotnetspeak.com/index.php/2010/07/unit-testing-silverlight-applications-ui-testing-and-test-automation/

The only problem with this was that I found that unless I wrap the "TestPanel.Children.Add(TargetView)" call in an "EnqueueCallBack" wrapper I can't guarantee that the binding has been setup before performing the actual test.

So, how to I either a) get StatLight to really instantiate the view correctly or b) force the binding without the view being loaded?

Coordinator
Jun 23, 2011 at 2:43 PM

Assuming you're using the most recent version of StatLight (1.4 at the time of writing this).

Are you running it with the "-b" option? If you don't you should see something like "Looks like your trying to use the Silverlight Test Framework's TestPanel. To use this you will need to use the -b option in StatLight"

Now, running with -b means you have to have your C.I. server's agent process running with desktop interactivity enabled. (Check statlight documentation to see how to configure this).

Have you successfully run StatLight locally? I was able to copy Jeremy Likness's example and see that it definitely doesn't work without the "-b" option, but when I gave it the "-b" it did work.

Hope this helps.

Jun 23, 2011 at 5:02 PM
Edited Jun 23, 2011 at 5:48 PM

Thanks - that seems to have done the trick.

However, there are three new problems.

Firstly, we're running the tests from TeamCity and when I tried running StatLight with both "-b" and "--teamcity" it didn't run any of the tests at all.

Secondly, when running with "-b" on it's own random tests fail each time. I'm experimenting with adding a delay after the "TargetView" is added to the TestPanel, but I don't think that this is a long term solution.

Thirdly, the unit test framework reports that there are 144 tests rather than the 72 that there actually are.

Coordinator
Jun 23, 2011 at 6:23 PM
Edited Jun 23, 2011 at 6:23 PM

Hello,

"Firstly, we're running the tests from TeamCity and when I tried running StatLight with both "-b" and "--teamcity" it didn't run any of the tests at all."
Is that it didn't _run_ any of the tests, or it didn't report any tests run? Could you look at the teamcity log and see what it's reported there? (Copy here, or email me the results)?

"Secondly, when running with "-b" on it's own random tests fail each time. I'm experimenting with adding a delay after the "TargetView" is added to the TestPanel, but I don't think that this is a long term solution."
Not sure I have anything for you here at this point. If you could create a sample test project that you could share that reproduced this I could take a look.

"Thirdly, the unit test framework reports that there are 144 tests rather than the 72 that there actually are."
This is a known issue with the way StatLight configures the Microsoft Silverlight Testing framework. (Just ignore the total count in the UI). I have reported this issue, and have not dug deep in to Microsoft Silverlight Testing enough to figure it out myself...

Hope this helps.

Jun 24, 2011 at 9:37 AM
Edited Jun 27, 2011 at 11:56 AM

On the "-b" and "--teamcity" apparent mismatch this is the output:

StatLight - Version 1.4.4147Copyright (C) 2009 Jason JarrettAll Rights Reserved.
##teamcity[testSuiteStarted name='...\ClientBin\SLUnitTests.xap']

--- Completed Test Run at: 23/06/2011 17:23:20. Total Run Time: 00:00:03.5340000
Test run results: Total 0, Successful 0, Failed 0, ##teamcity[testSuiteFinished name='...\ClientBin\SLUnitTests.xap']

I haven't got TeamCity installed on this machine, but I understood the "--teamcity" flag to mean that it just changes the output so it matches the format TeamCity can understand. If I run the tests with just the "--teamcity" option I get the following output:

##teamcity[testStarted name='SLUnitTests.WorkingTest' captureStandardOutput='true']
##teamcity[testFinished name='SLUnitTests.WorkingTest' duration='20']
Looks like your trying to use the Silverlight Test Framework's TestPanel. To use this you will need to use the -b option in StatLight
Looks like your trying to use the Silverlight Test Framework's TestPanel. To use this you will need to use the -b option in StatLight
Looks like your trying to use the Silverlight Test Framework's TestPanel. To use this you will need to use the -b option in StatLight
##teamcity[testStarted name='SLUnitTests.FailingTest' captureStandardOutput='true']
##teamcity[testFailed name='SLUnitTests.FailingTest' message=' ...']
##teamcity[testFinished name='SLUnitTests.FailingTest' duration='51']

So it looks like I don't need TeamCity locally for this.

Though there is a typo in the message you have "Looks like your trying..." when it should be "Looks like you're trying..." or even "Looks like you are trying...".

On the random failures:

I think this is a a timing issue. I was having problems using the "Loaded" mechanism (from the Jeremy Likness blog) when running the tests interactively and moving to the EnqueueCallback approach made that consistent. So I think there's something else I can do here. It could well be down to the web browser. The interactive tests are running in IE while StatLight has it's own. Perhaps I should try running in Firefox or Chrome and seeing if that behaves differently.

Adding "EnqueueDelay(10)" after the call to add the view to the TestPanel seems to cure the problem - but obviously it's not ideal.

Thanks for your help on this.

Jun 29, 2011 at 10:22 AM

I've just found this work item - http://statlight.codeplex.com/workitem/13654 - which describes the problem exactly.

Do you have any information on when this might be resolved?

Coordinator
Jul 2, 2011 at 4:46 AM

Should be fixed in the latest build (go ahead and pull from the StatLight teamcity server)

Jul 4, 2011 at 1:01 PM

It works a treat now.

Thanks again for your assistance with this.