How to debug

Nov 19, 2010 at 8:40 PM

I just downloaded statlight and am evaluating it as a solution to automate running SL unit tests.  Anyway, I've run into a couple of issues I was hoping someone could help with.

First, the number of unit tests reported by the tool is always double the number that are actually present in my XAP.  No big deal there.  Just an oddity. 

Second, I have some tests that make server calls and thus require the [Asynchronous] attribute and associated plumbing.  These tests register for notification of an event to signal that a response has been received from the server.  The test unregisters the event and calls EnqueueTestComplete() when the associated handler is invoked.  These tests succeed (pass) every time when run from VS 2010 but report random failures when run through statlight.  Output in the report file leads me to believe that sometimes event handlers aren't being unregistered correctly.  Unfortunately, I don't see what would cause this.  Plus, as I mentioned, the tests work as expected every time when run from VS.  For the most part, the tests follow a pattern similar to the following:

[TestMethod]
[Asynchronous]
[Description("Tests an expected normal (pass) case ...")]
public void LoadPreferences()
 {
    BusinessServicesTestHarness.RunTest(_servicesFactory, _parameters, (harness) =>
    {
         IList<string> prefs = new List<string>()
         {
            "prefA",
            "prefB",
            "prefC",
            "prefD",
            "prefE"
         };

        BusinessServiceCallEventHandler handler = null;
        handler = new BusinessServiceCallEventHandler((args) =>
        {
            harness.BusinessServiceCallCompleted -= handler;
            harness.Dispose();
            Assert.AreEqual(5, args.ResponsePacket.Preferences.Count);
            EnqueueTestComplete();
         });
        harness.BusinessServiceCallCompleted += handler;
        harness.LoadPreferences(prefs, TcPreferenceScope.All);
     });
}
To investigate the issue, I attempted to run statlight using the --webserveronly switch and attach the debugger by launching IE, 
copying the url into the address bar and invoking. When doing this, however, my breakpoints never get hit. Any suggestions
regarding how to properly attach the debugger so I can see what's going on?
 
Thanks for any assistance!
Coordinator
Nov 20, 2010 at 5:44 PM

Regarding the number reported in the Microsoft.Silverlight.Testing UI - it appears to be a bug in the Microsoft framework related to the setup that statlight uses to run the tests. This is only a visual issue, as StatLight should still report the correct number of tests. (Please let me know if you notice otherwise).

 

I'll give you the steps I took to debug a test and hopefully they help you.

  1. Created a brand new SL test project. (and compiled)
  2. Placed a breakpoint on the first line of the first test method.
  3. Ran - StatLight.exe "-x=.\SilverlightTest6\Bin\Debug\SilverlightTest6.xap" --webserveronly
  4. Copied the URL it gave and pasted in to I.E. (http://localhost:8887/GetHtmlTestPage)
  5. The tests should attempt to run initially (there's not really a way to do this otherwise) But either when it's done, or while it's going...
  6. Go back to Visual Studio and select from the menu "Debug->Attach to Process..."
  7. Find the I.E. instance that has "Silverlight, x86" in the "Type" column. Notice there may be two iexplorer.exe instances - one has a title called StatLight (but is NOT the one with the "Type" column set to Silverlight.
  8. After attaching to the Silverlight iexplorer process
  9. Refresh the browser.
  10. You should now hit your breakpoint.
  11. WARNING: There is a bug in the Microsoft silverlight test fx you can read about here http://statlight.codeplex.com/workitem/11906

Also keep in min that if your tests depend on outside resources (web services, etc) your test project needs to be configured to talk to an external service. If your code trys to talk back to "Home base" where that is the server it was loaded up from - it will be talking to statlight web server. 

Hope this helps.

Nov 22, 2010 at 10:57 PM

Thanks for the information.  I was able to connect and debug just fine.   I did have another (unrelated) question.  Is it possible to run OOB tests using statlight?

Coordinator
Nov 23, 2010 at 1:41 PM

Not immediately possible to run OOB tests with StatLight. I do think it might be possible with some effort. I have a feeling to do so StatLight would have to "install" the app before trying to kick off the tests, which would mean it should uninstall it when done. If you manage to figure out how to do this I'd love to hear about it.

Feb 10, 2011 at 2:32 AM

I created a workitem related to running the silverlight application elevated\oob during unit testing here:

http://statlight.codeplex.com/workitem/12983