Integration with Team Build?

Sep 8, 2010 at 4:03 PM

We have a build server which runs tests nightly and reports back the results. I am trying to integrate StatLight with it Team Build. 

The problem I'm having is getting the tests to run, however, since the team build process runs as a service and the whole "Interact With Desktop" feature of services appear to have been almost completely deprecated in Server 2008 (I'm seeing this mentioned all over the Internet, and I haven't been able to get it to work right). 

What is the most up-to-date recommended practice when integrating with Team Build? Are there any recommended practices? 

Thanks in advance, 

Ben

Coordinator
Sep 8, 2010 at 8:43 PM

TeamBuild and TeamCity will both have the same problem.

You will have to figure out how to start the team build service in interactive mode.

This may help. http://social.msdn.microsoft.com/Forums/en-US/tfsbuild/thread/07ca106b-1499-422a-9781-7aec0c73558c

If it does help would love a step-by-step how to, that we could add to the wiki.

Hope this helps.

 

Sep 9, 2010 at 3:16 AM
Edited Sep 9, 2010 at 5:10 AM

Yeah, I'm actually of the mindset now that this is not the correct way to go: tying into the build engine so as to launch StatLight.exe after the build completes as the whole situation seems wrought with disaster. 

For me, at least, the best solution thus far has been to copy the XAP output of my Silverlight Unit Tests project to a destination which StatLight.exe is watching (in continuous mode). I modified the code so that it will raise a Tested event, and from the event handler I send an email with the results (I presume this model could be extended to other reporting systems - posting to a URL, etc.). As it stands, no "TestResult" or whatever it's called, objects are actually used when the thing is in continuous mode (it seems), as continuous mode appears to be used more for TTD than build server integration. 

At any rate, what I have thus far is working with Team Build, even though it still requires an active desktop session. 

As a corollary, I'm not sure why it's so difficult to run Silverlight Unit Tests which have no GUI requirements in a windowless environment (I'm not attacking this product, just the entire Silverlight framework altogether). 

Coordinator
Sep 9, 2010 at 2:42 PM

In response to "...difficult to run Silverlight Unit Tests which have no GUI requirements in a windowless environment..."...

I completely agree. In fact the Microsoft.Silverlight.Testing framework _did_ work in this environment without issues prior to the framework's U.I. moving completely to Silverlight. Once the framework updated from an HTML/Javascript U.I. to a full silverlight U.I. all this became a requirement.

I've toyed with the idea of writing a headless version that had no U.I. (specifically for continuous integration) - just don't currently have the time.

 

Sep 9, 2010 at 3:31 PM

I would be interested in learning how you would plan to make a "headless version".

Coordinator
Sep 9, 2010 at 3:51 PM

Here's the easy way. But highly dependent on Microsoft. Their framework provide an option to not create/use any of it's U.I. and just run the tests. (Basically giving us what we had before they upgraded to the Silverlight U.I. - excluding the HTML/Javascript U.I.) This is 100% dependent on Microsoft. But would be the easiest - as others are already leveraging the MS test framework.

The hard way - essentially replicate all the code build into the MS testing framework (excluding the U.I. portions).

Jan 25, 2011 at 6:45 PM

For TFS 2010, you can try the following solution : http://pyxis-tech.com/blog/2011/01/24/how-to-run-silverlight-automated-tests-on-tfs-build-server/

It explain how to combine StatLight and a TFS 2010 custom activity to break your continuous integration build when a test fail in your solution. 

Feb 17, 2011 at 12:09 PM

i wrote a activity to merge the results back in tfs like real unit tests. It's not perfect yet (no test summery for example) but for a first version it is really usefulll.

you can download it here: http://statlightteambuild.codeplex.com/

Coordinator
Feb 17, 2011 at 7:42 PM
Edited Feb 17, 2011 at 7:42 PM

dsmits - I don't use TFS build so this is great that you've taken the initiative to extend StatLight in this manner. Let me know if we can consider changes to StatLight to make this easier. I have some thoughts, but this isn't the thread to discuss them on.

Feb 17, 2011 at 8:52 PM

Thanks staxmanada. at the moment i'm not missing anything for the integration. í'm really curios about your thoughts, maybe you can email me? dave@familie-smits.com

Feb 18, 2011 at 2:24 PM

Would be really good to have a way to run headless. We're having problems running the tests via TeamCity.  We can't set the build service to run under Local Account/Interaction mode as we do other things (deployments..) which require the build service user to have network permissions.  I've googled about turning on Interaction mode for a domain user but couldn't see any conclusive steps to do this.

Coordinator
Feb 18, 2011 at 8:09 PM

The best option we have currently is to setup your C.I. machine to auto-logon as the necessary domain account user. (you can use http://live.sysinternals.com/ AutoLogon.exe)

Then in a startup task launch the build agent as a command line tool.

That's how we're currently doing it.

Hope this helps.

Feb 19, 2011 at 11:47 PM

Thanks Staxmanade.  The only thing is that I work for a large corporate and am not sure that I will be able to do this without getting grief from the teams that manage the machines/security etc.. :(  You mentioned earlier that the SL Framework provided a means to turn off the UI.  Do you know where abouts in the the code that would be?

Coordinator
Feb 20, 2011 at 7:13 AM

Before the test framework was converted to a silverlight based U.I. It was "painted" with html and javascript - we didn't have these issues with the U.I. - However if you wanted to run any U.I. specific Silverlight tests, you would still have to run under the interactive mode. So it might be worth investigating what or how or if it's possible to turn off the U.I. all together. This might allow you to run your non/ui tests in the service without requiring interactivity mode. I haven't tried to do this so I can't say for sure how this would be done.

Coordinator
Feb 20, 2011 at 7:53 PM

Teaser... I might have found a possible work around for the "headless" approach. Not sure yet if it will work, having to re-install a VM to get my test environment up again. May be a few days before I know for sure, but I'll keep everyone up to date of the progress.

Feb 20, 2011 at 11:04 PM

That would be fantastic.  I'll keep my fingers crossed :)  

BTW for anyone interested I have implemented a build runner for Team City that allow you to configure StatLight via UI.  Will post the details once I've tidied up a couple of things.

Coordinator
Feb 23, 2011 at 5:20 AM

I'm currently baffled... I created a V.M. with win7 installed Silverlight and the latest TeamCity (after which I had to go to the agent service and turn off desktop interactivity). I then set it up to run a test with StatLight. In my case I'm using StatLight to test this project https://github.com/staxmanade/StatLight/tree/master/src/StatLight.IntegrationTests.Silverlight.LotsOfTests

However I can't seem to get StatLight 1.3 to fail to run. It keeps working (which is not what I expected...)

Regardless - I'd be curious if anyone could grab the following, build StatLight and let me know if helps with their build server not working without the desktop interactivity setup?

https://github.com/staxmanade/StatLight/commit/7ac70b6baa84f2e3fe36c5ab058dc0c427dfeca8

Until then I'll spend some time and try to get a build failing...

Feb 23, 2011 at 12:10 PM

Woohoo. That seems to make things better for me.  It is now running all my tests using a domain account user when not in Interactive mode.  Thanks StaxManade

Coordinator
Feb 24, 2011 at 1:14 AM

Based on that working for you I've pulled that into master...

Feb 24, 2011 at 1:58 PM

This sounds awesome. We definitely want to use StatLight, and want it to be in "headless" mode. I was concerned when I pulled down the code and I kept seeing too much interaction with the desktop in the samples. I thought I'd do a little more research. It looks like I didn't give the dialog boxes long enough to close down. 

I'm looking forward to using this product and watching it get better. :D