Whoa, Randy has a technology post about technology, coding and testing? Yep, I do. And if you’re neither tester nor developer, don’t bother reading the rest of this – you’ll just get pissed at me for wasting your time with techno-babble.

It seems that whenever I post something technical like this it’s as much for my own reference (and remembrance) as it is to share the information. Keeping that in mind, if you want some background on this, keep reading – if you don’t, jump down a few paragraphs.

When I first joined Microsoft I took a testing gig with MSDN, specifically with the In Product team. They’re responsible for testing the installation, uninstall, and overall function of the Document Explorer application. You know Document Explorer [DExplore]. It’s the application delivering in-product help for all versions of Visual Studio. And monthly for TechNet. And for MSDN Quarterly library. Yeah, you can say its got a good many eyes on it. Anyway, when I joined the team they already had a set of automated tests that would run with every build of DExplore, which was focused on Whidbey Beta 1 at that time. Well, it sorta ran. It ran often enough but it wasn’t as seamless as people would like it to be. My first task was to stabilize the automation harness. As part of that, I changed the lab machine, the imaging technology that was used for each runs, upgraded the scripts from VB 6 to VB.NET, and changed the testing framework to use some newer libraries. Over the last two years I moved it to C#, made some of my own additions to the shared framework, tripled the lab in size, and expanded the coverage base of OS’s. I also spent ten months working with NUnit and monitoring a bunch of tests that were activing working with web services.

Just one man’s tasks in a larger world. The point of it all is that all of the work above was .NET work. .NET is handy for stuff like this. The entire set of automation classes that I was using was written in three different languages; because it was .NET it was very, very easy to use and extend. And clean up behind you with memory. And write code quickly. And offered Reflection to get into objects as needed. Plus P/Invoke would allow access to the Win32 SDK when things got ugly. As always, I’m a firm believer in using the best tool for a job, be it .NET, C++, C#, OSX, iPod, blah blah blah.

Now I’m in Gaming. Within my team we have projects that span PC games and Xbox 360 games. Language of choice? Native C++. Whoo! It has been several years since I worked in Native Win32 C++. Sure I spent some time using MFC [as a class library] as recent as 2001… and I’ve opened the MultiPagez [C++] project from time to time… but it had been a while. My current projects involves things like COM and C++ libraries. The current automation testing tool is a console application that dumps out a bunch of text to the screen. Ideally, I would have liked to have an application that could tap hooks in my code, much like what I did with NUnit – since I can’t use Reflection, so that wasn’t an option. I took a look at CPPUnit, which lets you inject macros to partition your code, and that worked OK, but it was kinda light when compared to what I’m used to.

[[ Here’s the bit on Visual Studio Team System and Testing! ]]

What about VSTS? At Microsoft, it’s always good form to use or own products. Truth is that in most cases I’ll get a bunch of cooperative technical support for the product in question. I mean, if I find a problem with an internal tool, I ping the owner and get an active conversation going… that’s pretty good service.

The problem is that VSTS primarily focuses on work within the .NET Framework. Given the nature of the .NET Framework, I’d be able to ping the COM objects that I need using C#. It would be easy, in fact. The problem with that approach is that I don’t like it. If the target audience of this project is going to be using native C++, then so should the test. In fact, I can say with almost total assurance that the target audience would never be using this project in a .NET application. Just the nature of the business.

In most of the articles I have seen on Testing within VSTS, it’s focused on C# or VB. There’s some mention that you can use Managed C++ to write tests as well, but there’s many instances that state that you cannot use Native C++ for testing within VSTS. The good news is that that’s only partially true.

You can’t inject Unit Tests into production code, true, but you can use a combination of Native C++ code and Managed C++ code, in the same test, to test native code within VSTS. The basis for this is an MSDN Article: Unit Tests and C++, specifically the section on Mixed mode tests. Simply put, open your Project options, click on Configuration Properties -> General and change Common Language Runtime support to “Common Language Runtime Support (/clr)”. This will allow a mixture of Native C++ and Managed C++ in your test cases.

And that’s about it. From my point of view, you write up your test cases using C++ as if it wasn’t managed; the defaults of a new Test Project set up the [TestMethod] Attribute – you just tuck a C++ method under that. In fact, you can mix and match your code within those as well. For example, I’m using straight C++ to get at the COM object but I’m using managed classes for spitting out my own log file… little weird for me because I hadn’t heard of gcnew before this week, but once you get used to String ^ and not having to call delete for these managed objects, it’s a piece of a cake. You get little islands of Native C++ throughout a Managed C++ file.

The benefit of all of this is that this is all integrated into the VS IDE and I’ll be able to leverage VSTF[oundation Server] as the entire group starts to use the same methods of testing. In theory, anyone will be able to run the BVT’s and the tests that I’m working on… not to mention all of the built in logging, reporting, and stuff.

Seriously, this is some neat shit, and I’m pretty impressed by it. I just have to remember where that pesky CLR option is when I create a new project.

Of course, that’s the main reason for this post in the first place!