Typemock Isolator 4.2 released

Just a short announcement to tell you that Typemock 4.2 was just released out of beta. Download it here.
To make this short, you can find what's new and release notes here:
http://www.typemock.com/Docs/UserGuide/index.php?topic=ReleaseNotes42.html

 

Some of the cool features added:

• Highlight mocked methods within the VS Debugger so you know why you're not stepping into them
• Ability to evaluate during debugging the value of mocked methods and objects without ruining the test state
• Ability to mock direct fields which are not value types (properties were already supported)
• Events declared on abstract classes can now be mocked
• Events declared in derived interfaces can now be mocked
• Extension method on MSCorlib classes can now be mocked inside a chained statement
• A generic API for creating mocks is now available in the Community edition
• The new release of DotTrace profiler (3.0) is now supported and can be linked to run with Typemock Isolator
• more..

Your comments are appreciated!

Chains, Part I: Chain-Challenged

Chains are difficult to grasp, to develop and to understand. And some of them are even hidden, and therefore are not so easy to identify. So, what are they and what have they done to deserve a couple of blog post? Apparently it's a complicated feature to master from both the developer and user perspective. So I thought I'll clear the topic, top to bottom, until I, I mean WE, understand it better.

What are chains?

We use the term "chain" to describe a set of chained calls, where the return value of one method is the target where the next method is invoked on. And since describing it in words is to confusing enough, consider this simple chain:

int x = a.b().c();

As any compiler knows, this is equivalent to:

SomeClass temp1 = a.b();
int x = temp1.c();

And chains can be nested, as can be seen in the following simple example:

int x = a.b().c(d.e(f.g().h(2,i.j()));

Chains in Natural Mocks

Setting an expectation on a chain, originally meant setting it on the last call. Basically we don't care about the rest of the chains, and setting expectations on the beginning or middle doesn't matter for the test. But it's a bit more complicated. One of the new features in Typemock Isolator 4.2 clears a semantic issue. The Repeat and RepeatAlways methods now refer to the entire chain. For example:

using (RecordExpectations rec = RecorderManager.StartRecording())
{
 int x = a.b().c();
 rec.Return(5).RepeatAlways();
}

int t = a.b().c();  //returns 5
int s = new SomeClass().c(); // is not mocked, and returns its original value

That's a break from the former behavior (pre 4.2), when the expectation was set only on the c() method. This change, by the way, came from feedback we received from the community, and we were happy to oblige.

The Missing LINQ

How's LINQ connected to chains? (yes, some pun IS intended). Well, when the compiler parses LINQ queries, it creates a long chain of calls, full of expression, expression trees and other vegetation, which compile to chains and nested chains. So when someone tries to mock a LINQ query, they get a lot of confusing error messages. The error comes from the current implementation which handles nested chains sub-optimally. More on this later.

The Next Chain Chapter

So we are working on a fix for the LINQ problem. My next post will be about the how Isolator interprets chains, the pitfalls, and how we are progressing on the path to resolution. Which, by they way is full of fun.

Ad hoc Unit Tests with Snippet Compiler (Typemock Isolator Enabled)

If you're a fan of snippet compiler (if you're not you should seriously check it out)Travis Illig published a little template for writing Typemock Isolator test inside this handy little tool.

the reasons you'd need a specialized template in the first place to write these kinds of tests in snippet compiler:

1) Typemock Isolator uses the .NET profiling APIs to work its magic, so the .net process running your tests need to have a couple of environment variables enabled to work

2) His code template actually creates and runs a new process that triggers nunit-console.exe with the path of the current code you just wrote in snippet compiler allowing you to effecively write and run unit tests in snippet compiler!

3) the nunit-console process will already have the env. vars as mentioned in the first item set to it.

Travis' template will work for anything nunit can run, not just typemock isolator tests, which is pretty cool.

Full info and download here.

Video: Typemock Basics 2: Decorators

I'm happy to share with you a new video of the Typemock Isolator Basics Series - this time it is about a feature called "Decorators" - which allows you to extend the current testing framework you work with with custom attributes, regardless if you are working with NUnit MbUnit or otherwise.
In the movie I show how to write a custom attribute for [Repeat(times])] on a test, to be repeated X amount of times by creating an attribute that inherits from Typemock.DecoratorAttribute.




You can download the full wmv from here, and it will be available along with our other Typemock videos on our Multimedia page next week

Signing MSTests Private Accessors Assembly

Recently one of our users has stumbled upon the need to sign the generated private accessors assembly which is automatically created by the MSTest framework.

It all started when they tried to delay sign the production assembly. However, in that assembly they used the InternalVisibleTo in order to expose stuff for the Test assembly so they needed to sign that assembly as well. It didn't end there. The test assembly had the same attribute exposing things for the private accessors assembly which is created by the testing framework so they needed to sign that as well. Here is when things became a little problematic. the generated accessors assembly has no means for the user to change its signing.

To cut the story short, they eventually found the Publicize.exe tool developed by Microsoft that enabled them to solve this issue.

A warning - apparently the beta version has some serious issues with it, so for those of you which have yet to upgrade to the 2008 RTM, make sure that youre using the RTM version of the tool.

When Tests Fight Back

In the last couple of days, we were working on new API which rely on the registry for data storage and retrieval. Ohad and I paired for this development task.

We used TDD in this task. And we used Isolator for mocking the registry calls. Basically, we wrote unit tests for interactions - we set expectations on the registry calls in the test code, and then wrote the code that filled those expectations. We used Isolator's CheckArguments API to make sure that we were writing and reading the correct keys. And finally we called MockManager.Verify to verify the calls were made correctly, with the correct arguments.

We were actually doing what classic unit testing does - abstracting the environment, and focusing on the inside. Tests passed as wrote them, and due to the nature of tests (the expectations we set were detailed enough so we can write the code correctly) we didn't spend too much time on debugging. (Yay TDD!)

For a latter API we could not abstract the registry calls any longer, and so we wrote integration tests for it, using the registry API. And following that,  wrote examples for the new API, which uncovered a few bugs that the unit tests hid. When we fixed the bugs, the unit tests we wrote failed, and we had to fix those also.

We were discussing today what we can learn from this. Here are a couple of ideas:

• We were focusing on unit tests, while not having enough integration tests.
• We didn't check for all the arguments, and amazingly, these were the ones where the bug was.
• On the other hand, our tests are very specific already, and fixing them was our "reward". Maybe with less details, we would spend less time on their maintenance.

Unit tests are great, but they are not enough. Integration tests should accompany and test the "real" environment. And the tools we use (in this case, mocking with Isolator) should help us in writing simple tests. Detailed tests are ok, as long as you are ready to pay the price.

 

Did you ever get lost in that magical zone between integration and unit tests?

Typemock Community Edition *Is* Free

Jacob is right - We didn't make it clear: The community Edition of Typemock is totally free to use (as well as on open source projects), but when you download it you get an automatic demo mode of the full enterprise edition. after 21 days only the Community edition features remain active.

Here's the comparison chart between the current Typemock Versions.

Sorry!

Use Blocks with your mocks

It seems to me that not to many users of Typemock Isolator knows about the feature called expectation block.
This feature can be handy when you find that you are duplicating the same mocks setup across few tests with maybe some minor changes.
Before I start just note that this is an enterprise feature.

Here is an example:

  class ClassToMock
 {
     public void A()
     {}

     public int B()
     {
         return 1;
     }

     public string C()
     {
         return string.Empty;
     }
 }

 [TestFixture]
 public class TestClass
 {     
     private  Mock SetUpExpectation()
     {
         Mock mock = MockManager.Mock<ClassToMock>();
      
         mock.StartBlock("ClassToMockBlock", VerifyMode.Normal);
         mock.ExpectCall("A");
         mock.ExpectAndReturn("B", 5);
         mock.EndBlock();

         mock.ExpectAndReturn("C", "Isolator");
         return mock;
     }

     [Test]
     public void TestMethod1()
     {
         // Get the block with expectation for A() B() and C()
         Mock mock = SetUpExpectation();

         ClassToMock c = new ClassToMock();
         c.A();
         Assert.AreEqual(5, c.B());
         Assert.AreEqual("Isolator", c.C());

         MockManager.Verify();
     }

     [Test]
     public void TestMethod2()
     {        
         Mock mock = SetUpExpectation();
         // Everything inside the block is cleared
         // This leaves us with one expectation for method C()
         // We set the expectation for C() after the block ends.
         mock.Clear(mock.GetBlock("ClassToMockBlock"));

         ClassToMock c = new ClassToMock();
         Assert.AreEqual("Isolator", c.C());
         MockManager.Verify();
     }

     [Test]
     public void TestMethod3()
     {
         Mock mock = SetUpExpectation();
         mock.StartBlockBefore(mock.GetBlock("ClassToMockBlock"));
         // Add one more expectation for method A()
         // So we have expectation for A() A() B() and C()
         mock.ExpectCall("A");
         mock.EndBlock();

         ClassToMock c = new ClassToMock();
         c.A();
         Assert.AreEqual(5, c.B());
         Assert.AreEqual("Isolator", c.C());
         c.A();

         MockManager.Verify();
     }
 }    

Unlike most real test ClassToMock is used directly in the test methods for sake of clarity. It has three methods A() B() and C()

The interesting code starts at line 20 at SetUpExpectation method.
Here we give the block a name and set the block expectations.
Note that there is one call to C() outside the block, This means that any action
We will do on the block will not affect this expectation.

Now lets see how we can use the block in our tests.
In TestMethod1 I simply use the block 'as is' the mock that returned from SetUpExpectation method in line 20 has expectations for A() and B() inside the block and for method C() outside the block. That means that the actions we will do on the block will not affect method C()
You can see this in TestMethod2 we clear all expectation on the block but since C() is outside the block so we left only with expectation on C().
In TestMethod3 we use mock.StartBlockBefore() method to add expectation for method A() to our block. That means we'll have two expectation for method A().

That's the basic actions for mock Blocks. Please note that you can use more than one block for the same mock so you can do do 'more advanced' tricks with it.

Automocking and Making Life Easier

Jeremy Miller wrote about Automocking put into his StructureMap tool for dependency injection. To quote him on why he did not use Typemock he writes:

"If you're using TypeMock you probably don't care about dependency injection and automocking anyway"

This post is not about if I care about dependency injection (which is a good tool in my opinion, by the way). It is about how a tool in a framework makes life easier.

When you read Jeremy's post, see how concise the test code becomes, once the Automocking feature is implemented in the framework. The tool makes all the boiler-plate setup code go away. Which of course, let's the developer concentrate on the really cool stuff, like finding bugs :).

We write tools for developers to make them find bugs... er, develop with ease. Just like Automocking. And, I'm always excited to learn new ways to do that. It gives us more ideas we can implement, and make life easier. Maybe Automocking will not be a Typemock Isolator feature. But we can build on this idea for other features.

After all, that's our goal here: Simplify unit testing. It says so right there on the logo.

Decorator feature enhancement

Typemock-Isolator has a little know Decorator feature that allows developers to simply add capabilities to their test framework (all test framework).

Using this feature Typemock-Isolator creates the [VerifyMocks] decorator that automatically verifies the mocks at the end of your tests.

To create a custom decorator simply extend the DecoratorAttribute and implement the Execute method.

public override object Execute()

{

    // Trace start of method

    Console.WriteLine("Entering Method:"+

     OriginalMethod.ToString());

    // Execute the original method

    return base.CallDecoratedMethod();

}

To support multiple decorators on the same test method, Typemock-Isolator makes sure that even if a developer forgot to call CallDecoratedMethod() the test method will still run. This is done by calling the decorated method if it wasn't explicitly called.

We discovered that some Decorator implementers want more control and need to explicitly make sure that the test method was NOT called.

A new API was added in version 4.2 to do just that.

DontCallDecoratedMethod()

Once this is called (inside Execute), the decorator mechanism will not call the test method.

Of course if there is another decorator on the test method that calls CallDecoratedMethod() the test method will be executed.

Typemock Isolator 4.2.1 Beta Is Out

The updated 4.2.1 Beta is now on the Typemock site. Anyone who downloaded the 4.2.0 Beta, should uninstall it and install the 4.2.1 version. The unpatched version contains a bug that crashed the Configurator tool when it loads.

As Typemock's dev team is growing, we are feeling our way in the agile world. We are in a continuous learning mode. And the story of this patch is something we discussed today, in order to improve.

The bug was introduced in the latest stages towards Sunday's release. In order to save time, we began testing on an earlier version, which didn't have the bug in it. On Sunday, when we decided that it's time to release, we built a new installer, tested it quickly, and uploaded it to the website. Obviously, we didn't catch the bug then.

What could have we done better? For one thing, prior to release, we could test more expansively. We would probably have found it. But let's dig in more: we build our installer from our source control system. How did the rogue code get into the source control system in the first place?

Deadlines. The more you get closer, you let things go. At this critical point, we didn't review the code prior to check-in. Of course pressure is an excuse, and everyone was busy doing their things, but if we didn't let our guard down, we would have caught this prior before the bug crawled in.

Agile is about responsible behavior. And we should stick to our responsibilities especially in times of pressure. More so in times of pressure. We should have caught this one earlier. Best lessons are learned the hard way.

Creation of Mocks - On the difference between Mock and MockObject

One of the most confusing points when dealing with Typemock Isolator is the creation of Mock Controllers (the term Mock controller refers to the object which is used to define the behavior of the mocked object during the test). When creating these controller there are two sets of API one can use:

1. MockManager.Mock
2. MockManager.MockObject

The difference between the two is quite simple: to

Using the Mock API tells the framework to attach the created Mock Controller to the next instance that will be created (in the future). This means that the next time a new call will be issued for the given type, a Mock Controller will be attached to the created instance which will cause all calls on that instance to be mocked.

Using the MockObject API tells the framework to create a Mock Controller and an instance of that type and attach them to one another. The real instance that is mocked is created immediately and it can be accessed through the MockObject.Object property.

So when to use which?

The Mock API is used when the creating of the instance we wish to mock is done internally in the tested class and we have no convenient way of accessing it.

The MockObject API is used when the instance we want to mock is created outside the tested class and is passed to it. The nice thing about the MockObject API is that unlike the Mock API it can handle types that usually cant be instantiated such as interfaces & Abstract classes

Another related issue which usually goes hand in hand with the usage of these API's is the handling of Unexpected Calls (we refer to calls which have no expectations set on them as Unexpected Calls).

When using the Mock API we always mock concrete classes, for which all methods have real implementation. By default when mocking these types all Unexpected Calls will be directed to the real code. In most cases this is the intent of the user.

When using the MockObject API, more often we mock Interfaces/Abstract classes. And since for these not all methods have real implementation, by default issuing an Unexpected Calls will result in an exception being thrown. From our experience Unexpected Calls in these cases indicates a logic flaw in most cases.

(There are some cases in which the MockObject API is used to mock concrete classes, in these scenarios the behavior is the same as if using the Mock API since the actual default is determined by the type of the class being mocked and not by the API itself)