Here is a preview of some use cases from the new Typemock Isolator API coming out sometime in August. I have some questions for you which I'd love for you to answer in the comments.
First, a little background on the new API that should come out sometime in august.
- The new Isolator API goes in the direction of AAA - Arrange,Act,Assert and backs away from the more traditional Record-Replay model.
- No mocks or stubs. Everything is defaulted to a fake object, which can either be used as a stub or as a mock object later on. If at the end of the test you call "Verify" on that object, that it is logically a mock. if you tell that object to return specific values or throw exceptions during the test, it is a stub. but the API does not mention these words for clarity.
- No strings (Except for non public members)
- There is a single point of entry to the API, currently called "MyTest" which is used to create all the types of fakes.
- There are several categories of faking which can be done, and they are represented in the MyTest as properties: "Make", "Static", "Swap","NonPublic".
- Make - Creates instances of fake objects to be used later on in the test. can be interfaces or real classes.
- Static - used for faking out static methods or constructors
- Swap: used for replacing objects that will be created in the future with fake objects
- NonPublic - used as the entry point for faking out anything that is not accessible via publicly accessible apis.
Let's see some use cases:
Use case #1: fake
A simple fake that will be used as a stub(all stubs are non strict by default - you can call any method that is void without getting an exception):
ILogger fakeLogger = MyTest.Fake.Instance<ILogger>(); //arrange
MyTest.WhenCalled(() => fake.Log("s")).WillReturn(false);
new SomeObject(fakeLogger).DoSomething();//act
Assert.IsTrue(....) //assert
Here is the same use case with the fake logger used as a mock to check if it was called:
#2 verify on fake:
ILogger fakeLogger = MyTest.Fake.Instance<ILogger>(); //arrange
new SomeObject(fakeLogger).DoSomething(); //act
//assert: next three are possible
MyTest.Verify.WasCalledWithAnyArguments(() => fake.Log("")); /// passed arguments are ignored
MyTest.Verify.WasCalledWithExactArguments(() => logger.Log("a")); ///matching according to passed arguments
MyTest.Verify.WasNotCalled(() => logger.Log(""));
The API above will also work with real types, not just interfaces. Here is an example of create a type that calls the original methods and only some are overridden (partial mocks):
#3 partial fakes:
RealLogger real = MyTest.Fake.Instance<RealLogger>(Members.CallOriginalMembers); //arrange
MyTest.WhenCalled(() => real.Log("s")).WillReturn(false);//all other methods work on the real object except this one
new SomeObject(real).DoSomething();
Here is how we handle static methods: per method, on all static methods, and on the static constructor
#4 - statics:
MyTest.Static.WhenCalled(() => Logger.Log("s")).WillReturn(false);
MyTest.Static.WhenCalled(() => Logger.Log("s")).IgnoreCall();
MyTest.Static.FakeStaticMethods<RealLogger>();
MyTest.Static.FakeStaticConstructor<RealLogger>(StaticFields.AreNull); //or StaticFields.InitializedToDefaultValues
Isolator also allows setting behavior on objects that will be created in the future. for example, in your code under test someone creates a new Logger and calls write on it. Also it allows taking over (sapping) on all current and future instances of a type.
#5 - Future objects
RealLogger fake = MyTest.Fake.Instance<RealLogger>();
MyTest.WhenCalled(() => fake.Log("s")).WillReturn(false)
MyTest.Swap.FutureInstance(fake).WhenCalled(() => new RealLogger("ctor arg"));
MyTest.Swap.AllCurrentAndFutureInstances<RealLogger>(fake);
Isolator is able fake privates easily, but requires strings to do this. We are not sure how it will look. Here are several possible ways. your input can help us decide.
#7 - Non public members
RealLogger fake = MyTest.Fake.Instance<RealLogger>();
MyTest.NonPublic.WhenCalled(logger,"SomePrivateMethod").WillThrow(...) //option 1. If you put the name of a public method in the string you get an exception
MyTest.StringBased.WhenCalled(logger,"SomePrivateMethod").WillThrow(...) //option 2 - any method name will work public or private.
Isolator is able take an instance of an existing object
#8 - Existing Instances (live objects)
RealLogger log = new RealLogger(...);
MyTest.Fake.ExistingInstance(log)
MyTest.WhenCalled(() => real.Log("s")).WillReturn(false);
There are the basics. In future posts I will dive a little deeper into more features (custom argument checking, custom callbacks and more).
Your thoughts and comments would be highly appreciated!
Question: What are our thoughts on the NonPublic dillemma?
- Also see Eli's post on this
- Should it throw is we send it something which is public and instruct user to use the strongly typed version of this? (can make tests more brittle if user refactors to make something from private to public)
- should we name it NonPublic or something else? ("StringBased" for example)






9 comments:
Is the old API still going to be available? I'd hate to have to rewrite hundreds (of not thousands) of unit tests.
I'm assuming "MyTest" is not the final name of the object that will be the entry point. It'd just be confusing if not.
The top of the post references "Make" but all of the tests show "Fake." Typo?
System.Reflection uses "NonPublic" - I'd keep "NonPublic" to be consistent.
I think it'd be OK to throw an exception if you specify a public method in a nonpublic fake setup. Again, from a consistency perspective, if you tell reflection to list out nonpublic members, you're not going to magically get public ones in the list. I'd make sure the exception message is very clear, though, and include some sort of action - "You tried to call public method 'Foo' as a nonpublic. Use 'MyTest.WhenCalled' instead of 'MyTest.NonPublic.WhenCalled'"
That said, I liked Eli's post where there was only "MyTest.WhenCalled" and no "MyTest.NonPublic.WhenCalled" - the addition of .NonPublic in there felt inconsistent, like I should have a corresponding MyTest.Public.WhenCalled... but then, does it really matter? (If you lose the NonPublic thing entirely, then you shouldn't throw any exceptions if the person accesses a public method using a string name.)
Eli's post also loses the .Static for static method calls. I also like that. I can see there might be some interesting challenges when you get to handling nonpublic statics, but the idea holds - do you need to differentiate? (Eli's post just shows the name of the nonpublic static method being called... but not the type it's attached to.)
Is there a way to mock calls to base class methods? In the current API we have CallBase. Is there an equivalent here?
I also like Eli's "Members.ReturnFakes" example. That HttpRequest thing is even the precise use case where I could see it being handy.
Of course, it may be that the only thing I need to fake is something nested in the request, like a particular cookie coming in. Right now, I can use the recorder API with a chained method call and just have it "work." Not sure how I'd do that here, but it looks like it might be a bit more cumbersome.
I can see this would be very powerful in a DI scenario. Particularly with a framework like Autofac where you can register dependencies using lambdas. Create your fake right in the dependency registration for your tests.
Interesting stuff. Going to have to think on this some more.
Thanks for the input Travis.
The old API will still be available.
I kinda liked MyTest, what would you prefer?
We will support Base Calls using OnBase
MyTest.WhenCalled(() => fake.Log()).OnBase().WillReturn("555");
We will still have chained methods.
So you can do:
// I don't care about HttpWebRequest
var fakeWebRequest = MyTest.Fake.Instance<HttpWebRequest>(Members.ReturnFakes);
//setup a specific scenario
MyTest.WhenCalled(() => fakeWebRequest.GetResponse().Cookies["Travis"]).WillReturn("fakeCookie");
"MyTest" just feels like it's something that you use when you're posting an example code snippet and you're talking about your unit test or test fixture. Sort of like "MyClass" or "Foo." Maybe have a "Typemock" object or an "Isolator" object - Isolator.WhenCalled(...). That way I'd know that I'm acting on a fake/stub/mock thing and not on the test or test fixture.
Love the chained method example. Also the OnBase(). Perfect.
In general I liked the record model of TypeMock, however it did cause problems with stubbing. You'd have to set VerifyMode to pass if not called, and then have to set it back for your expectations. It would be great if you could do another block in the test setup for stubbing and then have an expectation block for each individual test. Otherwise much of the stubbing has to be repeated.
In general I don't like having to always put: recorder.ExpectAndReturn(person.Name, "Bob").Always() or MyTest.Blah().Blah().Whenever()
around all my code, which is why I liked the recorder. It would be very nice if stubbing were made first class and separate from expectations and be able to control the default repeat
recorder.StubMode = StubMode.AlwaysReturn;
recorder.Return(person.Name)
Or, even better:
For a recording stub block, make property assignments really mean that they will return what's assigned, kind of like a list initializer in C#:
Person person = Recorder.CreateMock <Person>();
using (StubRecorder stubber = Recorder.CreateStubRecorder)
{
person.Name = "Bob";
person.Age = 3;
person.CalculateSomething();
stubber.Return(7);
}
The propeerty assignements in the above block specify the return values. For methods, the standard recorder.Return() would suffice.
That way the syntax is nice, and way less noise of having to put MyTest.Expect.Always() around everything.
+1 on renaming "MyTest" to something like "Isolator". Also, the syntax here is not exactly pretty - how about this:
Mock Creation:
var mock = Isolator.Mock<ICommand>();
Public Instance Method:
Isolator.Expect(() => mock.Execute()).Returns(2);
Static Public Method:
Isolator.Expect(() => MyService.StaticMethod()).Returns(2);
Non-Public Instance Method:
Isolator.Expect(() => mock.NonPublicMethod(“MyPrivateMethod”)).Returns(2);
Public Members:
Isolator.Expect(() => mock.Name).Returns(“Bob”);
Non-Public Members:
Isolator.Expect(() => mock.NonPublicMember("_name")).Returns(“Bob”);
you went overboard w/ the MyTest.Something.RealMethod, get rid of the Something for most things.
I don't get #8, I assume you mean log.Log not real.Log. That said, why do I have to "Fake" it first? That's a wasteful line.
why do I have a separate API for stubbing different things? Just make one that can stub everything. takes string or delegate
I don't like WhenCalled/WillReturn, tries to hard to be english and fails at that. Just call it Stub/Returns/Throws
MyTest is really bad, call it Isolator or something like everyone suggested
I also don’t like MyTest. I’m writing my test and having something called MyTest is just confusing. My first thought was also Isolator, but anything better than MyTest would be welcome.
I share Travis’ concern about the old API going away. I wouldn’t mind if the new API would be a new system and we could use both in the same test run, even if only one could be used in each test.
I’ve suggested in the past the use of expression trees in reflective mocks, but what I really like are the natural mocks.
I make extensive use of MSTest helpers to access non public members and it works fine with natural mocks most of the times. I only use reflective mocks when, for some reason, I can’t use natural mocks.
I would also like to be able to create derived classes and add attributes and implicit and explicit interface implementation:
Isolator.FakeDerived.Instance[ILogger]().Implments[ISerializable](InterfaceImplementatio.Explicit).WithAttribute(new SerializableAttribute());
What is the empty parens syntax being shown with in the new API methods?
That does seem too intuitive to me.
Also, I'd like to agree on the "MyTest" and say I like "Isolator".
e.g. MyTest.Static.WhenCalled( () => Logger.Log )
Tim,
The empty parentheses are Lambda notation for delegates.
What do you mean by Too intuitive?
Gil
Post a Comment