Julian Jelfs’ Blog

WatiN Dialog Handler for unhandled javascript exceptions

Posted in Unit Testing, WatiN by julianjelfs on September 27, 2011

I’m not sure what WatiN does with javascript exception alerts by default, but I was finding that it seemed to silently close any exception dialog and the test would carry on and would possibly still pass. In my case, I would like the test to fail if there is a javscript error and I would like it to report the text of the javascript error as the reason for the test failure.

First I tried to do this using the NativeMethods supplied with WatiN but I wasn’t able to get at the text description of the error which was important for reporting the reason for test failure. So I then resorted to the UI Automation API found in C:\Program Files\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\UIAutomationClient.dll and came up with the following code:

using System.Windows.Automation;
using WatiN.Core.DialogHandlers;
using WatiN.Core.Native.Windows;

namespace WatiNExtensions
{
    public class ScriptExceptionDialogHandler : BaseDialogHandler
    {
        private readonly ITestFailureLogger _logger;

        readonly AndCondition _documentCondition = new AndCondition(new PropertyCondition(AutomationElement.IsEnabledProperty, true),
                                                                    new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Document));

        readonly AndCondition _buttonConditions = new AndCondition(new PropertyCondition(AutomationElement.IsEnabledProperty, true),
                                                                   new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Button));


        public ScriptExceptionDialogHandler(ITestFailureLogger logger)
        {
            _logger = logger;
        }

        public override bool HandleDialog(Window window)
        {
            if (CanHandleDialog(window))
            {
                var win = AutomationElement.FromHandle(window.Hwnd);
                var documents = win.FindAll(TreeScope.Children, _documentCondition);
                var buttons = win.FindAll(TreeScope.Children, _buttonConditions);

                foreach (AutomationElement document in documents)
                {
                    var textPattern = document.GetCurrentPattern(TextPattern.Pattern) as TextPattern;
                    var text = textPattern.DocumentRange.GetText(-1);
                    _logger.Log(string.Format("Unhandled javascript exception: {0}", text));
                }

                foreach (AutomationElement button in buttons)
                {
                    if(button.Current.AutomationId == "7")
                    {
                        var invokePattern = button.GetCurrentPattern(InvokePattern.Pattern) as InvokePattern;
                        invokePattern.Invoke();
                        break;
                    }
                }
                return true;
            }
            return false;
        }

        public override bool CanHandleDialog(Window window)
        {
            return window.StyleInHex == "94C808CC";
        }
    }
}

This accepts an ITestFailureLogger interface in the constructor, finds the text of the error and logs it, then clicks on the “No” button to dismiss the dialog. What the ITestFailureLogger does is up to you. In my case it just tracks the errors and then checks them in the NUnit test’s tear down. You cannot throw an exception in the dialog handler to stop the test in its tracks because any exception thrown by IDialogHandler.HandleDialog will be swallowed by the WatiN DialogWatcher (understandable but a little bit irritating).

Hope this is of some use to someone.

Tagged with: ,

Why FakeItEasy beats Rhino.Mocks

Posted in Unit Testing by julianjelfs on March 30, 2011

We all know that when we’re unit testing we should use mocks and stubs to test our component’s interactions. Like many people I have been using Rhino mocks for a long long time. The introduction of the AAA syntax a while ago has made things considerably easier, but I have still always found that as the complexity of the code under test increases, then the burden on the tests in terms of set up also increases. So too does the likelihood that a supposedly targeted unit test will be broken by some orthogonal code change.

This problem is exacerbated when people don’t understand the difference between mocks and stubs and also when people try to test too many things in a single test. The problem is that when you confuse mocks and stubs you almost inevitably end up testing more than one thing because you end up setting up a whole host of expectations which have little to do with your test just to get the thing to run. This leads to brittle, annoying tests which makes people stop wanting to write them.

The great thing about FakeItEasy is that everything you create is just a Fake and it will behave like a stub or a mock depending on how you interact with it. This makes it much much easier to write a targeted unit test against a complicated piece of code without getting swamped in set up code. It also makes it much less likely that you will use a mock when you need a stub and much less likely that your test will break when something unrelated changes.

If you consider the example of trying to fake interaction with the NHibernate criteria API. Even very simple criteria queries are a pain to mock because of the nature of the API. You might say that this is too low level and that NHibernate should be behind some sort of abstraction interface. Maybe, but the NHibernate criteria API is an abstraction interface and in this case it is precisely the exact calls being made to the API that I want to test.

Let’s say I have the following simple code (assume that the NHibernate session is injected):

   1: public IEnumerable<Thing> GetThings(string filter)
   2: {
   3:     var criteria = _session.CreateCriteria(typeof(Thing));

   4:

   5:     if (!string.IsNullOrEmpty(filter))

   6:         criteria.Add(Restrictions.InsensitiveLike("Property", string.Format("%{0}%", filter)));

   7:

   8:     return criteria.List<Thing>();

   9: }

Let’s say that what I want to test is that when a filter is supplied, the correct restriction is added to the query. With Rhino mocks I might try something like:

   1: [Test]

   2: public void FilterIsApplied()

   3: {

   4:     var criteria = MockRepository.GenerateMock<ICriteria>();

   5:     var session = MockRepository.GenerateStub<ISession>();

   6:     session.Stub(s => s.CreateCriteria(typeof (Thing)))

   7:         .Repeat.Once().Return(criteria);

   8:

   9:     criteria.Expect(c => c.Add(Restrictions.InsensitiveLike("Property", "%filter%"))).Repeat.Once();

  10:

  11:     var controller = new ThingController(session);

  12:

  13:     var result = controller.GetThings("filter");

  14:

  15:     criteria.VerifyAllExpectations();

  16: }

The main problem with this is that it doesn’t work. The reason it doesn’t work is because Restrictions.InsensitiveLike returns a new Criterion i.e. not the same Criterion that we have specified in our expectation and therefore the arguments don’t match. Our only option is to set IgnoreArguments. But in this case, that is the beginning and end of what we want to test, so if we IgnoreArguments we might as well delete the test.

Another problem is that, even if this test did work, it would be brittle. Suppose someone changed the code as follows:

   1: public IEnumerable<Thing> GetThing(string filter)

   2: {

   3:     var criteria = _session.CreateCriteria(typeof(Thing))

   4:         .AddOrder(Order.Desc("Property"));

   5:

   6:     if (!string.IsNullOrEmpty(filter))

   7:         criteria.Add(Restrictions.InsensitiveLike("Property", string.Format("%{0}%", filter)));

   8:

   9:     return criteria.List<Thing>();

  10: }

To add an order by clause. This has nothing to do with our test case, but it would break our test. We can solve both of these problems with FakeItEasy as follows:

   1: [Test]

   2: public void FilterIsApplied()

   3: {

   4:     var session = A.Fake<ISession>();

   5:     var criteria = A.Fake<ICriteria>();

   6:     Any.CallTo(session).WithReturnType<ICriteria>().Returns(criteria);

   7:     Any.CallTo(criteria).WithReturnType<ICriteria>().Returns(criteria);

   8:

   9:     var controller = new ThingController(session);

  10:

  11:     var result = controller.GetThings("filter");

  12:

  13:     A.CallTo(() => criteria.Add(

  14:         A<ICriterion>.That.Matches(c => c.ToString() == Restrictions.InsensitiveLike("Property", "%filter%").ToString()).Argument))

  15:         .MustHaveHappened(Repeated.AtLeast.Once);

  16:

  17: }

Line 7 means that the test does not break when the order by criteria is added – it shouldn’t and doesn’t care about it. But the really nice thing is the way we express our expectation. We do not have to resort to IgnoreArguments (though there is an analogous option if you need it) because we have the ability to supply a predicate to determine whether each individual argument matches our expectations. We can also choose to ignore individual arguments which will further reduce the probability of the test breaking for the wrong reasons. So what we are left with is an easy to write, targeted resilient unit test.

PS – don’t get too hung up on whether or not this is a test you should be writing. It’s just a test that demonstrates nicely why I think FakeItEasy is better than Rhino mocks.

Tagged with: