Throwing a 404 like StackOverflow's, without redirect, when using IOCControllerFactory

I found another question posted on here which provided a solution to the first part of my question, i.e. how to generate a 404 page in ASP.NET MVC WITHOUT redirecting to another page (so the originally requested url remains in the address bar).

ASP.NET MVC - How to throw a 404 page similar to that on StackOverflow

protected void Application_Error(object sender, EventArgs e)
    {
        var exception = Server.GetLastError();

        Response.Clear();
        var httpException = exception as HttpException;
        var routeData = new RouteData();
        routeData.Values.Add("controller", "Error");

        if (httpException != null)
        {
            routeData.Values.Add("action", httpException.GetHttpCode() == 404 ? "NotFound" : "Unknown");

            // clear the error, otherwise, we will always get the default error page.
            Server.ClearError();

            // call the controller with the route
            IController errorController = new ErrorController();
            errorController.Execute(new RequestContext(new HttpContextWrapper(Context), routeData));

        }
    }

The 404 side of things works fine for me if I don't use an IOCControllerFactory, but as soon as I do use a factory, the error code in global.asax isn't used and instead I get the following error:

The IControllerFactory 'MyNamespace.IOCControllerFactory' did not return a controller for the name 'blah'.

How do I get around this, without resorting to getting rid of my controllerfactory, and creating a parameterless constructor for each of my controllers?