Wednesday, June 18, 2014

StructureMap: hot swap cache

In the previous post I've shown how to cache the objects in StructureMap for a given period of time. As I mentioned in that post, there is one possibly serious downside of the approach presented - the penalty of cache rebuilding that kicks one unlucky user every caching period. If it takes more than several seconds for the cached object to be built, we probably don't want this to happen in-process, unless we're showing our users something like XKCD strips while waiting.

Ideally, we would be rebuilding our cache in some kind of off-process mechanism and when it's ready, just replacing the old cache object with the fresh one - like disk hot swapping. Is it also possible with StructureMap? Probably not with lifecycles - lifecycles does not control object creation, they just provide proper cache.

What we can do instead is to pre-build the cache object and inject it into the working container. But we can't use the container to prepare that cache object for us this time - the container will happily fulfill our request with the previously cached object. Although delegating the object creation process is actually one of the purposes we use IoC containers for, I can't see any neat way to delegate the responsibility for objects creation for the whole application lifetime except the cache pre-building.

So I've chosen the less neat way. I've created a cache factory that just news the cache up manually, while being itself created by StructureMap. That way, whenever the application asks for IDependency, it gets the cached instance quickly. But when the cache rebuilding task runs, it grabs DependencyFactory and creates a new object, a future cache.

Let's see the code. First, here is a base class for all the cache factories - CacheFactory. It smells like a conforming container a bit, but I find it not really harmful. It is not intended to be used in any context other than cache pre-building and it is specialized to create a single type of objects. Cache consumers should not know about it and just take ICache dependency through the constructor injection or any other legitimate way.

public abstract class CacheFactory
{
    public abstract object InternalBuild();
    public abstract Type PluginType { get; }
}

public abstract class CacheFactory<T> : CacheFactory
{
    public T Build()
    {
        return (T)InternalBuild();
    }

    public override Type PluginType
    {
        get { return typeof(T); }
    }
}

The non-generic class is the core here. It defines a method responsible for returning the actual cache instance. The generic class is just to keep the API nice and have the possibility to define strongly-typed constraints.

The second brick in the puzzle is the code that handles the actual cache hot swap. It spawns a new thread that wakes up every 600 seconds and traverses all the CacheFactories registered in the container, creating new cache instances and injecting it into the working container. This way up until the Inject call, StructureMap serves all the requests with the previously cached instance and the Inject call gets the new object, ready to be used without any further delays.

public class BackgroundCacheRefresher
{
    private readonly IContainer _container;
    private readonly ILog _log;

    public BackgroundCacheRefresher(IContainer container, ILog log)
    {
        _container = container;
        _log = log;
    }

    private class Worker
    {
        private readonly IContainer _container;
        private readonly IEnumerable<CacheFactory> _cacheFactories;
        private readonly ILog _log;

        public Worker(IContainer container, IEnumerable<CacheFactory> cacheFactories, ILog log)
        {
            _container = container;
            _cacheFactories = cacheFactories;
            _log = log;
        }

        public void RefreshAll()
        {
            foreach (var cacheFactory in _cacheFactories)
            {
                try
                {
                    _container.Inject(cacheFactory.PluginType, cacheFactory.InternalBuild());
                    _log.InfoFormat("Replaced instance of '{0}'.", cacheFactory.PluginType.Name);
                }
                catch (Exception e)
                {
                    _log.Error(String.Format("Failed to replace instance of '{0}' due to exception,"
                        + " will continue to use previously cached instance.", 
                        cacheFactory.PluginType.Name), e);
                }
            }
        }
    }

    private void RunLoop()
    {
        while (true)
        {
            var lifetime = 600; // seconds
            _log.InfoFormat("Will now go to sleep for {0} s.", lifetime);
            Thread.Sleep(TimeSpan.FromSeconds(lifetime));

            _log.Info("Woke up, starting refresh cycle.");
            _container.GetInstance<Worker>().RefreshAll();
        }
    }

    public void Execute()
    {
        new Thread(RunLoop).Start();
    }
}

I'm creating BackgroundCacheRefresher and calling its Execute method at the application startup. It starts with sleeping - the first cache is build "traditionally", as registered below.

Now we just need to wire things up in the Registry. I've created an extension method for the cache registration to make it clean and encapsulated. It registers both the cache object (as a singleton, to keep it in memory, but we'll replace it periodically with the code above) and its corresponding CacheFactory implementation.

public static class RegistryExtensions
{
    public static CacheBuilderDSL<T> UseHotSwapCache<T>(this CreatePluginFamilyExpression<T> expression)
    {
        return new CacheBuilderDSL<T>(expression);
    }

    public class CacheBuilderDSL<T>
    {
        private readonly CreatePluginFamilyExpression<T> _expression;

        public CacheBuilderDSL(CreatePluginFamilyExpression<T> expression)
        {
            _expression = expression;
        }

        public SmartInstance<TConcrete, T> With<TConcrete, TFactory>(Registry registry)
            where TConcrete : T
            where TFactory : CacheFactory<T>
        {
            registry.For<CacheFactory>().Use<TFactory>();
            return _expression.Singleton().Use<TConcrete>();
        }
    }
}

And here is how to use it:

For<IDependency>().UseHotSwapCache().With<ExpensiveDependency, ExpensiveDependencyFactory>(this);

The last thing is the factory - just newing up the cache object. Note that its dependencies can be provided in the typical, constructor-injected way.

public class ExpensiveDependencyFactory : CacheFactory<IDependency>
{
    private readonly IDependencyDependency _loader;

    public ExpensiveDependencyFactory(IDependencyDependency otherDependency)
    {
        _otherDependency = otherDependency;
    }

    public override object InternalBuild()
    {
        return new ExpensiveDependency(_otherDependency);
    }
}

Whoa, a bit of code here. Maybe there is something simpler available - if so, drop me a line, please! Otherwise, feel free to use it.

Tuesday, June 17, 2014

StructureMap: time expiring objects cache

StructureMap is my favorite .NET's IoC container. It has a very nice API and is quite well extensible. One of the things I use its extensibility points for is to have my expensive objects cached for some time. Not a singleton, as the cached values are changing from time to time and I want to see those changes eventually. Also not a transient nor per-request instance, as filling the cache is expensive - let's say it's a web service call that takes several seconds to complete. There is no such object lifecycle provided by StructureMap. Let's fix it!

What I need is a custom lifecycle object, so that I can configure my dependencies almost as usual - instead of for example:

For<IDependency>().HybridHttpOrThreadLocalScoped()
    .Use<NotSoExpensiveDependency>();

I'll use my own lifecycle using more generic LifecycleIs DSL method:

For<IDependency>().LifecycleIs(new TimeExpiringLifecycle(secondsToExpire: 600))
    .Use<DependencyFromWebService>();

LifecycleIs expects me to pass ILifecycle implementation in. That interface is responsible for keeping a cache for the objects. Its responsibility is to decide where that cache is and how long does it live. In our case, all we need to do is to use "singleton-like" cache (MainObjectCache) and make sure it is invalidated after a given period of time. Easy as that!

This is how it looks like for StructureMap 2.6 family:

public class TimeExpiringLifecycle : ILifecycle
{
    private readonly long _secondsToExpire;
    private readonly IObjectCache _cache = new MainObjectCache();

    private DateTime _lastExpired;

    public TimeExpiringLifecycle(long secondsToExpire)
    {
        _secondsToExpire = secondsToExpire;
        Expire();
    }

    private void Expire()
    {
        _lastExpired = DateTime.Now;
        _cache.DisposeAndClear();
    }

    public void EjectAll()
    {
        _cache.DisposeAndClear();
    }

    public IObjectCache FindCache()
    {
        if (DateTime.Now.AddSeconds(-_secondsToExpire) >= _lastExpired)
            Expire();

        return _cache;
    }

    public string Scope
    {
        get { return GetType().Name; }
    }
}

And here is the same for StructureMap 3.0 (there were some breaking names changes etc.)

>public class TimeExpiringLifecycle : ILifecycle
{
    private readonly long _secondsToExpire;
    private readonly IObjectCache _cache = new LifecycleObjectCache();

    private DateTime _lastExpired;

    public TimeExpiringLifecycle(long secondsToExpire)
    {
        _secondsToExpire = secondsToExpire;
        _cache.DisposeAndClear();
    }

    private void Expire()
    {
        _lastExpired = DateTime.Now;
        _cache.DisposeAndClear();
    }

    public void EjectAll(ILifecycleContext context)
    {
        _cache.DisposeAndClear();
    }

    public IObjectCache FindCache(ILifecycleContext context)
    {
        if (DateTime.Now.AddSeconds(-_secondsToExpire) >= _lastExpired)
            Expire();

        return _cache;
    }

    public string Description
    {
        get 
        {
            return "Lifecycle for StructureMap that keeps the objects for the period of given seconds."; 
        }
    }
}

StructureMap is responsible for reading and writing the cache, constructing the objects etc. - we don't need to care about that stuff at all. The only thing we should remember is that although all the requests within 600 seconds will be served with the cached object, after that time one of the requests will finally encounter a cache miss and will need to create that expensive cache, bearing the cost within that request.