Setting and deleting cache in Django with tags

The Django cache system is real help when you need to keep your site as fast as possible. It offers multiple back-ends and is super easy to get started using. I just wish it had the ability to tag content when you are setting it in the cache. If you could do that then you could do bulk deleting of content based on tags. One use-case that comes up again and again for me is caching paginated results. If you have blog-entry1-page1 and blog-entry1-page2 both stored in the cache and want to delete them when a new entry is added what do you do? It’s hard if not impossible you know which paged set is currently in cache and we need to know the keys in order to delete them.

I searched and searched for a solution to this problem with not much help from google. I did find this post by Eric Florenzano which is close to the approach I took, but I wanted something more generic. So I decided to add two new methods cache.set_with_tags and cache.delete_by_tags to whatever cache backend you are using. Just to note this is only useful if you intend to utilize the low-level cache framework.

First we need to import some modules:

from django.core.cache import cache as cache_backend
from django.core.cache.backends.base import BaseCache

Make sure to alias cache because we will be passing the alias to our new methods and assigning the alias to the local cache variable.

Now its time to create the method that sets the cache as well as tags the item:

def cache_set_with_tags(key, value, tags=[], timeout=None):
    for tag in tags:
        tag_list = cache_backend.get(tag)
        if tag_list:
            tag_list.append(key)
        else:
            tag_list = [key]
        cache_backend.set(tag, tag_list, timeout)
    cache_backend.set(key, value, timeout)

Basically what this does is loop over the tags and checks if the tag is already in cache. If the tag is already in the cache we append the new key to the tagged list, otherwise we create a new tagged list and add the key. Then we just save the tagged list back to the cache. Finally we save the value to the cache like we would normally do.

The deleting method is similar to setting:

def cache_delete_by_tag(tags=[]):
    for tag in tags:
        tag_list = cache_backend.get(tag)
        if tag_list:
           for key in tag_list:
               cache_backend.delete(key)
        cache_backend,delete(tag)

All this does is loop over the tags checks for the tag in cache. If found it loops over the list of keys and deletes them, then deletes the tag.

The last thing you need to do is hook these methods to your cache backend:

cache_backend.set_with_tags = cache_set_with_tags
cache_backend.delete_by_tags = cache_delete_by_tags
cache = cache_backend

Save this module something like cache_helpers and make sure its on your PYTHONPATH.

Here is an example use:

>>> from cache_helpers import cache   
>>> data1 = 'Some Data'
>>> print cache.get('data1')
<<< None
>>> cache.set_with_tags('data1', data1, ['data'])
>>> print cache.get('data1')
<<< Some Data
>>> data2 = 'Some More Data'
>>> print cache.get('data2')
<<< None
>>> cache.set_with_tags('data2', data2, ['data'])
>>> print cache.get('data2')
<<< Some More Data
>>> cache.delete_by_tags(['data'])
>>> print cache.get('data1')
<<< None
>>> print cache.get('data2')
<<< None

I am fairly new to Django and Python so help me out if there is a better way to do any of this, also this has only been tested on this blog. This method might not be the most efficient if you have a lot of items tagged and you have to loop over each one to delete. Also this is probably not the best for high traffic sites were users are adding and deleting content from the cache frequently. However for a simple blog like this it makes managing the cache much simpler. I do think this is a pretty common use-case so maybe someone much smarter then will be able to crack it.