About RedState Caching

Regular readers of Red State 3 feature discussions have noticed a common thread in many of my responses to them:

  • Are comments not appearing right away? Caching.
  • Kowalski button gone? Caching.
  • New comment displays not available? Caching.
  • Comments posted under other people’s names? Caching.
  • Recommend this diary button missing/moved to another page? Caching.
  • My Diary link goes to someone else’s diary? Caching.

So what is this caching, what’s it for, and do we really need it?I will answer the questions in reverse order:

Yes, we really do need caching. As it turns out, the Red State 3 software is slow. Remember on the first day when the site was up, after the morning was fine it got very, very difficult to browse? It got slow because we got complaints about the caching, and turned it off for diary viewing. As a results, there were constant 500 errors and the like because the software running the site is just so slow that the web server was giving up on it.

Obviously that just won’t do. So the good people at Eagle Publishing have spent a great amount of time since then trying to make the cache reader-friendly, and the result is the way the site works right now. We all can get in and post.

But what is the caching? It’s not complicated. Our caching software breaks up all Red State readers into three groups. Let’s call them Contributors, Commenters, and Lurkers. Contributors are all of us who get special buttons to help keep the site going. Commenters are the readers who are logged in and able to comment. Lurkers are anyone not logged in.

The software then keeps a copy of each page for each group of users saved, and just sends that copy if another user in that group asks for the same page. So if I open this diary, and three seconds later Erick views this diary, he will get the same page I viewed. Likewise, if Kowalski reads this diary, and a second later Gamecock pulls it up, Gamecock will get Kowalski’s copy of the diary. The same holds for any two people who read the site but are not logged in.

We only actually go and ask our site software to generate a new copy of the page if the ‘cached’ copy is older than a set amount of time (right now we cache diaries for 5 seconds for anyone logged in).

This works great, and keeps the site running, but only if there’s nothing on the page that’s tied to a particular user. This means diary recommendations, Reply to This buttons, My Diary links, the new comment form, and everything else had to be altered to work properly in this cached environment. Most of my Red State work over the last week has been to help make the site cacheable.

That leaves the New Comments tracking. Well, consider what would happen with caching. If Kowalski views a page, and Gamecock gets his copy, then Gamecock would get marked as NEW the comments that Kowalski hasn’t read yet. That means New Comments tracking isn’t very useful with caching enabled, but if we disable caching the site is inaccessible. Scylla and Charybdis come to mind.

And now you know why things work on a starship Red State.