Digg Top Stories -> RSS

Please note that all blog posts before 8 April 2007 were automatically imported from LiveJournal.  To see the comments and any LiveJournal-specific extras such as polls and user icons, please find the source posting at http://brianenigma.livejournal.com/2006/07/

A few months ago, Digg released a new version of their site. The release added a bunch of new stuff, but killed off a feature I had been using. They have RSS feeds of all sorts of things–the front page, your friends, “live” feeds of search terms, but their recent release killed off the one RSS feed I used the most. The problem with their main (front page or individual container) feeds is that they are giant and very active. If I leave my computer sitting over the weekend (or even all day), it picks up new stories from the feed every hour and caches them until I return. I then end up with 200+ stories to sort through. Most of those stories were crap, as “front page”!=”quality story.” I would often just ignore the whole group of 200+ stories, marking them all as read. This makes that particular feed useless to me. The top stories feed was great because it would change a little over the day as various stories fought for top billing. Returning to a computer that had been idle for a day would bring up 20-30 stories–a good, manageable amount, considering I knew most of them would be fairly high quality. A few weeks ago, I put in a feature request asking for the top stories feed back. After all, The page is still there, but the RSS link points to the main RSS feed, so it shouldn't be too difficult, right? Today, I got fed up of waiting for them to fix it and wrote my own script to give me that page as an RSS feed. It's a simple shell script (with embedded XSL), requiring curl, tidy, and xsltproc. A wrapper shell script (basically “diggtoday.sh > blah.xml”) can be dropped in /etc/cron.hourly to make things easier.

Posted in: Code

Leave a Reply

Your email address will not be published. Required fields are marked *