So anyone following twitter's architecture history know about their message queue creations, specifically starling. Well like at least one other person, I thought it would be neat to port that to python. I looked around to see if anyone had done it first and I found peafowl, however that project was written using a threaded socket server (threads == bad for python). I decided to take the weekend and port it using the twisted framework. Since twisted already had a LineReceiver written, all I had to do was implement the memcache protocol and I was done.
I call the result mamba (a fast snake) and the code can be found on github. At the moment I would like to finish the unit tests so that I can make sure that I am not releasing total crap, but I also see a few things that would be handy to implement:
- fix the persistence to be pluggable and use a separate process to write to disk
- a pluggable web front end (perhaps with a comet statistics stream + jquery)
- an asynchronous client wrapper
I'm not sure if I will get to any of this, however, I didn't see a good twisted message server, so maybe someone will take a shot at it (especially if unladen-swallow can create comparable performance with some of the respected queues).
