Welcome! This blog post, and the code to which it refers are still very much a work in progress. Please let me know if you have any questions about the content, or problems with the code. Thanks, Mike.
Update: I should really note that this code is not production ready. You’re going to take a serious performance hit if you try to dump your giant MySQL database into redis keys. Use with care.
In my spare time I’ve been working on a project with my nephew that is going to require some realtime data processing outside of my usual PHP/MySQL idiom. However, a good deal of the project still operates in the fashion of your typical web application. Without going into a lot of specifics, information entered into the web interface will be offloaded into redis and processed via worker jobs running elsewhere. Our issue was the idea of maintaining data in two separate places.
Now, I could have just written the web application to use redis exclusively, and that would probably have been fine. But I thought it might be more interesting to see how easily I could map the relational data to keys in redis. I’ve been using the FuelPHP framework on a number of projects recently, and I can say with complete honesty that I finally feel like I’m experiencing truly rapid application development. I can ramp up a project in hours instead of days — it’s been a pleasure to work with.
That being said, the Model implementation in Fuel, coupled with the built-in Redis class, made this task (heh) a lot simpler to accomplish than I first imagined.
FuelPHP is built with rapid application development in mind. Generating controllers, models and even full-blown admin interfaces can be done in almost no time. Models, when generated with Fuel’s built-in `oil` task, specify their properties in a protected, static parameter called, not surprisingly, $_properties.
Relationships, a must-have feature that I’ve found easier here than in any other framework or ORM library that I’ve used, are defined in a similar way, with static properties following an easy naming scheme – $_belongs_to, $_has_one, $_has_many, and the ever-popular but oft confusing $_many_many. More information Fuel’s ORM layer can be found in the FuelPHP Docs.
Redis is obviously not a relational database, but I was able to find some correlations that worked, at least for my purposes. How well they’ll work in the wild is for you to tell me. Seriously, lemme get some feedback, because if there’s a better way to do it, I’m all ears.
Siphon works on the idea that we can use redis Hashes to store object representations, and Sets to store the relationships. Let’s assume that we’re talking relationships based on the primary key, because that is all that’s being accounted for at the moment.
My work here has been heavily influenced by a number of fantastic resources that have been covering this sort of thing far longer than I. Articles by Phil Sturgeon and Justin Carmony have been especially helpful. If you’re interested in more about either Fuel or redis, take some time to browse the links below.
- Build an Admin Panel with the Fuel PHP Framework by Phil Sturgeon
- MySQL, Redis and a Billion Rows – A Love Story by Justin Carmony
- Blazing Data with Redis by Justin Carmony
- An Explanation of Redis Data Types
So, for the first time in my (almost) 15-year career as a professional developer, I’m releasing some code I wrote into the wild. I sincerely hope that someone else finds this helpful. If you have any questions, comments or diatribes that you’d like to share, please feel free to do so below. I’d love any feedback that you would like to share — reporting issues would be really helpful too. And if you’d like to contribute, feel free to Fork Siphon on Github. Thanks for reading!