Current project status – HSCALE
A lot of people keep asking me about the status of HSCALE so I thought it is best to just write about it.
Since I am waiting for the next GPL release of MySQL Proxy I concentrate on other things, both project-related and not project-related.
Continuous integration and test strategy enhancements
First of all there is a CI server up for almost 3 months now. Check out teamcity.hscale.org (Login as guest user). I also introduced a lint-process to discover bugs in my LUA code which arise mostly by mistyping variable names. By the way: You should definitely have a lint-like process in place if you are doing LUA programming or other languages of that type. It helps a lot.
What’s in svn trunk?
The current status is this: Spreading across multiple backends is fully implemented and there are a lot of tests for that. Check out the latest code at http://svn.hscale.org/trunk. The tests running against multiple backends are “cheating” a little bit: Since the proxy still does not allow for ad-hoc allocation of backend connections I just create a lot of “dummy” connections to the proxy and use the approach of connection pooling the way the proxy does it. See an example. Note: This will not work in production!
Dictionary partition lookup and auto-partitioning
Besides the modulus partition lookup, which is only used in tests, there is now the dictionary partition lookup which allows for explicit partition definition. Detailed documentation can be found in the project wiki. It works as described here.
Another feature implemented is auto-partitioning. Depending on the partitioning function used it is possible to create new partitions on the fly. This way you can automatically spread the load across your MySQL servers. Another benefit is that you have a fine grained partition set-up right from the start which makes re-partitioning a lot easier afterwards.
Please take a look at the code and especially the tests to see how it works.
Parallel setup of HSCALE servers
In order to eliminate the SPOF (single point of failure) and possible bottleneck that directing all traffic through a single proxy would imply, HSCALE is designed to run in parallel. This means you can set up multiple proxies running HSCALE in parallel. This way you can easily implement fail-over scenarios using heartbeat or your favourite high availability solution and spread the load.
Why should running HSCALE in parallel be a problem in the first place? The easy answer: Because partition information is cached within each instance. As soon as the partition information changes your HSCALE instances would run out of sync using different partition information which would be disastrous to your data integrity. To avoid this HSCALE (more precise the dictionary partition lookup) works in two different modes: “NORMAL” and “FORCE”. If the mode is set to “NORMAL” then partition information is cached internally and only refreshed in a configurable time interval (configuration parameter “reloadInterval”). Whenever a change is made to the partition set up the mode is changed to “FORCE” which forces all HSCALE instances to re-fetch the partition information prior executing any query. After a configurable amount of time the system switches back to “NORMAL”. This is only the big picture. The implementation itself is a bit more complicated.
This approach is simple and robust because no other components are involved (like message queues) and it is guaranteed by design that no HSCALE instance is able to run with a wrong partition mapping. Changes made to the partition information involve a little overhead since all HSCALE instances will reload the partition mapping quite often. But taken into account that the partition information does not change that often (100 times a day would be huge!) it is an affordable price to pay for data integrity.
Currently HSCALE is almost feature complete. The thing that’s missing – and making HSCALE production ready – is a MySQL Proxy version with different backend handling (Jan, please do not hate me!). Currently we (our company) do not have the resources digging into MySQL Proxy ourself and HSCALE is currently(!) not top-priority. So we will just wait and see.
Part of the problems we intended to solve with HSCALE are now moved to a Hadoop cluster since we have huge masses of log-style, read-only data which has to be analysed in multiple dimensions. HSCALE will gain focus right after that or as soon as a “suitable” GPL version of MySQL Proxy comes out.
If you are in need for a production ready, proxy-based sharding solution, please take a look at Spock Proxy. They use a different approach – they actually forked MySQL Proxy and implemented everything into it, thus no LUA is used – but the idea behind it is basically the same. They also offer some features HSCALE will not offer in the near future like handling of auto_increment columns across partitions. Some features are not there mostly because of the different design approach like arbitrary partitioning functions (they only offer range-based partitioning which is ok for many scenarios) or query hinting.
Speaking of Hadoop cluster – an idea that is ringing in my head for a while is to implement a MySQL Proxy LUA script that enables running (basic) queries against a cluster. It would be a little fun project. 😉