soma kAŠpek

hubs about JDBC Books to books WHERE orders.productName = books.name. I think implementing it as a MySQL Proxy LUA script or a JDBC driver would not make a big difference on the approach. I would prefer MySQL Proxy a lot though. …

MySQL Proxy is already very stable (partly due to the expertise Jan has gathered with developing lighttpd which architecture seems to be quite similar) and is maturing at a high speed in my opinion. I remember the time when you could have used a MySQL BETA version that would for sure much better than other products already claiming a “Service Pack” or two. These times seem to be gone for the server alone (but the quality is still very good even in early releases). I’m working with the proxy for a couple of weeks now and everything works like a charm.

At the end I don’t rely on the labeled version number or alpha status but on what our tests - especially unit-, concurrency-, load- and performance tests - say. And all looks good right now, we didn’t have a single problem related to the proxy.

As for what a manager would say: I’m in the comfortable (or uncomfortable if it goes terribly wrong) situation to be the executive “manager”.

cheers,
Marco

I agree with Ronald — help from someone who’s done this before would be worth its weight in gold. Having just walked a company through it for the first time, I can say there is a lot more to it than it seems like.

Archiving is also a good idea — but it needs to be both/and, not either/or. Maatkit has a handy archiving tool in it.

as said above: THIS IS JUST AN EXAMPLE with A SINGLE TABLE (the table “partition” is introduced as part of the solutions)! It does not reflect any aspect of any of the projects we are working on and it is indeed nothing one should apply to real life applications (as said in my previous comment).

I just thought it would help illustrate the possible partitioning solutions which I wanted to discuss in the first place.

Greetings,

Peter

[Edit] I cleaned up the example a bit and only left the redundancy on product category. [Edit]

I find a number of fundamental data architecture in-efficiencies in your schema. You really should invest in an MySQL expert to assist in the most optimal achitecture for large systems.

At worst, be sure to goto “The top 20 design tips for data architects” at the MySQL conference, you will learn a few things not to do!

Regards

Ronald Bradford
http://www.primebase.org
PBXT - The Community Storage Engine

OTOH some kind of redundant information (like having the product category in the orders - name and price are still just for illustration) would make partitioning a lot easier. (Using a categoryId that refers to a category table would make more sence).

The grade of redundancy depends on the type of data and the type of queries you want to perform against your data. If 90% of your queries on orders involve a productCategory it might make sense to store that information along with the order. Actually, I use this kind of redundancy in our application for almost all of the huge data sets. Otherwise certain queries would be impossible to run because of the JOIN size/overhead or at least would be painfully slow. And assuming that a productCategory cannot change after an order has been placed makes redundancy less evil. The latter assumption depends on your application and data design of course.

Anyway, normalization is a good thing and should be enforced where possible but sometimes you have to add redundancy.

Cheers,

Peter

CREATE TABLE orders (
orderId BIGINT(20) NOT NULL,

productCategory VARCHAR(50),
productName VARCHAR(255),
productPrice FLOAT,

PRIMARY KEY (orderId)
);

is not a normalized schema. Why wouldn’t you have a productID that was stored in the Products table with the productName, productPrice, productCategory, etc? Why is this information stored in the Orders table as redundant data?

Cheers,

Jay