Databasen - Revisited

[Paul Makepeace]

On 10/18/06, Matt Sergeant <msergeant at messagelabs.com> wrote:
> Here's another reality - despite advances in database replication
> there is still often a single point of access in most database
> applications. Optimising your database is critical for high
> performance (ask anyone who has built a large scale DB backed
> application).

Partitioning's a well-known and understood technique in scaling
databases; most datasets have a fairly clear path to partitioning.
A-C, D-G, this goes back to index cards. Partitioning on full text is
hard, still, but that's relatively less likely requirement in the
wild.

>  Even in a replicated setup, databases don't scale
> brilliantly across servers (too much replication overhead,
> application becomes too complex etc).

Depends...

http://labs.google.com/papers/bigtable.html
http://video.google.com/videoplay?docid=7278544055668715642

Paul