Welcome to the Bristlecone Project!

Cool News

Bristlecone 0.6 is out. If you attended the talks and are looking for software, this release contains the scenarios we discussed. Source and binary builds are available at the Forge∞.

More Cool News

Bristlecone has been featured at a couple of recent conferences.

• PostgreSQL East∞ - "Portable Scale-Out Benchmarks for PostgreSQL." Download slides here∞.
• MySQL User Conference∞ - "Portable Scale-Out Benchmarks for MySQL." Download slides here∞

Please do more talks about performance using Bristlecone!

What is Bristlecone?

Bristlecone provides tools for testing database performance. There are two main tools included in the package.

• Evaluator generates mixed loads of inserts, updates, deletes, and selects. Output can be built-in graphics, HTML, XML, or CSV.

• Benchmark runs performance test cases with systematically varying parameters. Output can be HTML or CSV.

The Bristlecone tools were designed for comparative evaluations of database clusters. They include built-in support for threading, ability to generate new tests quickly with simple configuration file changes, and the ability to do systematic tests across different database implementations.

Bristlecone is written in Java. It has mostly run on Linux but is easily adaptable to other platforms. Bristlecone has been tested against the following database types: PostgreSQL∞, MySQL∞, and HSQLDB (Hypersonic)∞. We have also run it extensively against Continuent uni/cluster.

Portable Scale-Out Benchmarks

Scale-out is an architectural approach for database systems that increases availability and performance by spreading multiple copies of data across hosts connected by a network. Such clusters can be built using native database replication or middleware replication such as Sequoia provides. More ambitious designs include pluggable replication as implemented by the GORDA Project∞.

Bristlecone comes with scenarios that constitute a "portable scale-out benchmark" to compare performance of database clusters that use different scale-out approaches. Scale-out architectures vary in their capabilities with respect to the following:

• Read scaling -- throughput, response time, and scaling

• Write scaling -- throughput, response time, and scaling

• Latency -- Delay on reads and writes induced by proxies; latency between replicated databases

• Transactions -- throughput and number of aborts or deadlocks under different transaction models

The behavior of particular architectures and machine configurations can vary greatly as you change things like the number of clients, number of hosts, width and length of data selected, data types, number of operations per transactions, or as you switch between full transactions and auto-commit operations.

For an idea of what the benchmarks look like, check out the talks in the News section.

Downloads

Builds can be downloaded from the Forge∞. Source and binary builds are available. We will update these at regular intervals. In the meantime, grab nightly snapshots or the source code.

Bristlecone requires Sun JDK 1.5∞ or higher to run. To build source you will also require Apache Ant 1.7.0∞ or greater.

Source code

Bristlecone is licensed under GPL v2. The source code is maintained in a Subversion repository. It can be checked out using the following command:

Or for developers:

The nightly SVN tree snapshot can be downloaded from the Forge∞

Important note: You must use port 32 with svn+ssh. For security reasons port 22 is disallowed.

Bug reports

View and report bugs in the JIRA∞

Mailing list

Subscribe, unsubscribe to the mailing list can be done here∞. To subscribe or unsubscribe to commits, go here∞.

Getting started

Untar the distribution and look at the README in the main directory to get oriented. Full documentation is included in the doc directory. (Well, as full as you are likely to get for free. ;) There are example configuration files in the config directory. These have additional README files to help you get started quickly.

What's Next

We are working on better graphics and thinking about how to get benchmarks out of Java and into simple scripted templates that can be used by anyone (a great suggestion from Jan Vieck of SLONY fame). You are welcome to help out--we are looking for people interested in this problem who would like to collaborate.