Variations of the Star Schema Benchmark to Test the Effects of Data Skew on Query Performance

Tilmann Rabl, Meikel Poess, Hans-Arno Jacobsen, Patrick O'Neil, and Elizabeth O'Neil.

In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering, 2013.


The Star Schema Benchmark (SSB), has been widely used to evaluate the performance of database management systems when executing star schema queries. SSB, based on the well known industry standard benchmark TPC-H, shares some of its drawbacks, most notably, its uniform data distributions. Today's systems rely heavily on sophisticated cost-based query optimizers to generate the most efficient query execution plans. A benchmark that evaluates optimizer's capability to generate optimal execution plans under all circumstances must provide the rich data set details on which optimizers rely (uniform and non-uniform distributions, data sparsity, etc.). This is also true for other database system parts, such as indices and operators, and ultimately holds for an end-to-end benchmark as well. SSB's data generator, based on TPC-H's dbgen, is not easy to adapt to different data distributions as its meta data and actual data generation implementations are not separated. In this paper, we motivate the need for a new revision of SSB that includes non-uniform data distributions. We list what specific modifications are required to SSB to implement non-uniform data sets and we demonstrate how to implement these modifications in the Parallel Data Generation Framework to generate both the data and query sets.


Tags: star schema benchmark, ssb, parallel data generation framework, pdgf, benchmarking, skew

Readers who enjoyed the above work, may also like the following:

  • Big Data Generation.
    Tilmann Rabl and Hans-Arno Jacobsen.
    In Proceedings of the Workshop on Big Data Benchmarking, pages 20-27, 2013.
    Tags: pdgf, big data, benchmarking
  • Just can't get enough - Synthesizing Big Data.
    Tilmann Rabl, Manuel Danisch, Michael Frank, Sebastian Schindler, and Hans-Arno Jacobsen.
    In Proceedings of the ACM SIGMOD Conference, 2015.
    Demonstration Track.
    Tags: pdgf, dbsynth, data generation
  • Rapid Development of Data Generators Using Meta Generators in PDGF.
    Tilmann Rabl, Meikel Poess, Manuel Danisch, and Hans-Arno Jacobsen.
    In 6th International Workshop on Testing Database Systems, 2013.
    Tags: pdgf, meta generator, data generation