Bench Testing ODBC Drivers

ODBC Bench Test is a 32 Bit C++ program based on the industry standard TPC-A benchmark (we will be extending this program to include the TPC-C and TPC-D benchmarks also). This program helps you compare the performance of Virtuoso against other backend database engines as well as compare the performance of various ODBC Drivers connecting to any ODBC compliant backend database.

1. Go to the OpenLink Data Access Drivers "Start Menu" item, then click on the "ODBC Bench Test 32 Bit" menu item. You will be presented with the "Bench Test" interface.

Figure: 1

2. Follow the File->Connect menu path which initializes the ODBC Driver Manager, which in turn presents you with a list of ODBC DSNs installed on your machine. Select the DSN that you want to benchmark, remember that by benchmarking a DSN you are benchmarking the ODBC Driver that serves the DSN in question and the backend database engine that serves the ODBC Driver. Choose the name of the datasource you want to benchmark.

Figure: 2

3. You will then be presented with a Login Dialog by the OpenLink Generic Driver for ODBC, enter a valid user name and password into the appropriate fields.

Figure: 3

4. Now follow the Bench-->Load Tables menu path and you will be presented with a dialog that enables you to configure key elements of your benchmark. Click the "Execute" button to commence the process of setting up your database for the benchmark tests. Please make sure you select the appropriate schema for the DBMS that you are connecting to so that the benchmark tables are created properly.

Figure: 4

5. As the process of loading data occurs, all the way up to completion, the benchmark program will provide status information into the benchmark output pane as shown below:

Figure: 5

6. Now that all the benchmark data has been loaded into your database, follow the Bench-->Run Benchmark menu path and then configure your actual benchmark session parameters:

These benchmark parameters fall into 3 categories, Timing Options, SQL Options, and Execution Options.

Figure: 6

Timing Options:

These setting allow you to configure the duration related aspects of this benchmark program

Minutes

this is the duration of each benchmark run

Runs

this controls how many iterations of the benchmarks you actually run (the default is one benchmark iteration with a duration of 5 minutes)

SQL Options:

These settings allow you to configure how your benchmark's SQL instructions are actually handled.

ExecDirect with SQL Text

this means that no form of repetitive SQL execution optimization is being applied (SQL statements are prepared and executed repetitively)

Prepare/Execute Bound Params

this means that the Parameter Binding SQL execution optimization is being applied (SQL is prepared once but executed many times without the overhead of re-preparing statements prior to execution)

Use Stored Procedures

this means that the Stored Procedure SQL optimization is being applied (benchmark instructions are stored within database being benchmarked)

Execution Options:

These settings allow you to configure the tone of your benchmark, for instance it could have Transaction scoping and a mix of record retrieval queries, or it could simply be input and update intensive with a minimal amount of record retrieval queries (the case when the 100 row query checkbox is unchecked a typical OLTP scenario)

Asynchronous

execute the benchmark instructions asynchronously

Use Transactions

make the benchmark use transaction control (instructions are scoped to transaction blocks)

Do 100 row Query

perform a simulation of a 100 record retrieval as part of the benchmark activity.

7. Click on the "Run All" button if you would like all the different benchmark type combinations to be performed.

8. When benchmark run complete benchmark data is written to the benchmark program's output pane.

Figure: 7

The key pieces of benchmark data that you need to look out for are:

Total Transactions

total number of transactions completed during the benchmark run

Transactions Per Second

number of transaction completed per second for the benchmark run

Information from this benchmark is automatically written to an Excel format CSV (the file c:\odbcbnch.csv) which makes it easy for you to graph and pivot data collated from several benchmark runs. A later version of this demo will actually write the benchmark data into an ODBC DSN that you provide thereby offering even more flexibility and accessibility to benchmark data.