Enzo SQL Baseline (open source)                  
 
Overview

Download Enzo SQL Baseline to run performance baselines in SQL Server and SQL Azure. You can then compare performance results side-by-side and analyze the various metrics provided on the screen. You can also export the performance results to Excel to perform additional analysis and create custom graphs.

This free, open source tool allows you to perform high-end performance baselines. For example the tool allows you to run multiple threads (running test runs in parallel) and run before, during and after SQL commands. The metrics also include averages and standard deviations on key metrics to understand the quality of your data.

Download Now
Online Help
General Information
This performance tool allows you to create a performance baseline and export it into Excel for future review and further analysis. The tool works by executing a primary statement and capturing two sets of metrics: the SQL Performance Metrics and the Provider Statistics metrics. The former capture SQL Server or SQL Azure metrics by reading the sys.dm_exec_sessions view and the latter read the statistics directly from ADO.NET. As a result, one provides a server view or your performance metrics while the other provides a consumer view. There is, however, one exception: the Duration metric of the SQL Performance metrics will also include the time it takes to the data to be completely server to the client, and as such it is impacted by the network speed.

How to Run a Basic Test
There are about 10 options to run a performance test. You can read more about the individual options by placing the mouse over each option. The following settings will allow you to run a basic performance test:

SQL CommandEnter a SQL command in the text box (in the Command area) or specify a stored procedure by selecting RPC Call. When using a stored procedure, type the name of the stored procedure in the first text field then select Fetch Paramters for Enzo to retrieve the necessary parameter names. You will need to edit each parameter to specify a value. A SQL command is required.
Database ConnectionSelect a database connection file. One must be created; use File -> Manage Connections to create one or more connection files. The password stored in the file is encrypted. This is a required setting.
Database NameThe name of the database to connect to. This is a required setting for SQL Azure.
Execute the same runBy default, the test will be run only once. Use this option to run your SQL Statement more than once. When more test runs are executed the performance averages have a tendency to be more precise.


Once the above options specified, click on Start Test Run Execution. When the execution is complete, you will notice the Threads label (at the bottom of the form) shows 0. Click on Last Execution Results to view the last results. You may also click on Test Run -> View Detailed Results in Excel to export the latest results in Excel.

If you like the results, click on the Comparative Analysis tab, and click on "Add last run to comparative analysis". This will add the latest run in memory for comparative analysis. Once multiple runs have been performed, select a baseline from this same tab to view how the various test runs compare.

Running a Test Against SQL Azure
Running a test against SQL Azure may yield surprising results. While you can expect SQL Azure to run on advanced servers with fast CPUs, the SQL Azure database may be very far away from you. In addition the network traffic is entirely encrypted. As a result, there may be more bytes being transferred than on a local test. Since the database is far, and the data is encrypted, you may notice that comparing SQL Azure and SQL Server performance metrics may not be very meaningful.

However, you should still take the metrics into serious consideration as a way to help you design a fast SQL Azure database. For example, you may have various versions of the same stored procedure, with various degrees of roundtrips. The metrics gathered will help you understand how the time is being spent.

About Parallel Executions
Enzo SQL Baseline allows you to run the same test multiple times, in parallel. You choose the number of threads that Enzo should start. Keep in mind that doing so will induce more stress on the database server, which is, of course, the whole point. However, when running against SQL Azure, make sure to run less than 20 threads or so.

Indeed, SQL Azure has a built-in firewall with a Denial of Service attack detection mechanism. You may be locked out of your database for a few hours if you generate too much performance activity.

SQL Performance Metrics (SQLPerformance)

The performance metrics with a type of SQLPerformance are taken from the dm_exec_sessions view. Enzo SQL Baseline displays a subset of these metrics. A complete description of the metrics can be found here: sys.dm_exec_sessions

These metrics are available for both SQL Server and SQL Azure. However, when using SQL Azure, these metrics are not available when connecting to the master database.

Here is a summary of the metrics being used:

CPU (SQLPerformance)cpu_time in dm_exec_sessions. CPU time, in milliseconds, that was used by the command
Reads (SQLPerformance)logical_reads in dm_exec_sessions. Logical reads that were generated by the command
Duration (SQLPerformance)total_elapsed_time in dm_exec_sessions; Total elapsed time, in milliseconds, to service the command. This time includes the time it takes to transfer the data to the client.
memory_usage (SQLPerformance)Number of 8-KB pages of memory used by this session
total_scheduled_time (SQLPerformance)Total time, in milliseconds, for which the session (requests within) were scheduled for execution
physical_reads (SQLPerformance)reads in dm_exec_sessions. Number of reads performed by the statement
Writes (SQLPerformance)Number of writes performed by the statement
RowCount (SQLPerformance)Number of rows returned by the statement
(SQLPerformance)Number of reads performed by the statement


Provider Statistics (LibraryPerformance)

The performance metrics collected by the ADO.NET Library can be found here: Provider Statistics for SQL Server

Advanced Considerations
Certain metrics are shown with a Standard Deviation calculation, such as the STD DEV CPU. This metric for example indicates how spread out the individual CPU metric are in the test runs. A small standard deviation indicates a predictable underlying metric. For example, if the Average CPU is 2000 and the STD DEV CPU is 0, this means that all the CPU measures from all the individual runs are 2000.

In certain cases the first few test runs use more resources as SQL Server or SQL Azure attempts to cache data and execution plans. This may lead to a few unusually large amounts of reads or CPU activity. The Comparative Analysis tab includes a check box that allows you to remove temporarily the minimum and maximum read, duration and CPU test runs, in order to remove the unusual measures. This usually has the effect of lowering the standard deviation measures.

Another way to improve the quality of the performance metrics is to check the Prime Database option before starting the test. This checkbox simply discards the metrics of the first test run. SQL Azure Warning
You should now that executing performance tests in SQL Azure can potentially have two ramifications:
CostThe more tests you run against SQL Azure, the higher your monthly bill. SQL Azure bills you for bytes transferred over the wire; so if you perform tests that return many records you may notice an increase in your monthly bill.
Denial of Service (DoS)SQL Azure has a feature that prevents too many connections from being established. After reaching about 50 connections or so, you may notice that access to SQL Azure is being blocked for a while. This will prevent any database account from connecting to the database that has been locked out. Other databases however may still be available. As a result you should use the Parallel Threads with caution in SQL Azure.


Sample Screenshots

Simple settings used to run a performance test against SQL Server or SQL Azure:




Side-by-side analysis of two test runs: