What is DA-bench?
DA-bench is the “Data Analyst Benchmark” - it is a series of questions that you would expect Data Analysts to be able to solve.
This benchmark was created to help us test Unsupervised and other data tools against real-world problems, so we can understand the strengths and weaknesses of various approaches to automating analytics.
DA-bench is a visual benchmark, you can see both the score and videos of how every tool performs on every test. See some examples below:
Leaderboard
| Tool | Date Tested |
Setup Score (40% of Overall) |
Test Score (60% of Overall) |
Overall Score | |
|---|---|---|---|---|---|
|
Unsupervised
Full Agent Use your Data Warehouse |
December 6, 2024 | 100.0 | 48.4 | 69.0% | VIEW DETAILS |
|
Databricks IQ
Co-pilot Use your Data Warehouse |
December 3, 2024 | 100.0 | 39.5 | 63.7% | VIEW DETAILS |
|
Databricks Genie
Full Agent Use your Data Warehouse |
December 2, 2024 | 100.0 | 32.6 | 59.5% | VIEW DETAILS |
|
MicroStrategy Auto Answers
Full Agent Use your Data Warehouse |
December 10, 2024 | 100.0 | 30.2 | 58.1% | VIEW DETAILS |
|
Amazon Q in QuickSight
Co-pilot Use your Data Warehouse |
December 3, 2024 | 100.0 | 24.7 | 54.8% | VIEW DETAILS |
|
Julius
Full Agent Use Data In Memory |
December 6, 2024 | 50.0 | 55.8 | 53.5% | VIEW DETAILS |
|
Einstein for Tableau
Co-pilot Use your Data Warehouse |
November 26, 2024 | 100.0 | 19.5 | 51.7% | VIEW DETAILS |
|
ChatGPT Data Analyst
Full Agent Use Data In Memory |
December 2, 2024 | 50.0 | 34.9 | 40.9% | VIEW DETAILS |
|
ThoughtSpot Sage
Natural Language Search Use your Data Warehouse |
November 28, 2024 | 100.0 | 0.0 | 40.0% | VIEW DETAILS |
|
Qlik Sense Insight Advisor
Natural Language Search Use your Data Warehouse |
November 26, 2024 | 100.0 | -0.5 | 39.7% | VIEW DETAILS |
|
Snowflake Copilot
Co-pilot Use your Data Warehouse |
December 5, 2024 | 16.7 | 48.8 | 36.0% | VIEW DETAILS |
|
BigQuery
Co-pilot Use your Data Warehouse |
November 25, 2024 | 33.3 | 30.2 | 31.5% | VIEW DETAILS |
|
Snowflake Cortex Analyst
Full Agent Use Data In Memory |
December 4, 2024 | 0.0 | 39.5 | 23.7% | VIEW DETAILS |
|
IBM Cognos Assistant AI
Natural Language Search Use Data In Memory |
November 4, 2024 | 33.3 | -3.3 | 11.3% | VIEW DETAILS |
|
SAP Just Ask
Natural Language Search Use your Data Warehouse |
November 27, 2024 | 16.7 | 0.0 | 6.7% | VIEW DETAILS |
About
The Data Analyst Benchmark is a collection of datasets and prompts that can be used to test how automated analytics tools handle common data analyst tasks.
We use this information to help us prioritize work to improve our AI. We are making it available publicly to help other companies improve their tools and to help users evaluate which tools are relevant to their problems.
DA-bench currently tests dozens of prompts across 9 categories. Evaluation is performed by manual testing by a third-party, scores and videos of test results are displayed on dabench.com.
More Info
DA-bench is maintained by Unsupervised.
Suggestions and contributions are welcome on the DA-bench Github Repository.