What is DA-bench?
DA-bench is the “Data Analyst Benchmark” - it is a series of questions that you would expect Data Analysts to be able to solve.
This benchmark was created to help us test Unsupervised and other data tools against real-world problems, so we can understand the strengths and weaknesses of various approaches to automating analytics.
DA-bench is a visual benchmark, you can see both the score and videos of how every tool performs on every test. See some examples below:
Leaderboard
| Tool | Date Tested | Score | |
|---|---|---|---|
|
Unsupervised
Full Agent Use your Data Warehouse |
2024-06-28 | 51.1% | VIEW DETAILS |
|
Julius
Full Agent Use Uploaded Data |
2024-06-09 | 37.6% | VIEW DETAILS |
|
Databricks IQ
Co-pilot Use your Data Warehouse |
2024-07-04 | 33.2% | VIEW DETAILS |
|
Einstein for Tableau
Co-pilot Use your Data Warehouse |
Not Yet Tested | ||
|
Snowflake Cortex Analyst
Co-pilot Use your Data Warehouse |
Not Yet Tested | ||
|
ChatGPT Data Analyst
Full Agent Use Uploaded Data |
Not Yet Tested | ||
|
ThoughtSpot Sage
Natural Language Search Use your Data Warehouse |
Not Yet Tested | ||
|
Copilot in Power BI
Co-pilot Use your Data Warehouse |
Not Yet Tested | ||
About
The Data Analyst Benchmark is a collection of datasets and prompts that can be used to test how automated analytics tools handle common data analyst tasks.
We use this information to help us prioritize work to improve our AI. We are making it available publicly to help other companies improve their tools and to help users evaluate which tools are relevant to their problems.
DA-bench currently tests dozens of prompts across 9 categories. Evaluation is performed by manual testing by a third-party, scores and videos of test results are displayed on dabench.com.
More Info
DA-bench is maintained by Unsupervised.
Suggestions and contributions are welcome on the DA-bench Github Repository.