DA-bench

Visual Benchmark for Data Analytics AI Agents

Run #13


DA-bench Results for Run #13 — Overall Score: 5.2% (11 / 210)

Data Querying (9 / 105)
Question Date Tested Score Video Recording

dq01
Perform an aggregation on an explicit column

May 31, 2024 - 10:12AM 1
 

dq02
Perform an aggregation with an explicit table but not an inferred column

May 31, 2024 - 10:29AM 1
 

dq03
Perform an aggregation with an implicit table and implicit column

May 31, 2024 - 10:37AM 0
 

dq04
Find and compare information across tables without joins

May 31, 2024 - 10:47AM 1
 

dq05
Work with non-literal values

May 31, 2024 - 10:55AM 2
 

dq06
Work with non-literal values and non-SQL data manipulation

May 31, 2024 - 11:05AM 1
 

dq07
Deal with common acronymns and more advanced aggregations

May 31, 2024 - 11:18AM 0
 

dq08
Multi-step queries

May 31, 2024 - 11:30AM 0
 

dq09
Aggregations with numeric predicates to filter

May 31, 2024 - 11:38AM 0
 

dq10
Aggregations with categorical predicates to filter

May 31, 2024 - 11:44AM 0
 

dq11
Schema review

May 31, 2024 - 12:10AM 0
 

dq12
Aggregate records that are filtered with a predicate requiring a join

May 31, 2024 - 12:14AM 0
 

dq13
Recognizes truly ambiguous queries.

May 31, 2024 - 12:34AM 0
 

dq14
Can handle boolean features

May 31, 2024 - 12:45AM 0
 

dq15
Handles ambiguous column names

May 31, 2024 - 12:52AM 0
 

dq16
Understands set operates require consideration of overlap

May 31, 2024 - 12:59AM 0
 

dq17
Finds relevant values inside a Column to answer questions

May 31, 2024 - 1:05 AM 0
 

dq18
Lookup a single record by ID

May 31, 2024 - 1:12 AM 1
 

dq19
Perform an aggregation by a different name and a second query from that

May 31, 2024 - 1:18 AM 1
 

dq20
Perform an aggregation based on a very different question name

May 31, 2024 - 1:22 AM 1
 

dq21
Perform a filter and an unusually-phrased aggregation in the correct order

May 31, 2024 - 1:25 AM 0
 
Visualization (0 / 15)
Question Date Tested Score Video Recording

v1
Basic Charting

May 31, 2024 - 2:15 AM 0
 

v2
Charting with two series

May 31, 2024 - 2:23 AM 0
 

v3
Categorical Charts

May 31, 2024 - 2:29 AM 0
 
Feature Engineering (2 / 35)
Question Date Tested Score Video Recording

fe1
Make an indicator feature for a criteria set

May 31, 2024 - 2:40 AM 0
 

fe2
Make a categorical feature from a criteria set

May 31, 2024 - 2:48 AM 1
 

fe3
Minmax normalization

May 31, 2024 - 2:53 AM 0
 

fe4
Combining two input columns

May 31, 2024 - 3:00 AM 1
 

fe5
Sentiment

May 31, 2024 - 3:06 AM 0
 

fe6
Phrase Identification in Text

May 31, 2024 - 3:09 AM 0
 

fe7
Advanced NLQ

May 31, 2024 - 3:11 AM 0
 
Insight Identification (0 / 40)
Question Date Tested Score Video Recording

ii1
Basic correlation

May 31, 2024 - 3:16 AM 0
 

ii2
Compare an aggregation for two distinct subsets of data

May 31, 2024 - 3:20 AM 0
 

ii3
Basic Correlation but requiring aggregation from another table

May 31, 2024 - 3:25 AM 0
 

ii4
Outlier identification

May 31, 2024 - 3:30 AM 0
 

ii5
Identifying basic trends on short timelines

May 31, 2024 - 3:35 AM 0
 

ii6
Understands statistical significance

May 31, 2024 - 3:40 AM 0
 

ii7
Understands derivitives

May 31, 2024 - 3:45 AM 0
 

ii8
Can use NLQ feature engineering as part of an insight request

May 31, 2024 - 3:50 AM 0
 
Learning (0 / 10)
Question Date Tested Score Video Recording

l1
Can remember the meanings of oddly-named columns

May 31, 2024 - 3:55 AM 0
 

l2
Can remember criteria sets under a single name

May 31, 2024 - 4:01 AM 0
 
Domain Knowledge (0 / 5)
Question Date Tested Score Video Recording

dk01
Column Relevance Determination

May 31, 2024 - 4:12 AM 0