cortex — Run #180 (2025-01-07)
etup score reflects Snowflake's data warehouse functionality, although the setup video demonstrates uploading csvs. Setup also includes creating YAML and updating Streamlit code. These steps are not captured in the setup video.
Overall Score:
52.3%
(40% Setup Score + 60% Test Score)
(40% Setup Score + 60% Test Score)
DA-bench Setup for Run #180 — Setup Score: 71.4% (5 / 7)
See how this tool was set up for the test.
-
Setup Less Than 20 Minutes
-
Connects to Data Warehouse
-
Handles 1TB Table
-
No Individual Upload of Files
-
No Python for Setup
-
No SQL for Setup
DA-bench Results for Run #180 — Test Score: 39.6% (93 / 235)
Data Querying (95 / 125)
20 Correct Answers,
1 Hallucination
Domain Knowledge (0 / 5)
0 Correct Answers,
0 Hallucinations
Question | Date Tested | Overall Score | Video Recording |
---|---|---|---|
dk01
|
2025-01-07 | 0 |
Feature Engineering (0 / 40)
1 Correct Answer,
1 Hallucination
Question | Date Tested | Overall Score | Video Recording |
---|---|---|---|
fe1
|
2025-01-07 | 5 | |
fe2
|
2025-01-07 | 0 | |
fe3
|
2025-01-07 | -5 | |
fe4
|
2025-01-07 | 0 | |
fe5
|
2025-01-07 | 0 | |
fe6
|
2025-01-07 | 0 | |
fe7
|
2025-01-07 | 0 | |
fe8
|
2025-01-07 | 0 |
Insight Identification (0 / 40)
1 Correct Answer,
1 Hallucination
Question | Date Tested | Overall Score | Video Recording |
---|---|---|---|
ii2
|
2025-01-07 | 5 | |
ii5
|
2025-01-07 | 0 | |
ii6
|
2025-01-07 | 0 | |
ii7
|
2025-01-07 | 0 | |
ii8
|
2025-01-07 | -5 | |
ii10
|
2025-01-07 | 0 | |
ii12
|
2025-01-07 | 0 | |
ii15
|
2025-01-07 | 0 |
Learning (-10 / 10)
0 Correct Answers,
2 Hallucinations
Question | Date Tested | Overall Score | Video Recording |
---|---|---|---|
l1
|
2025-01-07 | -5 | |
l2
|
2025-01-07 | -5 |
Visualization (8 / 15)
2 Correct Answers,
0 Hallucinations
Question | Date Tested | Overall Score | Video Recording |
---|---|---|---|
v1
|
2025-01-07 | 4 | |
v2
|
2025-01-07 | 0 | |
v3
|
2025-01-07 | 4 |