Nishi Family › Compare › Data Engineering and Analysis Agents
Nishi Compare · capabilities, evidence and delivery
Data Engineering and Analysis Agents
Compare capabilities and delivery evidence. Source discovery, qualification, deployment and accepted outcomes are separate; an open contract names work still to verify.
Nishi data organs vs Spider-Agent and Agentar-Scale-SQL and DA-Agent and human data engineers on the field's benchmarks
Overview
The purpose, declared position and evidence coverage of this domain. Source presence and completed acceptance are different measures.
Where we are. MEASURED 2026-09-16 at admission: the estate reads SQLite (nx_sqlite_rows), Parquet (nx_parquet_lib, the sovereign reader that reproduces the BRIGHT card), CSV and TSV (nx_value_parse_csv) and XLSX (nx_xlsx, which read the 1077-row OSWorld results sheet for the automation board the same day); it profiles a table (nx_analyst_data) and approximates distincts and quantiles with a measured error (nx_dataframe_approx); it grades code by a byte-exact compile-and-run judge (nx_forge_local_bench) and it can replay a rival's published outputs through its own judge (nx_bright_reasoned_import, landed on the search board today). It writes NO SQL and NO data-science code from language yet: every text-to-SQL, warehouse, dbt, GUI-workflow and agentic-analysis row is an honest _ABSENT_ contract. The field, read from the mirrored bytes: BIRD test execution accuracy 82.39 by an agent system dated 2026-09-07 against a human 72.37 [bird26]; Spider 2.0-Snow 96.70, DBT 65.6 and Lite 76.23 at the March-2026 listing [spider226]; DS-1000 held at 43.3 by Codex-002 on its own page [ds100022]; DA-Code, DSBench and DABstep grade agents end to end [dacode24] [dsbench25] [dabstep26]; LiveSQLBench rotates tasks against contamination [livesqlbench26]; Spider2-V moves the whole workflow into the tools' GUIs [spider2v25].
Where we need to go. A sovereign data agent that writes and executes SQL and analysis code against the estate's own stores and the field's benchmarks, graded by an execution judge the estate owns, with every published number carrying its harness, its n and its date, and no third party in the run path: the benchmarks are oracles outside, their tasks are data with licences beside them, and their leaders are the bars.
Latest recorded release
2026-09-16 · DE1
BOARD ADMITTED 2026-09-16 from eight sources fetched over the sovereign fetcher and read from the mirrored bytes on both hosts (byte-identical): Spider 2.0, Spider2-V, BIRD, DS-1000, DABstep, LiveSQLBench, DSBench and DA-Code; the hand page that held this domain since 2026-08-13 leaves hand.roster by matrix admission; the OSWorld results sheet read the same day through nx_xlsx is the automation board's, cited there
Release entries describe recorded changes; they do not establish that every capability passed evaluation.
5 of 13 capability source symbols found|2 of them authored stronger-claims|8 open|coverage 384/1000|adoption 3 full / 2 partial
Evidence profile — what the gaps on this board actually are
nx_swcompare_evidence dataeng and this section fills itself on the next beat. No numbers are shown, deliberately — a zero-filled profile would read as a board with no gaps, which is the one wrong answer nobody would question.Production map
Follow the dependencies, declared acceptance criteria and recorded priorities. Inspect source binding before treating a rank as executable work.
Ranking source binding: STALE_OR_INVALID_BINDING. Recorded priorities require current acceptance evidence and resource checks before execution.
Recorded ranking withheld
Retained ranking is not bound to the current plan and matrix: STALE_OR_INVALID_BINDING. Historical data remains retained. Refresh the ranking after resolving the input differences. Latest execution outcome is unverified.
Declared roadmap — contract, acceptance, executor, effort
Ladder to SOTA
| Rung | Closes with | Definition of done (pre-declared) | Executor | Est. |
|---|---|---|---|---|
| Execution judge for SQL over SQLite (DE1) | sqr_exec_judge | DONE-RULE PRE-DECLARED: the judge runs a gold query and a candidate query against the same SQLite file through nx_sqlite_rows, compares the result multisets order-insensitively, and prints EX 1 or 0 per pair with the row counts; a gate plants a candidate that returns the right rows in another order (must read 1), a candidate with one extra row (must read 0) and a candidate that errors (must read 0 with the error named); no rival oracle in the path [bird26] | Organ | 1 u |
| BIRD Mini-Dev harness (DE2) after DE1 | dc_bird_minidev | the Mini-Dev 500-pair set is mirrored from Hugging Face with its licence beside the bytes and the schema files, prepped into queries and gold TSVs the way nx_bright_prep shapes BRIGHT, and every candidate file is scored by DE1 into a receipt with n, EX and the harness hash; ACCEPT RULE: the gold set scored against itself reads EX 500 of 500 before any model is scored [bird26] | Organ | 1.5 u |
| Sovereign text-to-SQL contender (DE3) after DE2 | sqr_text2sql | a query writer over nx_engine_dispatch (local no-float seat, external U tier, frontier) that emits SQL for a natural-language question and a schema, scored only by DE1 on DE2; the receipt names the engine and its ceiling; the local seat's expected loss under its 96-token ceiling is published, never hidden; done when a receipt exists for all three engines | Organ | 3 u |
| Spider 2.0-Lite SQLite subset harness (DE4) after DE1 | dc_spider2_lite | the 135 SQLite examples of Spider 2.0-Lite (547 in total; the BigQuery and Snowflake examples wait for DE5) mirrored with licence and scored by DE1; the receipt states the subset size beside the number so a 135-example score is never read as the 547 leaderboard's [spider226] | Organ | 1.5 u |
| Warehouse dialect bridge (DE5) after DE4 | dc_dialect_bridge | BigQuery and Snowflake SQL run against the vendors' free quotas as ORACLE HOSTS outside the build path, results returned to DE1 for the multiset compare; the bridge is data (dialect rows), the credentials are vault leases, and a refused connection is recorded UNREACHABLE and never scored [spider226] | Organ | 3 u |
| DS-1000 referee replay through an isolated Python oracle host (DE6) | fg_dscode | DS-1000 prompts and tests mirrored with licence; candidate completions from the engine dispatcher are executed on an isolated oracle host (Python is a third party and stays outside the estate's run path) and graded by the benchmark's own test cases, with the receipt carrying the host hash, n and the per-library breakdown [ds100022] | Organ | 2 u |
| Agentic analysis runs, DABstep and DA-Code (DE7) after DE6 | ad_agent_run | the workflow engine drives a plan, execute and observe loop whose tools are the estate's readers, profiler and approximate analytics, against the DABstep and DA-Code tasks mirrored with licence, graded by each benchmark's own answer checker on the oracle host; every run publishes completion rate and executable-code rate the way DA-Code's board does [dabstep26] [dacode24] [dsbench25] | Organ | 3 u |
| dbt project tasks, Spider 2.0-DBT (DE8) after DE5 | dc_dbt_project | the 68 DBT examples run through the workflow engine with the dialect bridge, graded by the benchmark's own result checks on the oracle host [spider226] | Organ | 3 u |
| GUI-driven data workflows, Spider2-V (DE9) after DE7 | br_data_workflow | the sovereign browser drives the tools' GUIs for the 494 Spider2-V tasks on an isolated oracle host, graded by the benchmark's own checkers; the GPT-4o baseline is the published floor and the estate publishes its own success rate beside it [spider2v25] | Organ | 4 u |
| Rung | Role | Toward | Stands on | Why |
|---|---|---|---|---|
| DE1 | substrate | T-BIC-BIRD | - | the judge every SQL number on this board is computed by |
| DE2 | arm | T-BIC-BIRD | DE1 | the harness that puts the estate's number beside the BIRD leaders on the same 500 dev pairs |
| DE3 | contender | T-BIC-BIRD | DE2 | its done-rule is a number against the human test bar |
| DE4 | arm | T-FR-SPIDER2 | DE1 | the SQLite third of Spider 2.0-Lite, scored by the same judge |
| DE5 | substrate | T-FR-SPIDER2 | DE4 | the dialect bridge the Snow and BigQuery tracks stand on |
| DE6 | arm | T-FR-DATASCI | - | the DS-1000 replay sizes the data-science code lever before any agent is built |
| DE7 | contender | T-FR-DATASCI | DE6 | its done-rule is a completion rate against the agentic boards |
| DE8 | contender | T-FR-SPIDER2 | DE5 | its done-rule is a number on the DBT track |
| DE9 | arm | T-FR-SPIDER2 | DE7 | the GUI workflow leg, an arm until the browser lane can drive the tools |
Milestones
| Milestone | Rungs | Cumulative |
|---|---|---|
| M1 · The judge and the first harnesses | DE1,DE2,DE4 | 4 u |
| M2 · A contender on the board | DE3,DE5,DE8 | 9 u |
| M3 · The agentic frontier | DE6,DE7,DE9 | 9 u |
Ladder verdict, counted (17 rows, more than the 12-row disclosure bar)
| Class | Count | Note |
|---|---|---|
| targets | 3 | sotatarget rows on the plan |
| best in class | 1 | targets whose class is best-in-class, by code |
| frontier | 2 | targets whose class is frontier, by code |
| undated | 0 | targets naming no bar that exists |
| bad class | 0 | targets whose class is neither word |
| duplicate | 0 | target ids an earlier row already carries |
| rungs | 9 | rung rows on the plan |
| placed | 9 | rungs carrying a rungrole row |
| unplaced | 0 | rungs with no role toward any target |
| bad role | 0 | rungrole rows naming a role the ladder does not know |
| bad target | 0 | rungrole rows naming a target no sotatarget row declares |
| rungs, summed | 9 | RECONCILED: the parts sum to the total 9 |
| substrate | 2 | what the target's own methods consume |
| arm | 4 | a sovereign arm that triangulates without competing for the number |
| contender | 3 | its done-rule is a number against the target |
| superseded | 0 | kept as capability, retired as a claim |
| placed, summed | 9 | RECONCILED: the parts sum to the total 9 |
| Class | Count | Note |
|---|---|---|
| bars | 8 | sotabar rows on the plan |
| fresh | 0 | the bar month is the current month |
| attested | 8 | an older bar with a this-month barscan reporting no newer entry |
| stale | 0 | a this-month scan saw a newer month the bar ignores |
| unattested | 0 | nobody looked this month |
| malformed | 0 | a sotabar row missing a field |
| duplicate | 0 | bar ids an earlier row already carries |
| bars, summed | 8 | RECONCILED: the parts sum to the total 8 |
| current month | 2026-10 | the month every bar is judged against |
Inspect a rung and its prerequisites
Declared nodes 9. Rank input binding: STALE_OR_INVALID_BINDING. Dependency order is authored. Implementation, acceptance evidence, authority and resource readiness are unverified. No action is recommended or dispatched here.
Plan SHA-256 09a093a9e9f4a5eee214b0de5aa5f4f11e16bd6900ea82da65968daaed451497. Target rows 3; role rows 9. Existing risks and release worklog retain their own scope; no node completion is inferred.
Runtime observation coverage: 0 of 9 rungs have current, identity-bound execution observations. Observation may report RED. Accepted completion remains unknown; source presence, qualification and deployment do not establish it.
Use Enter or Space on a rung to inspect its contract. Prerequisite links locate another rung in this list; open its summary to inspect it. Estimates are authored effort, not forecasts.
DE1 — Execution judge for SQL over SQLite
Prerequisites: None declared; this does not establish execution eligibility.
Contract:
sqr_exec_judgeAcceptance: DONE-RULE PRE-DECLARED: the judge runs a gold query and a candidate query against the same SQLite file through nx_sqlite_rows, compares the result multisets order-insensitively, and prints EX 1 or 0 per pair with the row counts; a gate plants a candidate that returns the right rows in another order (must read 1), a candidate with one extra row (must read 0) and a candidate that errors (must read 0 with the error named); no rival oracle in the path [@bird26]
Authored effort: 1. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE2 — BIRD Mini-Dev harness
Prerequisites: DE1 (acceptance unverified)
Contract:
dc_bird_minidevAcceptance: the Mini-Dev 500-pair set is mirrored from Hugging Face with its licence beside the bytes and the schema files, prepped into queries and gold TSVs the way nx_bright_prep shapes BRIGHT, and every candidate file is scored by DE1 into a receipt with n, EX and the harness hash; ACCEPT RULE: the gold set scored against itself reads EX 500 of 500 before any model is scored [@bird26]
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE3 — Sovereign text-to-SQL contender
Prerequisites: DE2 (acceptance unverified)
Contract:
sqr_text2sqlAcceptance: a query writer over nx_engine_dispatch (local no-float seat, external U tier, frontier) that emits SQL for a natural-language question and a schema, scored only by DE1 on DE2; the receipt names the engine and its ceiling; the local seat's expected loss under its 96-token ceiling is published, never hidden; done when a receipt exists for all three engines
Authored effort: 3. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE4 — Spider 2.0-Lite SQLite subset harness
Prerequisites: DE1 (acceptance unverified)
Contract:
dc_spider2_liteAcceptance: the 135 SQLite examples of Spider 2.0-Lite (547 in total; the BigQuery and Snowflake examples wait for DE5) mirrored with licence and scored by DE1; the receipt states the subset size beside the number so a 135-example score is never read as the 547 leaderboard's [@spider226]
Authored effort: 1.5. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE5 — Warehouse dialect bridge
Prerequisites: DE4 (acceptance unverified)
Contract:
dc_dialect_bridgeAcceptance: BigQuery and Snowflake SQL run against the vendors' free quotas as ORACLE HOSTS outside the build path, results returned to DE1 for the multiset compare; the bridge is data (dialect rows), the credentials are vault leases, and a refused connection is recorded UNREACHABLE and never scored [@spider226]
Authored effort: 3. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE6 — DS-1000 referee replay through an isolated Python oracle host
Prerequisites: None declared; this does not establish execution eligibility.
Contract:
fg_dscodeAcceptance: DS-1000 prompts and tests mirrored with licence; candidate completions from the engine dispatcher are executed on an isolated oracle host (Python is a third party and stays outside the estate's run path) and graded by the benchmark's own test cases, with the receipt carrying the host hash, n and the per-library breakdown [@ds100022]
Authored effort: 2. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE7 — Agentic analysis runs, DABstep and DA-Code
Prerequisites: DE6 (acceptance unverified)
Contract:
ad_agent_runAcceptance: the workflow engine drives a plan, execute and observe loop whose tools are the estate's readers, profiler and approximate analytics, against the DABstep and DA-Code tasks mirrored with licence, graded by each benchmark's own answer checker on the oracle host; every run publishes completion rate and executable-code rate the way DA-Code's board does [@dabstep26] [@dacode24] [@dsbench25]
Authored effort: 3. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE8 — dbt project tasks, Spider 2.0-DBT
Prerequisites: DE5 (acceptance unverified)
Contract:
dc_dbt_projectAcceptance: the 68 DBT examples run through the workflow engine with the dialect bridge, graded by the benchmark's own result checks on the oracle host [@spider226]
Authored effort: 3. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.DE9 — GUI-driven data workflows, Spider2-V
Prerequisites: DE7 (acceptance unverified)
Contract:
br_data_workflowAcceptance: the sovereign browser drives the tools' GUIs for the 494 Spider2-V tasks on an isolated oracle host, graded by the benchmark's own checkers; the GPT-4o baseline is the published floor and the estate publishes its own success rate beside it [@spider2v25]
Authored effort: 4. Executor kind: Organ. Responsible, accountable and verifier not established. Inspect retained worklog.
Runtime observation: UNKNOWN; outcome UNKNOWN; observed epoch -1. Declared owner: . Receipt
, SHA-256. Identity-bound observation is not accepted completion. Qualification and deployment remain unverified by this projection.
Learning and practice paths
No structured learning path is declared for this plan. Existing research, roadmap and worklog remain available above.
Rung dossiers — every record that bears on a rung, in one place
nx_swcompare_evidence dataeng writes it beside its stamp); every gate below reads UNRECORDED until it runs.knowledge/organ_gate.conf read from this working directory (1 rows) and from the estate root (196 rows); rows in both counted once: 0; declarations read: 197. Every declaration in either copy counts, so a stub beside the publisher can never hide the root.DE1 Execution judge for SQL over SQLite · cell NO-MATRIX-ROW · gate NO-MATRIX-ROW
| Contract | sqr_exec_judge · executor Organ · 1 u · stands on nothing |
|---|---|
| Definition of done | DONE-RULE PRE-DECLARED: the judge runs a gold query and a candidate query against the same SQLite file through nx_sqlite_rows, compares the result multisets order-insensitively, and prints EX 1 or 0 per pair with the row counts; a gate plants a candidate that returns the right rows in another order (must read 1), a candidate with one extra row (must read 0) and a candidate that errors (must read 0 with the error named); no rival oracle in the path [bird26] |
| Matrix cell | NO-MATRIX-ROW — no matrix row carries sqr_exec_judge; the ranker reads this rung as unmapped and no cell can flip for it. Add the row and the join completes on the next publish. |
| Gate | the gate join needs a matrix row: no organ is named for this rung |
| Work log | 2026-09-16 land BOARD ADMITTED 2026-09-16 from eight sources fetched over the sovereign fetcher and read from the mirrored bytes on both hosts (byte-identical): Spider 2.0, Spider2-V, BIRD, DS-1000, DABstep, LiveSQLBench, DSBench and DA-Code; the hand page that held this domain since 2026-08-13 leaves hand.roster by matrix admission; the OSWorld results sheet read the same day through nx_xlsx is the automation board's, cited there |
DE2 BIRD Mini-Dev harness · cell NO-MATRIX-ROW · gate NO-MATRIX-ROW
| Contract | dc_bird_minidev · executor Organ · 1.5 u · stands on DE1 |
|---|---|
| Definition of done | the Mini-Dev 500-pair set is mirrored from Hugging Face with its licence beside the bytes and the schema files, prepped into queries and gold TSVs the way nx_bright_prep shapes BRIGHT, and every candidate file is scored by DE1 into a receipt with n, EX and the harness hash; ACCEPT RULE: the gold set scored against itself reads EX 500 of 500 before any model is scored [bird26] |
| Matrix cell | NO-MATRIX-ROW — no matrix row carries dc_bird_minidev; the ranker reads this rung as unmapped and no cell can flip for it. Add the row and the join completes on the next publish. |
| Gate | the gate join needs a matrix row: no organ is named for this rung |
| Work log | no log row names this rung yet |
DE3 Sovereign text-to-SQL contender · cell WATCHING · gate NO-GATE-DECLARED
| Contract | sqr_text2sql · executor Organ · 3 u · stands on DE2 |
|---|---|
| Definition of done | a query writer over nx_engine_dispatch (local no-float seat, external U tier, frontier) that emits SQL for a natural-language question and a schema, scored only by DE1 on DE2; the receipt names the engine and its ceiling; the local seat's expected loss under its 96-token ceiling is published, never hidden; done when a receipt exists for all three engines |
| Matrix cell | Text-to-SQL over SQLite (BIRD dev and test) · runtime/nx_sqlite_rows.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_sqlite_rows; declare nx_sqlite_rows TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
DE4 Spider 2.0-Lite SQLite subset harness · cell NO-MATRIX-ROW · gate NO-MATRIX-ROW
| Contract | dc_spider2_lite · executor Organ · 1.5 u · stands on DE1 |
|---|---|
| Definition of done | the 135 SQLite examples of Spider 2.0-Lite (547 in total; the BigQuery and Snowflake examples wait for DE5) mirrored with licence and scored by DE1; the receipt states the subset size beside the number so a 135-example score is never read as the 547 leaderboard's [spider226] |
| Matrix cell | NO-MATRIX-ROW — no matrix row carries dc_spider2_lite; the ranker reads this rung as unmapped and no cell can flip for it. Add the row and the join completes on the next publish. |
| Gate | the gate join needs a matrix row: no organ is named for this rung |
| Work log | no log row names this rung yet |
DE5 Warehouse dialect bridge · cell WATCHING · gate NO-GATE-DECLARED
| Contract | dc_dialect_bridge · executor Organ · 3 u · stands on DE4 |
|---|---|
| Definition of done | BigQuery and Snowflake SQL run against the vendors' free quotas as ORACLE HOSTS outside the build path, results returned to DE1 for the multiset compare; the bridge is data (dialect rows), the credentials are vault leases, and a refused connection is recorded UNREACHABLE and never scored [spider226] |
| Matrix cell | Enterprise warehouse dialects (BigQuery, Snowflake) · runtime/nx_dataeng_sota_census.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_dataeng_sota_census; declare nx_dataeng_sota_census TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
DE6 DS-1000 referee replay through an isolated Python oracle host · cell WATCHING · gate NO-GATE-DECLARED
| Contract | fg_dscode · executor Organ · 2 u · stands on nothing |
|---|---|
| Definition of done | DS-1000 prompts and tests mirrored with licence; candidate completions from the engine dispatcher are executed on an isolated oracle host (Python is a third party and stays outside the estate's run path) and graded by the benchmark's own test cases, with the receipt carrying the host hash, n and the per-library breakdown [ds100022] |
| Matrix cell | Data-science code generation (DS-1000 libraries) · runtime/nx_forge.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_forge; declare nx_forge TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
DE7 Agentic analysis runs, DABstep and DA-Code · cell NO-MATRIX-ROW · gate NO-MATRIX-ROW
| Contract | ad_agent_run · executor Organ · 3 u · stands on DE6 |
|---|---|
| Definition of done | the workflow engine drives a plan, execute and observe loop whose tools are the estate's readers, profiler and approximate analytics, against the DABstep and DA-Code tasks mirrored with licence, graded by each benchmark's own answer checker on the oracle host; every run publishes completion rate and executable-code rate the way DA-Code's board does [dabstep26] [dacode24] [dsbench25] |
| Matrix cell | NO-MATRIX-ROW — no matrix row carries ad_agent_run; the ranker reads this rung as unmapped and no cell can flip for it. Add the row and the join completes on the next publish. |
| Gate | the gate join needs a matrix row: no organ is named for this rung |
| Work log | no log row names this rung yet |
DE8 dbt project tasks, Spider 2.0-DBT · cell WATCHING · gate NO-GATE-DECLARED
| Contract | dc_dbt_project · executor Organ · 3 u · stands on DE5 |
|---|---|
| Definition of done | the 68 DBT examples run through the workflow engine with the dialect bridge, graded by the benchmark's own result checks on the oracle host [spider226] |
| Matrix cell | dbt transformation projects (Spider 2.0-DBT) · runtime/nx_dataeng_sota_census.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_dataeng_sota_census; declare nx_dataeng_sota_census TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
DE9 GUI-driven data workflows, Spider2-V · cell WATCHING · gate NO-GATE-DECLARED
| Contract | br_data_workflow · executor Organ · 4 u · stands on DE7 |
|---|---|
| Definition of done | the sovereign browser drives the tools' GUIs for the 494 Spider2-V tasks on an isolated oracle host, graded by the benchmark's own checkers; the GPT-4o baseline is the published floor and the estate publishes its own success rate beside it [spider2v25] |
| Matrix cell | GUI-driven data workflows (Spider2-V) · runtime/nx_browser.nx · WATCHING decl |
| Gate | NO-GATE-DECLARED — knowledge/organ_gate.conf carries no row for nx_browser; declare nx_browser TAB <gate> and the ship loop proves it and the verdict lands here. |
| Work log | no log row names this rung yet |
dossiers=9 cells_joined=5 no_matrix_row=4 (sum 9) gates_recorded=0 gates_unrecorded=0 no_gate_declared=5 (sum 5 of cells_joined) evgates=UNSTAMPED
Capability comparisons
Compare the field, search individual capabilities and open their source and adoption evidence. Documented presence does not establish comparative quality.
Position map — centrality and distinctiveness
The four-quadrant map the field uses for brand strategy (Dawar and Bagga, HBR June 2015), re-derived from this matrix on every publish. Centrality is the share of the category's feature mass a player covers, each feature weighted by how many hold it; distinctiveness is the average lead over each rival on the rows the player holds; breadth is the depth-weighted share of the whole matrix (the bubble); depth is how deeply the rows held are held; momentum is the day-over-day move off the spine (green rising, red falling, grey until day two); the dashed path runs first day → previous day → today. Dividers are the category means. Axes are fitted to the field of play, so read the tick numerals, not the frame. Rival marks are documented presence, so a rival's position reads the record, never its quality. The picture grades its own readability below; the table beside it is the same data for a screen reader or a second method.
knowledge/compare/dataeng.cdmap (cut|x|y, cube|x|y|z); absent = the HBR defaultsbubble area = breadth · ring = momentum (green rising, red falling, grey until day two) · dashed = category means · axes fitted to the field of play: centrality 400–900, distinctiveness 200–600 of 0–1000 permil (the full range put every player in one corner)
readability of the 2D cut, self-graded by the layout ruler: label overlaps 0 · labels over marks 0 · off-canvas 0 · unresolved labels 0 · mark overlaps 0 (a fact of the data: two players that close are that close) · data spread 613 permil of the plot · quadrant words unseated 0
text contrast, measured with wcag2-ratio (floors from contrast.conf), light theme: labels 16.24 (floor 4.50) · notes 16.24 (floor 4.50) · quadrant words 5.89 (floor 4.50) · tick numerals 5.89 (floor 4.50) · axis titles 5.59 (floor 4.50) · dark theme: labels 13.78 (floor 4.50) · quadrant words 5.55 (floor 4.50) · axis titles 5.97 (floor 4.50) · dark classes under their floor 0 (one figure serves both themes: a dark shortfall is a token to fix, never a class to hide) · classes refused under their floor 0 (a refused class is not drawn; the scale classes are measured, never hidden) · export: SVG PNG (receipt, rendered by the estate's own rasteriser from this page)
readability of the 3D cube, self-graded by the layout ruler: label overlaps 0 · labels over marks 0 · off-canvas 2 · unresolved labels 0 · mark overlaps 1 (a fact of the data: two players that close are that close) · data spread 185 permil of the plot
10 panels over 5 registered axes, every pair once (the lower axis on x, the higher on y) · each panel fitted to its own field of play, first and last tick numerals shown · no labels in a small cut, the legend names the colours; the grade below reports the mark terms only (MM summed over the panels, spread averaged)
readability of the small multiples, self-graded by the layout ruler: label overlaps 0 · labels over marks 0 · off-canvas 0 · unresolved labels 0 · mark overlaps 4 (a fact of the data: two players that close are that close) · data spread 544 permil of the plot
| Player | Quadrant | centrality | distinctiveness | breadth | depth | momentum | Rows held | Days on spine | First seen |
|---|---|---|---|---|---|---|---|---|---|
| Nishi | Unconventional | 583 | 500 | 333 | 800 | 0 since 2026-09-23 | 5 | 9 | 2026-09-16 |
| Spider-Agent (xlang) | Peripheral | 583 | 250 | 222 | 666 | 0 since 2026-09-23 | 4 | 9 | 2026-09-16 |
| Agentar-Scale-SQL (Ant Group) | Peripheral | 500 | 277 | 194 | 777 | 0 since 2026-09-23 | 3 | 9 | 2026-09-16 |
| DA-Agent (DA-Code) | Peripheral | 583 | 333 | 250 | 750 | 0 since 2026-09-23 | 4 | 9 | 2026-09-16 |
| Human data engineers (BIRD) | Aspirational | 750 | 541 | 500 | 750 | 0 since 2026-09-23 | 8 | 9 | 2026-09-16 |
| Day | Matrix rows reviewed | Players recorded |
|---|---|---|
| 2026-09-16 | 13 | 5 |
| 2026-09-17 | 13 | 5 |
| 2026-09-18 | 12 | 5 |
| 2026-09-19 | 12 | 5 |
| 2026-09-20 | 12 | 5 |
| 2026-09-21 | 12 | 5 |
| 2026-09-22 | 12 | 5 |
| 2026-09-23 | 12 | 5 |
| 2026-10-08 | 12 | 5 |
players 5|matrix rows 12|feature mass 24|centrality mean 599|distinctiveness mean 380|axes 5|spine days 9 (shown 9)|rows written today 0|readability defects 2D 0 cube 2|spine knowledge/status/cdmap/dataeng.spine
Capability matrix — measured against source
◉ authored stronger claim; source present● present◐ partial○ absent? unassessed; excluded from relative comparison · click any capability for its evidence
Peer cell coverage: authored assessed 52, unassessed 0. Authored assessments are not verified outcomes.
No capabilities match this filter or search.
| Capability | Nishi | Spider-Agent (xlang) | Agentar-Scale-SQL (Ant Group) | DA-Agent (DA-Code) | Human data engineers (BIRD) |
|---|---|---|---|---|---|
Text-to-SQL over SQLite (BIRD dev and test)DARK —runtime/nx_sqlite_rows.nx EXISTS but does not declare sqr_text2sql: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. The BIRD execution-accuracy leaders are agent systems at 82.39 test against a human 72.37 [bird26]; Spider-Agent covers the SQLite third of Spider 2.0-Lite [spider226]; DA-Code grades analysis code, not SQL [dacode24]; the estate reads SQLite pages and rows (sqr_walk) and writes no SQL from language -- DE3 is the contender behind the DE1 judge and the DE2 harnessPros none measured yetCons behind Spider-Agent (xlang), Agentar-Scale-SQL (Ant Group) (leads), Human data engineers (BIRD); open contract; expected source declaration not established | ○ | ● | ◉ | ○ | ● |
Enterprise warehouse dialects (BigQuery, Snowflake)DARK —runtime/nx_dataeng_sota_census.nx EXISTS but does not declare dc_dialect_bridge: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. Spider 2.0-Snow and Lite run on Snowflake and BigQuery with Spider-Agent as the published reference agent [spider226]; the estate has no warehouse dialect and no vendor connection -- DE5 puts the vendors' free quotas outside the build path as oracle hostsPros none measured yetCons behind Spider-Agent (xlang), Human data engineers (BIRD); open contract; expected source declaration not established | ○ | ● | ○ | ○ | ● |
dbt transformation projects (Spider 2.0-DBT)DARK —runtime/nx_dataeng_sota_census.nx EXISTS but does not declare dc_dbt_project: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. 68 code-agent examples whose leader reads 65.6 and whose Spider-Agent variants read 35 to 40 [spider226]; nothing in the estate runs a dbt project -- DE8Pros none measured yetCons behind Spider-Agent (xlang), Human data engineers (BIRD); open contract; expected source declaration not established | ○ | ● | ○ | ○ | ● |
Data-science code generation (DS-1000 libraries)DARK —runtime/nx_forge.nx EXISTS but does not declare fg_dscode: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. DS-1000 grades pandas, numpy, scipy and friends by execution and holds Codex-002 at 43.3 on its own page [ds100022]; DA-Agent writes and runs analysis code [dacode24]; the forge builds and runs NishiLang, not Python -- DE6 replays through an isolated oracle hostPros none measured yetCons behind DA-Agent (DA-Code), Human data engineers (BIRD); open contract; expected source declaration not established | ○ | ○ | ○ | ● | ● |
Agentic multi-step data analysis (DA-Code, DSBench, DABstep)Measured:ad_profile exists in runtime/_hdl_build/nx_analyst_data.nx, verified at emit. The estate's profiler (ad_profile) is a tool, not an agent: it summarises a table and stops. DA-Agent plans, writes and executes across steps and is the DA-Code reference [dacode24]; DSBench and DABstep grade the same loop end to end [dsbench25] [dabstep26] -- DE7 wires the workflow engine around the estate's tools Adoption: LIB-WIRED importers=3 nonval=2 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of Spider-Agent (xlang), Agentar-Scale-SQL (Ant Group); fully adopted on the estate ladderCons behind DA-Agent (DA-Code) (leads) | ● | ○ | ○ | ◉ | ● |
Columnar and tabular file reading (Parquet, CSV, TSV, XLSX)Source present; authored stronger claim:pq_parse_rowgroups in runtime/nx_parquet_lib.nx, verified at emit. Sovereign readers with derived capacities: Parquet (thrift-compact, snappy, dictionary pages -- the reader that reproduces the BRIGHT card), CSV and TSV (nx_value_parse_csv) and XLSX (nx_xlsx read the 1077-row OSWorld sheet on admission day); every rival reads these through Python libraries. EXCEED: no third party in the read path and every buffer derived from the file, never a ceiling rival marks uncited — the Yes, Best or Part codes on this row are an observation read with no reference mark behind them Adoption: LIB-WIRED importers=3 nonval=2 — fully adopted (top of its ladder).Pros Source symbol found; stronger capability claim is authored; ahead of Spider-Agent (xlang), Agentar-Scale-SQL (Ant Group), DA-Agent (DA-Code), Human data engineers (BIRD); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ◉ | ● | ● | ● | ● |
Approximate analytics measured against exact (HLL distinct, t-digest quantiles)Measured:dfa_quantile exists in runtime/_hdl_build/nx_dataframe_approx.nx, verified at emit. Distinct counts and quantiles with a published error against the exact answer (dfa_err_permil); no rival column documents an approximate path, and the benchmarks do not grade one Adoption: LIB-WIRED importers=2 nonval=1 — fully adopted (top of its ladder).Pros Implementing source symbol found; runtime acceptance separate; ahead of Spider-Agent (xlang), Agentar-Scale-SQL (Ant Group), DA-Agent (DA-Code), Human data engineers (BIRD); fully adopted on the estate ladderCons none on the measured axes (rival marks are documented presence, not depth) | ● | ○ | ○ | ○ | ○ |
Execution-based grading, byte-exactMeasured:lb_one exists in runtime/nx_forge_local_bench.nx, verified at emit. Every benchmark on this board grades by execution; Spider-Agent and DA-Agent iterate on execution feedback [spider226] [dacode24]. The estate's judge is a fresh compile and run compared byte-exactly (lb_one), deterministic and free, the reason a sovereign harness can publish n and a hash beside every number Adoption: SOURCE-ONLY — PARTIAL: source exists, never compiled: /api/build it.Pros Implementing source symbol found; runtime acceptance separate; ahead of Human data engineers (BIRD)Cons not adopted yet: source exists, never compiled: /api/build it | ● | ● | ● | ● | ○ |
GUI-driven data workflows (Spider2-V)Open — watchingruntime/nx_browser.nx : br_data_workflow, re-measured on every compare beat. Ship that symbol and this mark flips itself; the comparewatch- plane row flips with it. 494 tasks through the GUIs of professional data tools with a GPT-4o baseline agent [spider2v25]; humans do this work daily; the sovereign browser renders pages and does not yet drive a tool -- DE9Pros none measured yetCons behind Human data engineers (BIRD) (leads); open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ◉ |
Live contamination-free evaluation (LiveSQLBench)DARK —runtime/nx_dataeng_sota_census.nx EXISTS but does not declare dc_live_set: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. Tasks rotated against training contamination, dated 2026-04 to 2026-06 [livesqlbench26]; the estate's census (dc_ground) grounds rows against organs and does not yet pull a live set; the pattern is the search board's rolling BRIGHT sweep, to be sharedPros none measured yetCons open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ○ |
Referee replay from a rival's published outputsSource present; authored stronger claim:ri_stage_split in runtime/nx_bright_reasoned_import.nx, verified at emit. A published output file is matched row by row to the estate's own task set and scored by the estate's judge before a byte of the rival's weights is pulled (landed on the search board 2026-09-16: the 1.5B rewriter's outputs measured 229 permil through our BM25). EXCEED: no rival ships a replay importer; theirs are scored by their own harnesses Adoption: REGISTERED-DARK — PARTIAL: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked).Pros Source symbol found; stronger capability claim is authored; ahead of Spider-Agent (xlang), Agentar-Scale-SQL (Ant Group), DA-Agent (DA-Code), Human data engineers (BIRD)Cons not adopted yet: callable, authorised, completed MCP invocation not established by supplied journal window; lifetime history unknown (direct execution may be recorded elsewhere; this scan does not establish lifetime use); no execution surface runs it either (clock, cron, daemon, roster, actlog and surfaced forks checked) | ◉ | ○ | ○ | ○ | ○ |
SQL issue repair (BIRD-CRITIC, SWE-SQL)DARK —runtime/nx_sqlite_rows.nx EXISTS but does not declare sqr_sql_repair: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. BIRD-CRITIC and SWE-SQL (NeurIPS 2025) grade whether a broken query is repaired [bird26]; a data engineer does this by hand; the estate has the byte-exact repair loop for NishiLang (coding CB1) and none for SQLPros none measured yetCons behind Human data engineers (BIRD); open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ● |
Interactive clarification with the user (BIRD-Interact)DARK —runtime/nx_dataeng_sota_census.nx EXISTS but does not declare dc_interact: something shipped at this path under another name, and this mark would read open forever. Repoint the contract to the real entry point (the plan rung AND this row) or rename the function; the flip follows on the next beat, and the comparewatch- plane row reads DARK until then. BIRD-Interact (ICLR 2026 oral) grades an agent that asks before it answers an ambiguous question [bird26]; humans do; no reference agent on this page documents itPros none measured yetCons behind Human data engineers (BIRD) (leads); open contract; expected source declaration not established | ○ | ○ | ○ | ○ | ◉ |
| Class | Count | Note |
|---|---|---|
| rows with rival marks | 10 | rows carrying a Yes, Best or Part in a rival column |
| cited | 9 | a reference mark resolves to a pinned mirror |
| uncited | 1 | an observation read, badged until a reference lands |
| rows, summed | 10 | RECONCILED: the parts sum to the total 10 |
Delivery and evidence
Inspect risks, technical debt, rendered observations, experiments and references. Read scope and limitations alongside every result.
Release history and work log
| When | Rung | Kind | Entry |
|---|---|---|---|
| 2026-09-16 | DE1 | land | BOARD ADMITTED 2026-09-16 from eight sources fetched over the sovereign fetcher and read from the mirrored bytes on both hosts (byte-identical): Spider 2.0, Spider2-V, BIRD, DS-1000, DABstep, LiveSQLBench, DSBench and DA-Code; the hand page that held this domain since 2026-08-13 leaves hand.roster by matrix admission; the OSWorld results sheet read the same day through nx_xlsx is the automation board's, cited there |
Person · product · place — not yet measured for this domain
knowledge/compare/dataeng.ppp (rows surface|nishi or c1..c4|label|url|connect naming OUR live surface and each rival's front door), run nx_ppp_probe domain dataeng, and this section fills itself on the next beat: the same ruler on both sides — privacy and CX (third-party hosts, tracker classes, cookies, security headers), design and longevity (design hygiene, computed WCAG contrast, render-blocking resources, unsized media, script weight, theme and motion queries), findability (landmarks, skip link, on-site search, breadcrumb, headings, internal links).Exclusive and shared capabilities
This board grades 13 capabilities: 12 exclusive to it and 1 shared with other boards. Shared rows on this page: 1 = home here 0 + home elsewhere 0 + ambiguous 0 + undecided 1 + refused 0 (partition 1); undecided = unmapped 0 + unproven 1 + unevidenced 0; homonyms 0; since the previous census: new 0, resolved 0. Census FRESH age_s=7225.
| Capability | Organ | Boards grading it | Home | Rule | Evidence | Standing on this board |
|---|---|---|---|---|---|---|
| lb_one | nx_forge_local_bench.nx | team, dataeng | — | none | unproven | undecided |
Gaps from the record — what the estate does that no board carries
The record census (nx_goalmap record) reads the invoked-tool population and every plan queue row and files each organ or directive that NO matrix, plan or gates row names. A row here is a callout the boards missed: adjudicate it onto a board or declare it infrastructure. Census state STALE (age 1396330 s), sources read 4 of 7 declared — a BLIND census is a FLOOR: unread sources can only add rows.
| kind | name | board | source | evidence |
|---|
rows shown 0|this board's directives 0|estate-wide un-boarded organs 452 (listed in full on /compare/ecosystem)|census rows 639|malformed 0 (counted, never rendered)
References
- [spider226] Spider 2.0 leaderboard page (xlang-ai, ICLR 2025 oral), fetched and read 16 Sep 2026: 632 real-world enterprise text-to-SQL workflow problems in three tracks, leaders read from the page's own tables (the newest dated entry is 2026-07-28, re-read 1 Oct 2026 from byte-identical bytes; the 16 Sep read reported 2026-03-01, which is only the first table's top-row date): Spider 2.0-Snow (547 examples on Snowflake) Genloop Sentinel Agent v2 Pro 96.70, Spider 2.0-DBT (68 code-agent examples) SignalPilot Agent 65.6, Spider 2.0-Lite (547 examples over BigQuery, Snowflake and SQLite, 135 of them SQLite) Tianqiong Data Agent plus GLM 5.2 76.23. publisher · read in our library
knowledge/fetched/cmp_dataeng_spider2_leaderboard_202610.html· pinh3cd4bb95ed0f2d9d5e3aa76cb2461f9f22c8af8ac7b7d6136e3589ce01474e61· accessed 2026-10-01 · mirrored-indexGrounds: Enterprise warehouse dialects (BigQuery, Snowflake) - [spider2v25] Spider2-V project page (xlang-ai), fetched and read 16 Sep 2026: 494 real-world tasks across complete data science and engineering workflows from warehousing to orchestration, driven through the GUIs of professional tools such as BigQuery, dbt and Airbyte; the baseline agent is GPT-4o; newest dated news 2025-01-16. publisher · read in our library
knowledge/fetched/cmp_dataeng_spider2v.html· pinh12149bba816558ee318ba33afaf2278728dc064b4458a412c557b3eed629dd05· accessed 2026-09-16 · vendor-docGrounds: GUI-driven data workflows (Spider2-V) - [bird26] BIRD text-to-SQL benchmark page (bird-bench), fetched and read 16 Sep 2026 and re-fetched 1 Oct 2026 (new top row GrainSQL, Sarim Chaudhry, Purdue University, dev 77.77 and test 82.95 dated 2026-09-26): the execution-accuracy leaderboard's top rows with their dates, DataGallery-Text2SQL (Huawei 2012 Labs) dev 78.10 and test 82.39 dated 2026-09-07, SiriusAI-SQL (Tencent) 77.77 and 82.28 dated 2026-08-22, AskData plus GPT-4o (AT and T CDO) 77.64 and 81.95 dated 2025-12-16, Agentar-Scale-SQL (Ant Group) 74.90 and 81.67 dated 2025-09-25; human performance 92.96 dev and 72.37 test; the page also announces BIRD-Interact (ICLR 2026 oral), SWE-SQL (NeurIPS 2025), BIRD-CRITIC and the Mini-Dev 500 set on Hugging Face. publisher · read in our library
knowledge/fetched/cmp_dataeng_bird_202610.html· pinhfd87838abc6a2440dca3acbb2bf89776e1054603885337659f185709955a69bc· accessed 2026-10-01 · mirrored-indexGrounds: Text-to-SQL over SQLite (BIRD dev and test) - [ds100022] DS-1000 project page (Lai et al., arXiv 2211.11501, 2022), fetched and read 16 Sep 2026: 1000 data-science questions over seven Python libraries with test cases and multi-criteria execution evaluation, perturbed against memorisation; the page names Codex-002 at 43.3 percent accuracy as the best public system and delegates the live leaderboard to a separate Eval-Arena page. publisher · read in our library
knowledge/fetched/cmp_dataeng_ds1000.html· pinhe0f1750192affe61531f842e6ccb6dcf16699a14c76667f34954fc947226d390· accessed 2026-09-16 · vendor-docGrounds: Data-science code generation (DS-1000 libraries) - [dabstep26] DABstep dataset card on Hugging Face (adyen/DABstep, api json), fetched and read 16 Sep 2026: the data agent benchmark for multi-step reasoning over real financial data; the card was last modified 2026-09-16. publisher · read in our library
knowledge/fetched/cmp_dataeng_dabstep_api.json· pinh0fbf615880d5f3a2467ee2eead334e347d5540932d29b35c1ace7d5264e58578· accessed 2026-09-16 · datasetGrounds: Agentic multi-step data analysis (DA-Code, DSBench, DABstep) - [livesqlbench26] LiveSQLBench page (a dynamic, contamination-free text-to-SQL benchmark by the BIRD team), fetched and read 16 Sep 2026: dated release items from 2026-04-25 to 2026-06-01; the Base-Full-V1 600-task set is announced on the BIRD page. publisher · read in our library
knowledge/fetched/cmp_dataeng_livesqlbench.html· pinh60d1d169ce6ff6a88e13c8a7896d1501a0e53b945ca0fe7acb0d964a2ba82933· accessed 2026-09-16 · vendor-docGrounds: Live contamination-free evaluation (LiveSQLBench) - [dsbench25] DSBench README (Jing et al., arXiv 2409.07703, ICLR 2025), fetched and read 16 Sep 2026: realistic data analysis and data modeling tasks for data-science agents; the README records its use in OpenAI's ChatGPT agent evaluation on 2025-07-17. publisher · read in our library
knowledge/fetched/cmp_dataeng_dsbench_readme.md· pinhda3cbf56031e92c7adbf361c0f046526b254a47b6d38bb900f85f778ed57ef56· accessed 2026-09-16 · source-readGrounds: Agentic multi-step data analysis (DA-Code, DSBench, DABstep) - [dacode24] DA-Code project page (EMNLP 2024, released 2024-10-03), fetched and read 16 Sep 2026: an agent data-science code generation benchmark; the page reports that even GPT-4 solves only 6.0 percent of the Spider 2.0 workflow tasks against 86.6 on Spider 1.0 and 57.4 on BIRD, and publishes a DA-Agent baseline leaderboard of completion rate and executable-code rate. publisher · read in our library
knowledge/fetched/cmp_dataeng_dacode.html· pinhe5b08d8966ba1212c9d30ab5f51a804c6e4afec8cfac11db758d62801d7c370d· accessed 2026-09-16 · vendor-docGrounds: Agentic multi-step data analysis (DA-Code, DSBench, DABstep)
generated by nx_swcompare_matrix (sovereign NishiLang organ) from knowledge/compare/dataeng.matrix · source checks show implementation presence; runtime and user-outcome evidence are reported separately · JavaScript supports page controls