Tasks/Software Engineering/Cross-Language Porting

Dart code formatter in Haskell

Rebuild the Dart formatter in Haskell, byte-for-byte

dart_style_haskell Software Engineering Cross-Language Porting
instruction.mdthis is what the agent is given

Implement a formatter for the Dart programming language as a Haskell program. Byte-for-byte output fidelity against the reference Dart formatter (dart_style 3.1.4) is what is graded.

There is no Dart SDK in this environment or in the verifier — you are building the formatter itself, from scratch, in Haskell.

Deliverable

A buildable cabal project at /app/formatter whose executable target is named dartfmt. The verifier builds it with

cabal build --offline exe:dartfmt

locates the binary via cabal list-bin dartfmt, and runs it once per test case. If the offline build fails, the score is 0 — keep the project buildable at all times.

CLI contract

dartfmt reads Dart source on stdin and writes the formatted result to stdout, exit code 0. Flags (every invocation passes them explicitly):

  • --statement | --compilation-unit — parse the input as a single statement (function-body context) or as a whole compilation unit.
  • --page-width N — target line width in columns.
  • --indent N — number of leading spaces of extra indentation to apply to every line of output.
  • --language-version M.m — Dart language version governing both grammar and style (see below).
  • --trailing-commas automate|preserve — trailing-comma handling mode.
  • --enable-experiment <name> — may be passed zero or more times (experiment flags a Dart parser may need to accept).

Output contract: with --compilation-unit the output ends with exactly one trailing newline; with --statement it ends with no trailing newline. A non-zero exit code fails the case.

Style semantics

The reference formatter implements two styles selected by --language-version:

  • >= 3.7 — the "tall" style (the modern formatting style),
  • <= 3.6 — the "short" style (the older formatting style).

The authoritative behavioral specification is the visible test battery at /app/visible — real reference-formatter test fixtures with golden outputs produced by dart_style 3.1.4. Fixture file format: an optional first line whose | marks the page width; >>> starts a case (options and description may follow on the header line); <<< starts the expected output (optionally version-tagged <<< M.m); ### lines are comments; ×hh escapes a Unicode code point. .stmt files hold statements, .unit files whole compilation units. /app/harness.py (the same runner the hidden pipeline uses) parses all of this for you — you do not need to parse fixtures yourself.

Scoring

After your run ends, the verifier scores your binary against a hidden pipeline that is a strict superset of the visible battery: every visible case is in it (at roughly 10% of the total weight), plus corner-case fixtures, a real-world Dart package corpus, and generator-produced corpus covering modern language features (records, patterns, switch expressions, cascades, null-aware elements, multiline strings with interpolation, tight page-width geometries, comment placements, and more). Expect roughly ten times the visible case count.

Each case contributes two scoring units:

  1. match — your output equals the golden output byte-for-byte;
  2. idempotence — counted only on top of a match: feeding your own output back through your formatter (same options) must return it unchanged. A formatter whose output is not a fixed point of itself loses this unit even when the first pass matched.

Score = passed units / total units — a raw, continuous pass rate. Feedback is aggregate only (overall, per-bucket and per bucket/layout subgroup rates — enough to see WHERE misses concentrate); no per-case identities or goldens are returned.

Grading budget

Declared so you can size your implementation; all values are the verifier's.

Stage Verifier budget Your selfcheck.py
offline cabal build --offline -j8 exe:dartfmt 2400 s (exceeded ⇒ score 0) no time limit at all
per invocation of your dartfmt (one case = 1–2 invocations) 30 s, then SIGKILL 30 s
whole scoring phase over the hidden pipeline 5400 s; cases that never run count as failed n/a (945 visible cases)
whole verifier stage 12600 s wall-clock, 8 CPUs / 4 GiB your container is also 8 CPUs / 4 GiB

The per-invocation cap is now the same 30 s locally and at grading, so the self-check no longer flatters a slow formatter case by case. One asymmetry remains: the self-check puts no limit on the build, while grading caps it at 2400 s. A case killed at the per-invocation cap simply scores as a failed unit (it is never reported as a grader error), and so does every case that never ran because the whole scoring phase hit its budget.

python3 /app/selfcheck.py builds your project offline and scores it on the visible battery with exactly these semantics. The visible battery is deliberately the basic-functionality slice: matching it is necessary but far from sufficient — the hidden mass is dominated by edge cases. Handling what the visible spec implies (width-splitting decisions, comments, strings, the full expression and declaration grammar) is your job even where no visible case shows it.

What you can use

  • GHC 9.6 and cabal (offline; the package store is pre-warmed).
  • Haskell packages, pinned in /app/formatter/cabal.project.freeze: megaparsec, parsec, attoparsec, text, bytestring, containers, unordered-containers, hashable, vector, mtl, transformers, optparse-applicative, prettyprinter, split, extra, array, deepseq, directory, filepath, process (plus the GHC boot libraries), and the alex/happy parser generators. Keep the freeze file — only these versions exist in the offline store.
  • /app/visible, /app/harness.py, /app/selfcheck.py.

What you cannot use

  • Any existing Dart formatter or Dart SDK, in any form: no FFI, no embedding, no shelling out (none exists in either environment — the rule is enforced physically).
  • Network access (there is none, in either environment).
  • Anything other than Haskell for the deliverable: the graded artifact is the dartfmt binary built by cabal from your Haskell sources.

Work autonomously; do not stop to ask for input.

Metric

unit pass rate over the hidden formatter pipeline · higher is better

9,896 hidden cases x 2 units: output byte-equal to dart_style 3.1.4's golden, plus idempotence of that output.

anchorvisible setheld-outreward
Bidentity stub (cat), as shipped0.0560850.0537590.00
where reward 0.3 lands (unmeasured)0.3376310.30
where reward 0.6 lands (unmeasured)0.6215040.60
Utheoretical bound: every unit passes1.01.01.00
normalisation
m <= B0
B < m < U(m - B) / (U - B)
m >= U1

m = this run's held-out metric  ·  B = identity stub (cat), as shipped  ·  U = theoretical bound: every unit passes

B = 0.053759, U = 1.0. Bounded rate, so linear, no log or soft cap. One rate for the whole pipeline, mapped once; buckets never weighted.

Rollouts

135 minwall clock
$99.63spend
160.5Mtokens
46versions, 46 kept
0.45 0.60 0.75 0.90 1.05 $0 $20 $40 $60 $80 cumulative spend on the run visible battery pass rate (945 cases), higher is better v1 v2 v4 v6 v7 v8 v9 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v32 v33 v34 v35 v36 v37 v38 v39 v43 v45 v46 v48 v49 v50 v53 v54 v55 v56 v57
keptrevertedno scoreturning point
  1. v1Whole formatter in one pass: lexer, recursive-descent parser, Wadler Doc engineWritten end to end before any measurement: tokenizer, full-grammar parser, printer with ordered choices. Later versions add rules.0.48677220 min · $5.18
  2. v2Hanging comments, first-line-fits `choice`, forced splits, function-literal parse0.63830 min · $10.85
  3. v4Speculative parses roll back on synthesized tokens; arg-list flat/block/split0.70436 min · $13.23
  4. v6Functions and top level: type-list commas, operator== spacing, => bodies0.76442 min · $15.29
  5. v7Assignment as an ordered choice: flatten the LHS so `=` splits before its parts0.77354542 min · $15.80
  6. v8Short style: argument lists hang at +4, enum trailing comma kept not added0.78645 min · $17.25
  7. v9`const [..]` no longer a pattern; control-flow elements split header with body0.81347 min · $18.06
  8. v11Collections keep the author's line structure when they hold a line comment0.85450 min · $20.44
  9. v12Tall and short rule sets separated where they had been conflatedOne rule set served both styles. Short only preserves trailing commas and splits after the arrow; separating them unlocked short.0.87452 min · $21.61
  10. v13Hugging layouts, empty statement bodies, object-pattern declarations0.87455 min · $24.23
  11. v14Comments on dropped commas carried to the next token instead of deleted0.88658 min · $26.21
  12. v15Tall/short case-clause indent; only comments in a type push the name down0.88959 min · $27.03
  13. v16Trim trailing whitespace on every output line, as dart_style's writer does0.89566 min · $32.94
  14. v17Hand-written stress file instead of fixtures: interpolation, `await for`The battery stops driving. Hand-written Dart found a swallowed `$`, `await for` and typed patterns the fixtures never show.0.89667 min · $33.74
  15. v18Parameter-group hug beats the comma-forced layout; self-fixed-point check0.89768 min · $34.76
  16. v19Block-argument rule tightened: one block-shaped arg, every other one simple0.91270 min · $35.61
  17. v20`=>` declaration bodies use the assignment layout: break after the arrow0.91270 min · $36.59
  18. v21Block arguments anywhere when the rest are trivial; named args break at `:`0.93172 min · $37.27
  19. v22Renderer bug: a group nested in an already-flat parent re-decided its mode0.94573 min · $37.96
  20. v23Hang levels per comment kind; leading comments hoisted out of arguments0.95874 min · $38.63
  21. v24Export configuration clauses were dropped; delimited LHS patterns split alone0.9775 min · $39.54
  22. v25A trailing line comment forces the assignment to split; continuations indent0.97977 min · $40.21
  23. v26Map values take a full continuation indent after a comment; `;` follows value0.98778 min · $41.09
  24. v27Second stress corpus: digit separators, `operator >>`, `>>>=`, `is` ternaries0.98782 min · $44.50
  25. v28tokencheck.py: output must carry the input's tokens; found three silent losses0.98984 min · $46.13
  26. v29`fill` layout in the Doc engine; short-style type lists fill, not one per line0.99153485 min · $47.11
  27. v30Null-aware collection elements (`[?a]`, `{?k: v}`), absent from the battery0.99153486 min · $47.73
  28. v32Fuzz round 2: comments on multiline or interpolated strings were dropped0.99153491 min · $52.07
  29. v33`switch (x) { pat => v }` as a statement was parsed as a switch statementmeasured at v3593 min · $53.06
  30. v34Chained `is`/`as` accepted by the expression parsermeasured at v3594 min · $53.77
  31. v35`?[` split so `?[1, 2]` is a null-aware element; symbol literals less greedy0.99153495 min · $55.12
  32. v36Comment hoisting fires only when stripping really removed the commentsmeasured at v3796 min · $55.81
  33. v37Interpolation contents stay flat; short drops record-type trailing commas0.99153498 min · $57.54
  34. v38Permissive chained `==` in the parsermeasured at v39100 min · $58.88
  35. v39Symbol literals follow the grammar instead of eating `]`; pattern postfix0.991534101 min · $59.60
  36. v43A chain's target splits alone only when block-shaped; leading props redone0.991534107 min · $66.58
  37. v45Tall assignments break after `=` unless the value hugs, like AssignPiece0.991534110 min · $70.14
  38. v46`library augment 'uri.dart';` parsed and printed instead of mangled0.991534112 min · $73.70
  39. v48A named argument's value hugs unless the hardness is at its start0.995767114 min · $76.59
  40. v49`return` never breaks right after the keyword; the value simply wraps0.995767115 min · $78.08
  41. v50Third stress corpus: named extension-type constructors, bare spreads0.995767118 min · $80.74
  42. v53`x as Map<String, Object?>` mangled: `?` disambiguation cleared in type args0.995767121 min · $84.85
  43. v54`hangSep`: after a hanging block comment, a space or a break by case0.995767124 min · $88.02
  44. v55`fits` tracks the last character emitted, so a comment claims room only when hanging0.995767127 min · $90.82
  45. v56The same last-character threading in `fitsLine` and in the `fill` layout0.995767128 min · $92.01
  46. v57Housekeeping: removed a stray src/src copy, verified a clean offline build0.995767130 min · $93.96

46 of the 58 snapshots reach the timeline. One experiment was ever reverted, and it was not snapshotted. 135 min, $99.63; visible 0.487 -> 0.996.

On the hidden set

held-out metricreward
identity stub (cat), as shipped0.0537590.00
where reward 0.3 lands (unmeasured)0.3376310.30
where reward 0.6 lands (unmeasured)0.6215040.60
theoretical bound: every unit passes1.01.00
this run0.6119640.5899
385 minwall clock
$146.20spend
232.7Mtokens
22versions, 22 kept
0 0.2 0.4 0.6 0.8 1.0 $0 $40 $80 $120 cumulative spend on the run visible battery pass rate (945 cases), higher is better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21
keptrevertedno scoreturning point
  1. v0Untouched ByteString passthrough; exposed the installed GHC for an offline build0.0560852 min · $0.54
  2. v1Unicode-aware Dart lexer, CLI contract, lexical spacing, block and switch indentPassthrough replaced by a lexer plus a spacing and indent model; layout deferred until the basics were right.0.4592599 min · $1.95
  3. v2Empty collections, shorthand colons, unary spacing, case dedent, Unicode trailing ws0.52275112 min · $2.56
  4. v3Width-aware group layout for delimiters; automated flat/split trailing commasPage width enters the design: a recursive measurement decides per delimiter group whether it fits on one line.0.60423317 min · $3.56
  5. v4Direct-group analysis; old-style forced splitting separated from width splitting0.70370425 min · $6.85
  6. v5Nested-comment fixed points; width-aware assignment, arrow, binary and cascade breaks0.75132336 min · $11.10
  7. v6Nested generics, ternary and case hanging, continuation frames, initializer geometry0.82539755 min · $19.39
  8. v7Control-block continuation leakage, the dominant pattern error, fixed0.92804268 min · $26.73
  9. v8Braced interpolations formatted recursively; a real numeric lexer; arrow edge breaks0.96190579 min · $29.47
  10. v9Triple-literal rows split from interpolation code; type angles become nested frames0.99365185 min · $31.35
  11. v10A split parenthesized pattern owns the assignment break; extension-type comments0.99576791 min · $33.25
  12. v11Comment-forced extension type-angle splits; representation comment spacing kept194 min · $34.31
  13. v12Own matrix: 513 sources x 5 widths x 2 styles x 2 trailing modes, 20,520 invocationsThe visible battery hit 1.0 at v11, so validation moved to a self-built matrix of widths, styles and trailing modes.1104 min · $37.60
  14. v13Null-aware elements, generator keywords, symbol literals, triple-string normalization1119 min · $43.23
  15. v14Vector-backed token stream replaces linked-list scans; 25 KB timeout down to 0.63 s1140 min · $56.64
  16. v15Linear reversed-buffer block-comment scan; 1 MB pathological input in 0.23 s1143 min · $58.08
  17. v16Switch-label alignment, `>=`/`>>=` generic closers, directives split from decls1166 min · $64.52
  18. v1761-source modern-grammar corpus; dense sweep of every width from 10 to 1001199 min · $75.30
  19. v18UTF-16 layout columns, chain-decision caching, manual Flutter and HTTP audit1253 min · $97.17
  20. v19Constructor initializers own commas; quadratic annotation and header scans removed1310 min · $119.84
  21. v20Delimiter enclosure and matching precomputed once for every group query1327 min · $125.13
  22. v21Closure initializers join the constructor body; comma sizing under inline layout1384 min · $146.08

Visible saturated at 1.0 by v11, 94 min in; the last ten versions chased self-built matrices and speed. 385 min, $146.20, none reverted.

On the hidden set

held-out metricreward
identity stub (cat), as shipped0.0537590.00
where reward 0.3 lands (unmeasured)0.3376310.30
where reward 0.6 lands (unmeasured)0.6215040.60
theoretical bound: every unit passes1.01.00
this run0.4815080.4521
30 minwall clock
$8.46spend
11.5Mtokens
12versions, 12 kept
0 0.15 0.30 0.45 0.60 0.75 $0 $2 $4 $6 $8 cumulative spend on the run visible battery pass rate (945 cases), higher is better v0 v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11
keptrevertedno scoreturning point
  1. v0Inherited identity stub; established the offline build and the scoring loop0.0560851 min · $0.28
  2. v1Dart-aware lexer, whitespace normalization, block indent, output terminatorsThe baseline missed inputs as elementary as `1<2;`, so the bottleneck was normalization, not parsing; layout could come later.0.1841274 min · $0.55
  3. v2Generic-angle classification, type adjacency, block breaks, empty blocks0.335455 min · $0.77
  4. v3Recursive delimiter fit measurement; width and trailing-comma multiline groupsPage width enters the design: a recursive measurement decides per delimiter group whether it fits on one line.0.4455037 min · $1.14
  5. v4Nullable/conditional punctuation, nested generics, call vs record splitting0.55767211 min · $1.71
  6. v5Operator/conditional/case breaks, collection pressure, labels, enum groups0.58306914 min · $2.27
  7. v6Label-vs-colon parsing, constructor splits, adjacent strings, prefix ops0.62751318 min · $3.05
  8. v7Repaired the v6 short-style regressions: generic decls as blocks, blank lines0.63915319 min · $3.58
  9. v8Unary classification, operator overflow, persistent switch and chain layoutLayout state that persists across a whole construct, plus precedence and overflow accounting, replacing token-at-a-time decisions.0.68148124 min · $5.22
  10. v9Comment anchors and source-line-preserving packing for commented collections0.69629626 min · $6.37
  11. v10Pattern-colon and generic fixes, spread/set layout, chain and cascade splits0.71534429 min · $7.47
  12. v11Cascades split only on multiple cascades or overflow; tall-only empty bodies0.72698430 min · $8.29

Twelve versions in thirty minutes for $8.46, every one kept - the lineage never forks. The agent stopped after v11 with budget left.

On the hidden set

held-out metricreward
identity stub (cat), as shipped0.0537590.00
where reward 0.3 lands (unmeasured)0.3376310.30
where reward 0.6 lands (unmeasured)0.6215040.60
theoretical bound: every unit passes1.01.00
this run0.3175020.2787
65 minwall clock
-spend
-tokens
5versions, 5 kept
0.555 0.562 0.570 0.577 0 50 100 150 200 agent step (this harness reports no tokens or timestamps) visible battery pass rate (945 cases), higher is better v1 v2 v3 v4 v5
keptrevertedno scoreturning point
  1. v1Lexer, parser and piece renderer from scratch; identity fallback when parsing failsFull grammar and dart_style-shaped list groups instead of fixture hacks; unparsable input is echoed back unchanged.own subset, 297/556
  2. v2Comments bound only until the next token; list commas ahead of them; flat root renderThe first span-based attachment stole later `//` comments, so each comment was re-bound to the token it precedes.0.553439
  3. v3`<` is type args only before `(`; ctor/enum trailing-comma split; empty-list commentsComparisons like `1<2` had been parsed as type arguments, so elementary expressions could never match.0.565079
  4. v4Collection trailing-comma split, Function optional params, short mixed block args0.57672
  5. v5Comments around binary operators; trailing comments no longer force a break0.578836

Five snapshots in 65 min, none reverted; visible 0.553 at v2 to 0.579 at v5. A type-arg split tried after v4 was built away, never snapshotted.

On the hidden set

held-out metricreward
identity stub (cat), as shipped0.0537590.00
where reward 0.3 lands (unmeasured)0.3376310.30
where reward 0.6 lands (unmeasured)0.6215040.60
theoretical bound: every unit passes1.01.00
this run0.2607110.2187
23 minwall clock
$2.70spend
14.9Mtokens
0versions, 0 kept

No trajectory curve: this run left no comparable self-check measurement, so there is nothing to plot against spend. The versions and what each one changed are below.

No version was ever snapshotted and no log kept: one Main.hs edited in place, eleven self-checks from the 0.056 stub to 0.420 in 23 min.

On the hidden set

held-out metricreward
identity stub (cat), as shipped0.0537590.00
where reward 0.3 lands (unmeasured)0.3376310.30
where reward 0.6 lands (unmeasured)0.6215040.60
theoretical bound: every unit passes1.01.00
this run0.1787590.1321