Commit 396c22c
refactor(reports): generate the pricing mirror and split the reports layer (#176)
* feat(pricing): 1/5 — generate the evalboard rate table from pricing.py
The frontend's rate table was a hand-copy of `_PRICING`, kept honest by five
layers of bookkeeping: a regex parser that re-read pricing.py at test time, a
meta-guard against that regex silently narrowing, a DELIBERATELY_UNMIRRORED
exemption set, a staleness guard for the exemption set, and a comment asking the
next reader to keep the set honest. It still shipped a real bug — claude-sonnet-5,
gpt-5.6-sol, gpt-5.6-terra and gpt-5.6-luna sat in the exemption set under "the
evalboard never runs them" while appearing ~32k / ~2k / ~17k / ~2k times in the
run corpus, so those runs rendered "—" for cost with nothing failing.
If a test can read the table, a generator can emit it. `make pricing-mirror` now
renders evalboard/lib/pricing.generated.ts from pricing.builtin_rates(), and
CE065 re-renders and diffs it. All five bookkeeping layers delete.
The one thing the exemption set encoded that was NOT bookkeeping — that three
OpenRouter models must stay unpriced on the frontend so runs.ts apportions the
provider's real per-call bill instead of a static estimate — becomes data on the
rate itself: ModelPricing.per_request_billing. The generator skips those rows, so
nothing has to remember them. The field is defaulted and last, so 4-positional
construction (including the out-of-tree coder_eval_uipath rate card) is unaffected;
it participates in register_pricing's anti-shadow comparison, which is correct.
Generating the table deliberately ADDS gpt-5.4-mini, gpt-5.4-nano, gpt-5.4-pro and
gpt-5.5-pro to the frontend — the four exemption entries that were pure drift.
Also corrects every surface that told a reader to hand-edit pricing.ts or named the
deleted parity test: the Makefile and CI-job comments, litellm/README.md's
"register in both tables" step, CLAUDE.md, and two stale evalboard consumer comments.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SGRvWucHgYgpcGhmyHrdBg
* refactor(reports): 2/5 — two DRY fixes and hoist 18 function-local imports
SLOW_PARAMS_PREVIEW_CHARS was defined in reports.py and used by reports_html.py,
while the module that DEFINES it truncated with the literal 50 twice. The variant
Token Usage card re-derived TokenUsage.total_tokens inline — a fourth home for
arithmetic the model already owns. Both now have exactly one definition; neither
changes a rendered byte at today's values, which is the point.
Of the 19 function-local imports across the three report modules, 15 had their
module edge at top level already, so hoisting them is adding a name to an existing
line. Three add a genuinely new top-level edge (analysis in reports.py and
reports_html.py, reports_html in reports_experiment.py — verified acyclic).
ONE stays deferred: reports.py's `from .criteria import ...`. Importing
coder_eval.criteria runs pkgutil auto-discovery with registry side effects, which
hoisting would put on the path of every `import coder_eval.reports`. It now carries
a comment saying so, and a subprocess test asserts coder_eval.criteria stays out of
sys.modules — the decision is pinned rather than remembered.
Also documents the run.json `input_tokens` seam at its writer: the key carries
uncached_input_tokens, NOT the derived TokenUsage.input_tokens total, and
evalboard/lib/runs.ts depends on that reading. Same word, two quantities — the name
is fixed by the run.json contract and cannot change without breaking archived runs.
No golden value is re-baselined: the test diff is additions only.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SGRvWucHgYgpcGhmyHrdBg
* refactor(reports): 3/5 — split reports_stats.py into stats, result_metrics and helpers
reports_stats.py had become three unrelated modules sharing a file: a
distribution-free numeric core, a set of EvaluationResult metrics the
ORCHESTRATOR consumes mid-run, and the report renderers' own helpers. The
orchestrator importing a "reports" module for a number it needs during a run is
the layering wart; the file was the reason it had to.
stats.py pure statistics. Imports NOTHING from coder_eval — asserted
by a test, not by convention, because that is the whole
reason the numeric core can be reasoned about in isolation.
result_metrics.py turn_time_buckets, visible_turn_count, has_final_reply,
expected_turns_overage. Consumed by the orchestrator during
a run as well as by the reporters, so it is not a report.
Deliberately NOT folded into timing.py, which has no
EvaluationResult dependency and is imported by every agent.
reports_stats.py what is left: report-shaped helpers over variant and
experiment results, plus the display formatters
(fmt_mean_sd, fmt_p) — presentation, not computation.
Its cycle rationale is restated rather than deleted: it is LIVE, not historical.
experiment -> html -> helpers, so folding the helpers into the experiment
reporter would close a cycle.
Also moves eval_result_to_task_dict to run_record.py. It writes one run.json row
— a run-record serializer, not a report — and its placement was the only reason
orchestration/batch.py reached into the reports layer at all. Moving it is what
lets Phase 4's CE066 allowlist be purely writers instead of carrying a serializer
as a permanent exception.
Every one of the 21 moved definitions is byte-identical to its pre-move form,
verified by AST comparison. A characterization test pins the full run.json row
against a snapshot captured before the move; the non-finite sensor is retargeted
to coder_eval.stats and still guards the same 7 functions.
Two test files that held only tests for moved names are merged into the files
named for those names, rather than left behind as misnamed orphans.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SGRvWucHgYgpcGhmyHrdBg
* refactor(reports): 4/5 — reports/ package, durations.py, and CE066
The five reports_*.py modules become a package (markdown/html/experiment/junit/
helpers) whose __init__ re-exports the public writer surface. Private names stay
private: the 12 that tests reach for are imported from their submodule, so the
package's API is not a function of its test suite.
`format_ms` moves to durations.py. formatting.py imports claude_agent_sdk for the
payload formatters, and the reports package should not reach through an SDK-shaped
module for a 14-line duration formatter. NOTE this does NOT make the package
SDK-free — models/agent_config.py imports ClaudeAgentOptions and every report
module needs models. The docstring and tests say what is actually true rather than
what the plan hoped.
CE066 pins the layering the split establishes: core may import only the package's
public WRITERS. An allowlist, not a denylist, so a new report helper is banned from
core by default. It checks BOTH the absolute and the relative spelling — the first
draft matched only `node.module`, which for `from ..reports import X` holds
"reports" with the dots in `node.level`, so it fired on neither of the two real
edges in the tree and its tests passed because they used the absolute form. An
unrun assertion is documentation, not enforcement.
The core-layer predicate moves to a shared _layers.py that CE004 and CE066 both
read, so a package added to one cannot escape the other. It is stated as "every
module directly under src/coder_eval/ is core": naming only orchestrator.py left
result_metrics.py exempt — the module CE066's own fix message tells you to move
your metric into.
Also fixes a latent packaging break: .gitignore's bare `reports/` (meant for run
output) matched src/coder_eval/reports/, leaving __init__.py untracked and building
a wheel with zero files under coder_eval/reports/. Anchored to /reports/.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: 5/5 — retarget every reports_* reference and record the rationale
The tree, the review-command module lists and a handful of prose references still
named modules that no longer exist. CLAUDE.md's `reports_html.py` line also called
it "the evalboard's static twin" — a parity promise nobody was keeping: it has ~40
private renderers and nothing tests it against the 31k-line evalboard. That line
goes with the entry.
Adds `## The reports package` to .claude/architecture-notes.md, which is where
CLAUDE.md's preamble sends a reader for rationale. It records the CE066 layering,
why stats.py / result_metrics.py / run_record.py each sit outside the package, the
relative-import trap CE066's first draft fell into, and — the part a future reader
would otherwise re-litigate — the DECISION NOT to build a shared section-data layer,
with the measurement behind it: only 1 of the 4 "duplicated" section pairs shares an
input shape, and the differences in the rest are per-surface presentation, not drift.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(lint): CE004 never fired on the relative import spelling
CE004 matched `node.module` against `^coder_eval\.cli`, but a relative import
keeps its dots in `node.level` and leaves the rest in `node.module` — so
`from ..cli import run_command` arrives as `level=2, module="cli"` and matched
nothing. The relative form is this codebase's dominant idiom, so the rule has
been guarding roughly nothing since it was written, with its tests green because
they used the absolute spelling.
Found while fixing the identical bug in CE066 during the reports split. Two rules
independently falling into the same trap is the definition of a shared helper, so
the matching moves to `_layers.imports_package` next to the core-layer predicate
both rules already share — the same reasoning that put `is_core_path` there.
The helper also fixes a narrower bug the regex had: `^coder_eval\.cli` prefix-
matched `coder_eval.client`. Matching is now on a package boundary.
Adds the regression tests both rules were missing: every spelling of a banned
import, plus the prefix-bleed case.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* chore: record two deferred harness candidates from the reports consolidation
Both are real gaps with a known shape but neither is ~30 minutes of work: a
hand-edit guard for generated surfaces needs a checksum gate rather than a diff,
and a prose-path resolver has the same tree-parsing problem the plan already
measured and declined for CLAUDE.md alone.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: code review fixes for the reports consolidation
Three findings from the cross-phase review, all whole-diff-only:
CE004 and CE066 both missed a THIRD import spelling. `imports_package` handled
the absolute and relative forms, and `is_bare_package_import` handled
`from . import reports` — but `from coder_eval import reports` is level=0 with
module="coder_eval", which matched neither. It binds the package, so every
attribute read through it was invisible to both rules. That spelling is a real
in-tree idiom (`from coder_eval import __version__`), which the tests now pin as
the negative case alongside the positives.
reports/__init__ published nine names nothing imports. The export list was
measured before packaging as "every name imported from a reports module", which
at that point included the five modules importing EACH OTHER; those became
intra-package `from .markdown import …` and need no re-export. 25 -> 16, and the
docstring no longer claims more than it delivers.
Stale cross-repo pointers the phase greps missed by stopping at the src/
boundary: two evalboard tests still cite the deleted pricing-parity test as the
authority on rates (it is CE065 now), runs.ts and variants.ts still name
reports_experiment.py / reports_junit.py, and a workflow comment does too. The
runs.ts one matters most — it is the far half of the `input_tokens` seam Phase 2
deliberately documented at its writer.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(lint): 1/4 — is_core_path covers the whole core layer
`_layers.is_core_path` was a denylist of ten directory names plus a
top-level-module regex, so `isolation/` — the `driver: docker` evaluation
path — was invisible to BOTH CE004 and CE066. Its unanchored
`[/\\]coder_eval[/\\]` also classified a repo-root file as core, because
this project's own checkout directory is named `coder_eval`.
Replace it with the anchored allowlist its own docstring already described:
everything under `src/coder_eval/` is core except the `cli/` and `reports/`
packages, so a new subpackage is core by default rather than exempt until
someone notices. Uses the established `(?:^|[/\\])src[/\\]coder_eval[/\\]`
spelling rather than a new variant — the defect being fixed was a regex that
disagreed with its siblings.
Core-set delta: +isolation/__init__.py, +isolation/docker_runner.py,
+resources/__init__.py; zero removals; zero new CE004/CE066 violations.
Four CE004 fixtures in tests/test_lint_runner.py built paths without a `src/`
segment; three stop firing under the anchor and the fourth passes vacuously,
so all four are re-anchored. `make lint` does not run that file.
`TestCoreLayerMembership` pins the non-core set against the real filesystem
and pins the residual `~/src/coder_eval` collision as unreachable rather than
asserting it away.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(lint): 2/4 — declare CE044 and CE065 to ruff, and pin the id space
`[tool.ruff.lint] external` is what stops ruff reporting RUF102 for a
`# noqa: CE0xx` it does not own. It was two ids short: CE065, and CE044 —
which the review missed and only prototyping the sensor found.
Both are `@pytest.mark.lint` classes rather than BaseRules, and that is
exactly why they slipped: `TestRuffExternalCoversEveryRule` already guarded
this list, but derived the known ids from `ALL_RULES` alone, so it was blind
to half the rule space it was meant to cover.
Extend that class rather than adding a second one beside it: `_known()` now
unions `ALL_RULES` with the `class TestCE\d{3}` ids scraped from this file,
and `test_no_dead_entry_survives` asserts the other direction, so a declared
id for a deleted rule fails too. Both messages name pyproject.toml and the
offending ids.
`tests/lint/runner.py`'s id-claiming note carried the same hand-maintained
enumeration and had fallen behind CE044 identically; it now points at the
grep instead of listing the ids.
Nothing was red for want of these two entries — no `# noqa: CE044` or
`# noqa: CE065` exists in the tree — so the fix is pre-emptive.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(reports): 3/4 — cover the HTML slowest-commands truncation branch
`reports/html.py:825` — the `params_preview += "..."` arm — was the one
uncovered line in the renderer. Its markdown twin got both a truncation and
a boundary test when SLOW_PARAMS_PREVIEW_CHARS was introduced; the HTML side
got neither.
Adds the twin pair. Both read the constant rather than the literal 50, so
they survive a change to it, and both were mutation-checked: flipping `>` to
`>=` fails the boundary case, deleting the ellipsis arm fails the truncation
case.
`str(dict)` emits single quotes that `_esc` renders as `'`, so the raw
cell is longer than the preview. The assertion unescapes before measuring;
the renderer's escaping is untouched.
No source change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: 4/4 — retarget prose that names deleted report modules
The consolidation's own greps could not reach these: nine present-tense
references in tests/ naming `reports.py`, `reports_html.py`,
`reports_experiment.py` or `reports_junit.py` — modules that no longer
exist — plus CE053's fixture default, whose `: str = ` spacing hid it from
a `filepath="src/…"` grep.
Classified rather than sed'd: eight further sites keep their wording because
they date a past incident in past tense, name a test file that still exists,
or are the deliberate `reports_html` local alias. All eight verified
unedited.
Also deletes CLAUDE.md's `optimize/` tree entry. The directory is absent
from HEAD and from main; it lives only on the unmerged
`feat/plugin-optimize-skill` branch, so the tree as documented did not
match the tree as shipped.
No source file changed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: code review fixes for the reports-consolidation review fixes
Two Medium findings, each flagged independently by a reviewer:
- `_layers.py` spelled the 30-char package prefix twice, in the module whose
own docstring argues a second copy of a definition is how the two drift.
`_NON_CORE` now derives from `_PKG.pattern`, so widening one cannot leave
the other behind — a divergence no test could have seen, because both
assertions only ever feed src-layout paths. Compiled pattern unchanged.
- The trailing `[/\\]` in `(cli|reports)[/\\]` is what keeps a top-level
module whose name merely STARTS with `reports` or `cli` core. Nothing
pinned it: deleting the separator left all 738 tests green. Pins added for
`reports_legacy.py` and `cli_helpers.py`; both fail under that mutation.
Also, from the same reviews:
- Pin the reachability argument that made the `~/src/coder_eval` residual
safe. It lived only as docstring prose; adding CE004 or CE066 to
`_ALSO_SCAN_TESTS` would hand them a whole `tests/` tree that matches
`_PKG` on an ordinary clone layout. Now one assertion.
- `tests/lint/runner.py`'s new grep told the next author to run
`^class Test(CE\d{3})`. GNU and BSD `grep -E` read `\d` as a literal `d`
and report zero hits — "no ids taken", the exact miss the note exists to
prevent. Respelled `[0-9]{3}`.
- CLAUDE.md's tree audit ran one way only. Deleting the phantom `optimize/`
entry was right, but `errors/` and `plugins.py` exist and were absent —
and `plugins.py` is the SPI the "Adding a New Agent" section points at.
- Record `ce048`'s near-variant of the shared path regex in the candidates
entry.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(lint): CE067 — CLAUDE.md's tree must name every top-level package member
Harness loop, closed in-session. The directory tree is the map an assistant
reads before touching this package, and it drifted in both directions across
two consecutive plans while only one direction was ever audited: a phantom
`optimize/` row survived for a directory that lives solely on an unmerged
branch, and the sweep that removed it walked entries -> filesystem, so it
could not see that `errors/` and `plugins.py` were missing — the second being
the plugin SPI that "Adding a New Agent" tells you to use.
Asserts both directions over the top-level rows only; nested rows stay
illustrative, so a new sibling module is not a forced docs edit. Proven to
fire each way: re-adding the `optimize/` row and deleting the `plugins.py`
row each fail with the offending name.
Writing it also found a third omission the hand audit missed — `__init__.py`,
here deliberately ignored along with the build and typing markers.
Claims id 067; the Phase 2 parity sensor required the pyproject entry
immediately, which is the sensor doing its job.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(lint): drop a personal path and de-duplicate the isolation pin
No test datum should carry a developer's home directory. The repo-root case
now uses `/home/dev/src/exp/coder_eval/conftest.py`, which keeps the shape
that mattered — a `src` component that is NOT the package's parent, so it
exercises the anchor rather than merely the absence of `src`.
`ISOLATION` and a verbatim three-line docstring were copied into both the
CE004 and CE066 test classes. Hoisted to one module-level `CORE_ISOLATION`
with the rationale stated once, so the two pins cannot drift to different
paths. Verified the pins still bite: re-exempting `isolation/` in `_NON_CORE`
fails four tests.
Also renames `test_ce008_skips_files_outside_scope` to `ce009` — it sits in
the CE009 block and exercises `YamlModelsForbidExtras` (CE009), while CE008's
own tests cover `ReadTextExplicitEncoding`. Pre-existing mislabel, flagged in
review.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* test(lint): drop two function-local json imports that shadow the module one
`tests/test_custom_lint.py` imports `json` at module level, and two tests
re-imported it inside the function body. CodeQL flagged the one this PR
added (in the pricing-mirror test); the other, in the activation-rows test,
predates the PR and has the same shape, so both go.
Ruff has no rule for a repeated import inside a function body, which is
why `make check` never saw either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(lint): CE004 checks reports/ — stop borrowing CE066's exemption set
CE004 and CE066 shared one predicate, `is_core_path`, whose exemption set is
`{cli, reports}`. That set is CE066's: the reports package may reach into
itself. CE004's is only `{cli}` — the reports package runs without the CLI
(the orchestrator writes a task report mid-run), so a `cli` import added
inside `reports/` closes a cli -> orchestration -> reports -> cli cycle, and
CE004 would have stayed silent. Latent, not live: widening the scope finds
0 violations.
The rules still share what must not drift — the anchored package regex and
the `cli/` boundary, now `is_package_path` and `is_cli_path` — but each
states its own exemptions. CE066 keeps `is_core_path`; CE004's scope is the
package minus `cli/`.
Tests, each mutation-checked against CE004 going back to the core predicate:
- `test_the_reports_package_is_in_scope` — a `cli` import in reports/ violates.
- `TestCoreLayerMembership.test_ce004_scope_is_every_module_outside_cli` runs
the RULE at every real module path. Its first draft recomputed the scope
from the helpers and passed under that mutation, so it now calls the rule.
- `test_the_reports_package_itself_stays_exempt` pins that CE066's scope did
not widen with CE004's.
Closes the deferred harness candidate; architecture notes updated.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(lint): resolve a relative import against the importing file
CE004 and CE066 matched `node.module` against the bare package name whenever
`node.level` was non-zero, so `from .reports import x` inside `orchestration/` —
i.e. `coder_eval.orchestration.reports` — read as the reports layer, and
`from ...reports import x` from a sub-package read as it too although it escapes
`coder_eval` entirely. Latent: nothing in the tree is nested that way today.
`_absolute_module` now resolves the dots against the file's own package, so both
rules compare one fully-qualified name in either spelling.
The CE004 scope probe moves to the absolute spelling: a relative one resolves
against the importing file, so `..cli` names the cli layer only from inside a
sub-package and that test would measure depth instead of scope.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix: reconcile the reports split with main's docs restructure
Rebase fallout from #174/#177, which replaced CLAUDE.md's fenced directory
tree with `ls` plus selective bullets, split `.claude/architecture-notes.md`
into `.claude/notes/`, and added the `make docs-budget` prose gate.
- Drop CE067. It asserted that CLAUDE.md's tree names every top-level package
member; that tree no longer exists, and the bullets that replaced it are
deliberately not exhaustive, so the rule's invariant is now false by design.
The surviving half — no bullet may name a path that does not exist — is
recorded in .claude/harness-candidates.md rather than rebuilt here.
- Move the reports-package rationale into `.claude/notes/reporting.md`, and
retarget `reports_stats` / `reports_junit` in notes/ to their new homes.
- Bring `result_metrics.py` and `run_record.py` under the prose budget by
adopting main's trimmed wording for the comments and docstrings they
inherited, including its `.claude/notes/` pointers.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(pricing): carry the exemption set into the generated mirror
The generator kept only `per_request_billing` as an exemption axis, so it
reproduced three of the hand-copy's seven exclusions and silently priced the
other four. Those four were not drift: `gpt-5.4-mini`, `gpt-5.4-nano`,
`gpt-5.4-pro` and `gpt-5.5-pro` were enumerated exemptions in the parity test
this branch deletes. Pricing them is a behaviour change smuggled in as a
refactor — latent only because no experiment config references them today.
The exemption set encoded two different claims and they now survive separately.
That a routed model MUST NOT be priced is a fact about the rate, and stays on
`ModelPricing.per_request_billing`. That a heavy frontier variant is not worth
pricing on the board is a fact about the FRONTEND, so it is
`DELIBERATELY_UNMIRRORED` beside the generator.
What the hand-copy got wrong was not having an exemption set but letting it go
stale unnoticed, so the deleted test's staleness guard comes back as
`_assert_exemptions_are_live`: an id that has left `pricing.py` fails
`make pricing-mirror` and CE065 rather than sitting there silencing nothing.
The generated table now reproduces the hand-copy's 52 keys exactly — no
additions, no removals. CE065 asserts both axes from their declared sources,
and the evalboard's consumption guard pins the four as unpriced.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>1 parent 30f78f4 commit 396c22c
85 files changed
Lines changed: 3326 additions & 1627 deletions
File tree
- .claude
- commands
- notes
- .github/workflows
- docs
- agents
- evalboard/lib
- __tests__
- litellm
- src/coder_eval
- agents
- cli
- models
- orchestration
- reports
- streaming
- tests
- lint
- rules
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
348 | 348 | | |
349 | 349 | | |
350 | 350 | | |
351 | | - | |
| 351 | + | |
352 | 352 | | |
353 | 353 | | |
354 | 354 | | |
| |||
401 | 401 | | |
402 | 402 | | |
403 | 403 | | |
404 | | - | |
| 404 | + | |
405 | 405 | | |
406 | 406 | | |
407 | 407 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
450 | 450 | | |
451 | 451 | | |
452 | 452 | | |
453 | | - | |
| 453 | + | |
454 | 454 | | |
455 | 455 | | |
456 | 456 | | |
| |||
859 | 859 | | |
860 | 860 | | |
861 | 861 | | |
862 | | - | |
| 862 | + | |
863 | 863 | | |
864 | 864 | | |
865 | 865 | | |
| |||
916 | 916 | | |
917 | 917 | | |
918 | 918 | | |
| 919 | + | |
| 920 | + | |
| 921 | + | |
| 922 | + | |
| 923 | + | |
| 924 | + | |
| 925 | + | |
| 926 | + | |
| 927 | + | |
| 928 | + | |
| 929 | + | |
| 930 | + | |
| 931 | + | |
| 932 | + | |
| 933 | + | |
| 934 | + | |
| 935 | + | |
| 936 | + | |
| 937 | + | |
| 938 | + | |
| 939 | + | |
| 940 | + | |
| 941 | + | |
| 942 | + | |
| 943 | + | |
| 944 | + | |
| 945 | + | |
| 946 | + | |
| 947 | + | |
| 948 | + | |
| 949 | + | |
| 950 | + | |
| 951 | + | |
| 952 | + | |
| 953 | + | |
| 954 | + | |
| 955 | + | |
| 956 | + | |
| 957 | + | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
48 | 48 | | |
49 | 49 | | |
50 | 50 | | |
51 | | - | |
| 51 | + | |
52 | 52 | | |
53 | 53 | | |
54 | 54 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
184 | 184 | | |
185 | 185 | | |
186 | 186 | | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
| 195 | + | |
| 196 | + | |
| 197 | + | |
| 198 | + | |
| 199 | + | |
| 200 | + | |
| 201 | + | |
| 202 | + | |
| 203 | + | |
| 204 | + | |
| 205 | + | |
| 206 | + | |
| 207 | + | |
| 208 | + | |
| 209 | + | |
| 210 | + | |
| 211 | + | |
| 212 | + | |
| 213 | + | |
| 214 | + | |
| 215 | + | |
| 216 | + | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
| 220 | + | |
| 221 | + | |
| 222 | + | |
| 223 | + | |
| 224 | + | |
| 225 | + | |
| 226 | + | |
| 227 | + | |
| 228 | + | |
| 229 | + | |
| 230 | + | |
| 231 | + | |
| 232 | + | |
| 233 | + | |
| 234 | + | |
| 235 | + | |
| 236 | + | |
| 237 | + | |
| 238 | + | |
| 239 | + | |
| 240 | + | |
| 241 | + | |
| 242 | + | |
| 243 | + | |
| 244 | + | |
| 245 | + | |
| 246 | + | |
| 247 | + | |
| 248 | + | |
| 249 | + | |
| 250 | + | |
| 251 | + | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
187 | 256 | | |
188 | 257 | | |
189 | | - | |
| 258 | + | |
190 | 259 | | |
191 | | - | |
| 260 | + | |
192 | 261 | | |
193 | 262 | | |
194 | 263 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
147 | | - | |
| 147 | + | |
148 | 148 | | |
149 | 149 | | |
150 | 150 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
181 | 181 | | |
182 | 182 | | |
183 | 183 | | |
184 | | - | |
185 | | - | |
186 | | - | |
187 | | - | |
188 | | - | |
189 | | - | |
190 | | - | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
| 193 | + | |
| 194 | + | |
191 | 195 | | |
192 | 196 | | |
193 | | - | |
194 | | - | |
195 | | - | |
196 | | - | |
| 197 | + | |
| 198 | + | |
197 | 199 | | |
198 | 200 | | |
199 | 201 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
463 | 463 | | |
464 | 464 | | |
465 | 465 | | |
466 | | - | |
| 466 | + | |
467 | 467 | | |
468 | 468 | | |
469 | 469 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
61 | 61 | | |
62 | 62 | | |
63 | 63 | | |
64 | | - | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
65 | 67 | | |
66 | 68 | | |
67 | 69 | | |
| |||
0 commit comments