DeepSeek Harness benchmark path: run, validate, report

[initial-d]

A small DeepSeek Harness path is now available for ml-quant-trading benchmark reports.

The optional plugin lives here:

https://github.com/initial-d/dsh-plugin-mlquant-benchmark

It is intentionally narrow. It gives a DSH agent four tools:

• run the fixed protocol v1 CPU benchmark;
• read the generated artifacts/benchmark-v1.json;
• validate the benchmark artifact against the protocol fields;
• draft an issue-ready benchmark report.

Install:

dsh plugin --profile web add github:initial-d/dsh-plugin-mlquant-benchmark

Suggested prompt:

Read AGENTS.md, docs/benchmarking.md, docs/deepseek_harness_recipe.md, and docs/reality_check.md.
Use the mlquant benchmark tools to run the protocol v1 CPU benchmark, validate and read the generated JSON artifact, and draft a DeepSeek Harness benchmark report. Keep the result as an engineering reproducibility benchmark, not a trading-performance claim.

Reports are welcome through the dedicated template:

https://github.com/initial-d/ml-quant-trading/issues/new?template=deepseek_harness_benchmark.yml

The useful question is simple: can a coding agent reproduce the benchmark, preserve the artifact, and avoid overstating runtime numbers as trading evidence?

[initial-d]

Follow-up: I added a broader quant-agent framing page for teams evaluating LLM trading agents or agent harnesses:

https://github.com/initial-d/ml-quant-trading/blob/main/docs/quant_agent_reproducibility_target.md

The point is to offer a fixed reproducibility target: run the documented benchmark, preserve artifacts/benchmark-v1.json, validate the artifact, and keep runtime/public-data outputs separate from trading-alpha claims.

I also opened a listing PR to Awesome LLM Quantitative Trading Papers under the financial benchmark/evaluation angle:

Tom-roujiang/Awesome-LLM-Quantitative-Trading-Papers#4