From 172af2812b841bea15f1caf864f952f22c47056c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Nov 2026 18:28:02 +0000 Subject: [PATCH 1/9] Keep the samples behind each guided result, and stop a slow sample rate recording a no A draft held only the verdict of each guided test and a sentence about it. When the automatic-blip detector turned out to read the driver's carried throttle as a blip, nothing in the drafts could be re-read, so 84 records were flagged for a second drive. The drive now keeps the last six seconds (at most 300 samples) behind every result the driver accepts, with the channels as SimHub published them and the resting throttle and clutch, and the draft stores them under an optional `telemetry` block, one sample to a line. A stored trace fed back through the detector reproduces its answer, which the .NET tests assert. The gear count has no trace, and retried or skipped attempts are not kept. The importer stages exactly what it did before; a Python test holds it to that. The detector also never asked how fast samples were arriving. An automatic blip or an ignition cut lasts about a tenth of a second, so at ten samples a second one can pass between two samples, and its absence was still recorded as "no automatic blip". The drive now measures the median rate per attempt and for the whole drive. Below 20 samples a second an unseen blip is `unknown` rather than `no`, and the cut and blip method text says why. A spike that was sampled still counts at any rate, and a rate is only reported from eight or more intervals. The plugin adapter change is four lines carrying the telemetry from the guided results into the saved draft. It was not compiled here: that needs the SimHub SDK. The core and its tests were built with Mono as C# 5. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- PRIVACY.md | 6 +- docs/contribution-intake.md | 4 +- docs/verification-observations.md | 26 ++ .../v1/verification-observation.schema.json | 103 +++++++ simhub/AsDriven.Core.Tests/Program.cs | 175 ++++++++++++ .../AsDriven.Core/GuidedVerificationDrive.cs | 259 +++++++++++++++++- .../AsDriven.Core/VerificationObservation.cs | 142 ++++++++++ simhub/AsDriven.Plugin/VerificationControl.cs | 6 +- tests/test_import_observation.py | 38 +++ 9 files changed, 747 insertions(+), 12 deletions(-) diff --git a/PRIVACY.md b/PRIVACY.md index 7b5be6d3..33ac5dd7 100644 --- a/PRIVACY.md +++ b/PRIVACY.md @@ -53,8 +53,10 @@ verification. It stores the following information locally: Verification drafts may contain the observer name entered in the form, exact simulator and vehicle identity, game and SimHub versions, test observations, -notes, and timestamps. They are never added to the curated database or sent -elsewhere automatically. +notes, and timestamps. A draft from a guided drive also holds a few seconds of +the telemetry each accepted test was decided from: gear, pedal positions, +engine speed and torque, and road speed. They are never added to the curated +database or sent elsewhere automatically. The plugin can open a public contribution form only after the user explicitly asks it to. It does not attach or transmit the draft; the user chooses the file diff --git a/docs/contribution-intake.md b/docs/contribution-intake.md index 2daa3b7b..94a2ea7f 100644 --- a/docs/contribution-intake.md +++ b/docs/contribution-intake.md @@ -30,7 +30,9 @@ A full public draft can contain: - the chosen observer name or handle and exact observation timestamp; - simulator, game, client, and dataset versions; -- exact telemetry names, internal identifiers, test results, and notes; and +- exact telemetry names, internal identifiers, test results, and notes; +- for a guided drive, a few seconds of gear, pedal, engine and speed samples + around each accepted test; and - for Assetto Corsa, the driven package's content id, author, declared version, and fingerprint. diff --git a/docs/verification-observations.md b/docs/verification-observations.md index 45589f70..df341c00 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -113,6 +113,32 @@ is detected only from a shift-local engine-torque collapse while throttle demand stays high. Imports of older ACC drafts degrade any throttle-derived cut answer to `unknown` because ACC exposes no engine-torque channel through SimHub. +## Recorded telemetry and sample rate + +A draft keeps the samples each accepted guided result was decided from, under +the optional `telemetry` block. Each trace holds at most the last six seconds +or 300 samples of one test, ending at the sample that settled it, with the +channels exactly as SimHub published them. The gear count has no trace because +the driver ends it. A retried or skipped attempt is not kept, and a draft +answered by hand has no `telemetry` block at all. + +The traces exist so a detector fault found later does not cost a second drive. +Until 2026-08-17 the automatic-blip test read the driver's own carried throttle +as a blip, and because drafts held only verdicts, every exposed record had to be +flagged in `research/auto-blip-premeasurement.json` for a re-drive. A corrected +detector can instead be run over the stored samples. `resting_throttle` and +`resting_clutch` record what those channels read with the car stopped, which is +what decides whether a later reading can be attributed to the pedal. + +The draft also records the median sample rate, for the drive and for each +attempt. An automatic blip or an ignition cut lasts about a tenth of a second, +so at ten samples a second one can pass between two samples unseen. Below 20 +samples a second the absence of a blip is therefore recorded as `unknown` +rather than `no`, and the cut and blip method text says so. A blip or cut that +was sampled is still recorded at any rate, and clutchless-shift acceptance does +not depend on the rate at all. A rate is reported only from at least eight +intervals; with fewer the result is taken at face value. + Cockpit mechanism and wheel fields carry an orange `OPTIONAL · REVIEW` badge until the tester supplies them. These fields improve a draft but never block submission: a partial, falsifiable observation is preferable to a guess. If a diff --git a/schema/v1/verification-observation.schema.json b/schema/v1/verification-observation.schema.json index 92615361..8836c77d 100644 --- a/schema/v1/verification-observation.schema.json +++ b/schema/v1/verification-observation.schema.json @@ -274,6 +274,43 @@ "type": "string" } }, + "telemetry": { + "description": "The samples the guided-drive results were decided from, kept so a corrected detector can be run over a drive that was already taken instead of asking for it again. Optional: a draft answered by hand, or written before the plugin recorded them, has none. Channels are stored as the telemetry client published them, before any resting offset is subtracted.", + "type": "object", + "additionalProperties": false, + "required": [ + "traces" + ], + "properties": { + "sample_rate_hz": { + "description": "Median sample rate across the whole drive; null when too few samples arrived to measure it.", + "type": [ + "number", + "null" + ] + }, + "resting_throttle": { + "description": "Published throttle with the car stopped and untouched before the first test; null when that moment was not seen. A channel already open at rest is not reporting the pedal alone.", + "type": [ + "number", + "null" + ] + }, + "resting_clutch": { + "description": "Published clutch at the same moment.", + "type": [ + "number", + "null" + ] + }, + "traces": { + "type": "array", + "items": { + "$ref": "#/$defs/telemetryTrace" + } + } + } + }, "review_status": { "type": "string", "enum": [ @@ -291,6 +328,72 @@ } }, "$defs": { + "telemetryTrace": { + "description": "One guided test's samples, ending at the sample that settled the result. Only an attempt whose result was accepted into the draft is kept; a retried or skipped attempt is not.", + "type": "object", + "additionalProperties": false, + "required": [ + "test", + "outcome", + "columns", + "samples" + ], + "properties": { + "test": { + "type": "string", + "enum": [ + "move_off", + "full_throttle_upshift", + "lifted_upshift", + "coast_downshift", + "manual_blip_downshift" + ] + }, + "outcome": { + "description": "Whether the detector accepted the maneuver. A rejected first stage is evidence too: it is what establishes that a lift or a blip was needed.", + "type": "string", + "enum": [ + "accepted", + "rejected" + ] + }, + "sample_rate_hz": { + "description": "Median sample rate over this attempt; null when too few samples arrived to measure it. Below 20 the absence of a brief cut or blip is recorded as unknown rather than no.", + "type": [ + "number", + "null" + ] + }, + "columns": { + "description": "What each position in a sample holds. t_ms counts from the first sample of the trace; clutch, throttle and brake are percentages; engine_started is 1 or 0. A channel with no usable value is null.", + "const": [ + "t_ms", + "gear", + "clutch", + "throttle", + "brake", + "rpm", + "speed_kmh", + "engine_torque", + "engine_started" + ] + }, + "samples": { + "type": "array", + "minItems": 1, + "items": { + "type": "array", + "minItems": 9, + "items": { + "type": [ + "number", + "null" + ] + } + } + } + } + }, "wheelOpenTopState": { "type": "string", "enum": [ diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 2cb204b8..12f22aad 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1547,6 +1547,141 @@ private static int Main(string[] args) Equal("yes", guidedResults.ClutchlessDownshift, "prefills accepted clutchless downshift"); Equal("yes", guidedResults.AutomaticBlip, "prefills telemetry-supported automatic blip"); + // The drive keeps the samples each accepted result was + // decided from. A draft that holds only verdicts cannot be + // re-examined when a detector fault is found later, which + // is what left 84 records waiting on a second drive after + // the automatic-blip fault. + True(guidedResults.Telemetry != null, "keeps the samples behind the guided results"); + Equal( + "move_off,full_throttle_upshift,coast_downshift", + string.Join(",", guidedResults.Telemetry.Traces.Select(trace => trace.Test)), + "keeps one trace per accepted test and none for the driver-ended gear count"); + True( + guidedResults.Telemetry.Traces.All(trace => trace.Outcome == "accepted"), + "records that each kept attempt was accepted"); + True( + guidedResults.Telemetry.Traces.All(trace => !trace.SampleRateHz.HasValue), + "reports no sample rate from a handful of samples"); + { + // Sixty samples a second with no blip is a measured no. + GuidedVerificationDrive fast = CoastDownshiftDrive(now, 16, 1000, false); + Equal("no", fast.GetResults().AutomaticBlip, + "a blip that never appeared at a fast sample rate is recorded as no"); + GuidedTestTrace fastTrace = fast.GetResults().Telemetry.Traces + .Single(trace => trace.Test == "coast_downshift"); + True(fastTrace.SampleRateHz.HasValue + && Math.Abs(fastTrace.SampleRateHz.Value - 62.5) < 0.01, + "measures the attempt's median sample rate"); + True(fast.GetResults().Telemetry.SampleRateHz.HasValue, + "measures a sample rate for the drive as a whole"); + + // Ten a second can step straight over one, so the same + // absence proves nothing and the answer stays unknown. + GuidedVerificationDrive slow = CoastDownshiftDrive(now, 100, 1000, false); + Equal("yes", slow.GetResults().ClutchlessDownshift, + "slow telemetry still establishes that the downshift was accepted"); + Equal("unknown", slow.GetResults().AutomaticBlip, + "an unseen blip at a slow sample rate stays unknown rather than no"); + True(slow.GetResults().AutomaticBlipMethod.Contains("too slow to rule out an automatic blip"), + "says why the slow-rate blip answer is unknown"); + + // A spike that was sampled is a spike at any rate. + GuidedVerificationDrive slowBlip = CoastDownshiftDrive(now, 100, 1000, true); + Equal("yes", slowBlip.GetResults().AutomaticBlip, + "a blip seen at a slow sample rate is still recorded"); + + // A long run-up must not grow the draft without bound. + GuidedVerificationDrive longCoast = CoastDownshiftDrive(now, 16, 20000, false); + GuidedTestTrace longTrace = longCoast.GetResults().Telemetry.Traces + .Single(trace => trace.Test == "coast_downshift"); + True(longTrace.Samples.Length <= 300, "caps the samples kept for one attempt"); + True(longTrace.Samples.Any(sample => sample.Gear == 3) + && longTrace.Samples.Any(sample => sample.Gear == 4), + "keeps the end of the attempt, where the shift happened"); + + // The same holds for the torque cut on the upshift. + GuidedVerificationDrive slowUpshift = new GuidedVerificationDrive(); + slowUpshift.Start(6); + for (int guard = 0; guard < 40 + && slowUpshift.GetSnapshot().Title != "Full-throttle upshift"; guard++) + { + slowUpshift.AddSample(GuidedSample(now, 3, 0, 0, 4000, 80, 100, true)); + slowUpshift.Next(); + } + for (int tick = 1; tick <= 12; tick++) + { + slowUpshift.AddSample(GuidedSample( + now.AddMilliseconds(tick * 100), tick < 12 ? 3 : 4, 0, 95, 6000, 120, 300, true)); + } + True(slowUpshift.GetSnapshot().ResultReady, "accepts the slow-rate full-throttle upshift"); + True(slowUpshift.GetSnapshot().Result.Contains("too slow to rule out an automatic cut"), + "does not present held torque at a slow sample rate as the absence of a cut"); + + // The draft stores the trace one sample to a line, and + // a corrected detector can be run over it later. + GuidedDriveTelemetry recorded = slow.GetResults().Telemetry; + GuidedTestTrace recordedCoast = recorded.Traces + .Single(trace => trace.Test == "coast_downshift"); + recordedCoast.Samples[0].EngineTorque = double.NaN; + verificationDraft.Telemetry = recorded; + string traceDirectory = Path.Combine( + Path.GetTempPath(), "as-driven-trace-" + Guid.NewGuid().ToString("N")); + try + { + string tracePath = VerificationObservationWriter.WriteDraft( + traceDirectory, verificationDraft); + string traceText = File.ReadAllText(tracePath); + True(traceText.Contains("[0,4,0,0,0,4500,80,null,1]"), + "writes each sample on one line and a missing torque value as null"); + JObject traceJson = JObject.Parse(traceText); + JToken storedCoast = traceJson["telemetry"]["traces"] + .Single(trace => (string)trace["test"] == "coast_downshift"); + Equal(9, ((JArray)storedCoast["columns"]).Count, "names every stored channel"); + Equal(recordedCoast.Samples.Length, ((JArray)storedCoast["samples"]).Count, + "stores every kept sample"); + Equal(10.0, (double)storedCoast["sample_rate_hz"], "stores the attempt's sample rate"); + + GuidedVerificationDrive replay = new GuidedVerificationDrive(); + replay.Start(6); + for (int guard = 0; guard < 40 + && replay.GetSnapshot().Title != "Downshift without pedal input"; guard++) + { + replay.AddSample(GuidedSample(now, 4, 0, 0, 4000, 80, 100, true)); + replay.Next(); + } + foreach (JToken row in (JArray)storedCoast["samples"]) + { + replay.AddSample(GuidedSample( + now.AddMilliseconds((double)row[0]), + (int)row[1], + (double)row[2], + (double)row[3], + (double)row[5], + (double)row[6], + row[7].Type == JTokenType.Null ? double.NaN : (double)row[7], + (int)row[8] == 1, + (double)row[4])); + } + True(replay.GetSnapshot().ResultReady, "a stored trace settles the same test when replayed"); + replay.Next(); + Equal(slow.GetResults().AutomaticBlip, replay.GetResults().AutomaticBlip, + "replaying a stored trace reproduces the recorded answer"); + + string redactedTracePath = VerificationObservationWriter.WriteRedactedCopy(tracePath); + True(File.ReadAllText(redactedTracePath).Contains("[0,4,0,0,0,4500,80,null,1]"), + "the redacted copy keeps the trace and its one-line samples"); + } + finally + { + verificationDraft.Telemetry = null; + if (Directory.Exists(traceDirectory)) + { + Directory.Delete(traceDirectory, true); + } + } + } + // A result-ready phase still receives live telemetry while // it waits for Next. Preserve that current gear when the // coast test opens, or a first-attempt downshift whose blip @@ -2909,6 +3044,46 @@ private static GuidanceSnapshot LoadSyntheticGuidance( return database.Match("Automobilista2", telemetryName); } + /// + /// Drives a fresh guided session to the coast-downshift test, coasts in + /// 4th for , takes 3rd, and accepts + /// the result. Samples arrive every . + /// + private static GuidedVerificationDrive CoastDownshiftDrive( + DateTime now, + int intervalMilliseconds, + int coastMilliseconds, + bool blip) + { + GuidedVerificationDrive drive = new GuidedVerificationDrive(); + drive.Start(6); + for (int guard = 0; guard < 40 + && drive.GetSnapshot().Title != "Downshift without pedal input"; guard++) + { + drive.AddSample(GuidedSample(now, 4, 0, 0, 4000, 80, 100, true)); + drive.Next(); + } + for (int elapsed = 0; + elapsed <= coastMilliseconds + 3000 && !drive.GetSnapshot().ResultReady; + elapsed += intervalMilliseconds) + { + bool shifted = elapsed >= coastMilliseconds; + bool blipping = blip && shifted && elapsed < coastMilliseconds + 150; + drive.AddSample(GuidedSample( + now.AddMilliseconds(elapsed), + shifted ? 3 : 4, + 0, + blipping ? 60 : 0, + shifted ? 6000 : 4500, + shifted ? 78 : 80, + 100, + true)); + } + True(drive.GetSnapshot().ResultReady, "the simulated coast downshift settles"); + drive.Next(); + return drive; + } + private static GuidedTelemetrySample GuidedSample( DateTime timestamp, int gear, diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index eb8a071d..7e48113a 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -1,4 +1,5 @@ using System; +using System.Collections.Generic; using System.Globalization; namespace AsDriven.Core @@ -38,6 +39,11 @@ public sealed class GuidedDriveResults public string FullThrottleUpshift { get; set; } public string CoastDownshift { get; set; } public string EvidenceNote { get; set; } + /// + /// The samples each accepted result was decided from. Null until the + /// drive has recorded at least one attempt. + /// + public GuidedDriveTelemetry Telemetry { get; set; } public GuidedDriveResults() { @@ -60,6 +66,36 @@ public GuidedDriveResults Clone() } } + /// + /// The samples one guided test was decided from. + /// + /// A draft used to keep only the verdict and a sentence about it, so a + /// detector fault found later could be corrected only by driving every + /// affected car again: the automatic-blip fault of 2026-08-17 left 84 + /// records flagged for exactly that reason. With the samples kept, a + /// corrected detector can be run over the drive that was already taken. + /// + public sealed class GuidedTestTrace + { + public string Test { get; set; } + /// Whether the detector accepted the maneuver: "accepted" or "rejected". + public string Outcome { get; set; } + /// Median rate over this attempt, or null when too few samples arrived to say. + public double? SampleRateHz { get; set; } + public GuidedTelemetrySample[] Samples { get; set; } + } + + public sealed class GuidedDriveTelemetry + { + /// Median rate over the whole drive, or null when it could not be measured. + public double? SampleRateHz { get; set; } + /// Published throttle with the car stopped and untouched, when that moment was seen. + public double? RestingThrottle { get; set; } + /// Published clutch at the same moment. + public double? RestingClutch { get; set; } + public GuidedTestTrace[] Traces { get; set; } + } + public sealed class GuidedDriveSnapshot { public bool Visible { get; set; } @@ -99,6 +135,27 @@ public sealed class GuidedVerificationDrive private static readonly TimeSpan RunningShiftRefusalWindow = TimeSpan.FromMilliseconds(900.0); + // What an attempt keeps for the draft. The newest samples are the ones + // the result was decided from, so the window is trimmed from the old + // end. Five traces of this size stay well inside the observation + // intake limit of 256 KB. + private static readonly TimeSpan TraceWindow = TimeSpan.FromSeconds(6.0); + private const int TraceMaximumSamples = 300; + + // A rate is only reported from enough intervals to take a median of. + // Fewer than this is a handful of samples, not a measurement. + private const int RateMinimumIntervals = 8; + + /// + /// The slowest telemetry at which "nothing was seen" still means + /// nothing happened. An automatic blip or an ignition cut lasts on the + /// order of a tenth of a second, so at ten samples a second - which is + /// what an unlicensed SimHub delivers - one can pass entirely between + /// two samples. A spike that was seen is still a spike at any rate; + /// only its absence needs the rate to vouch for it. + /// + public const double MinimumRateForNegativeHz = 20.0; + private enum Phase { Idle, @@ -193,6 +250,17 @@ private enum Phase private double _upshiftMaximumThrottle; private double _upshiftMinimumThrottle; private DateTime? _upshiftGearChangedUtc; + private readonly Queue _attemptSamples = + new Queue(); + private readonly List _traces = new List(); + /// + /// Whole-millisecond gaps between consecutive samples across the drive, + /// the last bucket holding everything slower. A median from this is + /// not moved by a paused game or a stalled frame. + /// + private readonly int[] _sessionIntervals = new int[1001]; + private int _sessionIntervalCount; + private DateTime? _previousSampleUtc; public void Start(int? suggestedForwardGears, string simulator = null) { @@ -212,6 +280,10 @@ public void Start(int? suggestedForwardGears, string simulator = null) _restingClutchSeen = false; _restingThrottle = 0.0; _restingThrottleSeen = false; + _traces.Clear(); + Array.Clear(_sessionIntervals, 0, _sessionIntervals.Length); + _sessionIntervalCount = 0; + _previousSampleUtc = null; ResetTrace(); } } @@ -238,11 +310,13 @@ public void AddSample(GuidedTelemetrySample sample) lock (_sync) { _lastSample = sample; + RecordInterval(sample); if (!IsTestPhase(_phase) || (_resultReady && _phase != Phase.GearCount)) { return; } + RecordAttemptSample(sample); UpdateTrace(sample); DetectResult(sample); } @@ -344,7 +418,9 @@ public GuidedDriveResults GetResults() { lock (_sync) { - return _results.Clone(); + GuidedDriveResults results = _results.Clone(); + results.Telemetry = BuildTelemetry(); + return results; } } @@ -626,7 +702,7 @@ private void DetectUpshift(GuidedTelemetrySample sample, bool requireLift) : "Full-throttle clutchless upshift accepted. " + (automaticCut ? "A shift-local torque interruption was detected while throttle demand stayed high." - : CutEvidenceSummary()) + : CutEvidenceSummary() + SlowTelemetrySummary("an automatic cut")) + VehicleClutchSummary(); SetResult(true, automaticCut, message); } @@ -773,7 +849,8 @@ private void DetectDownshift(GuidedTelemetrySample sample, bool manualBlip) + _downshiftArmedThrottle.ToString("0", CultureInfo.InvariantCulture) + "% (the threshold is 15%)." + RestingThrottleSummary() - : "No automatic throttle spike was detected.") + : "No automatic throttle spike was detected." + + SlowTelemetrySummary("an automatic blip")) ) + VehicleClutchSummary()); return; @@ -817,6 +894,7 @@ private void SetResult(bool accepted, bool automaticAction, string message) private void AcceptResultAndAdvance() { RememberVehicleClutchTelemetry(); + KeepAttemptTrace(); if (_attemptNotTested) { // Nothing was measured, so every downstream flag stays as it @@ -879,9 +957,15 @@ private void AcceptResultAndAdvance() // unknown rather than no when nothing could be measured. _results.AutomaticBlip = !AutomaticBlipIsMeasurable() ? "unknown" - : ThrottleChannelFollowsPedal() - ? (_automaticActionObserved ? "yes" : "no") - : "unknown"; + : !ThrottleChannelFollowsPedal() + ? "unknown" + : _automaticActionObserved + ? "yes" + // A spike that was never sampled looks the same as + // one that never happened. The message already says + // the telemetry was too slow to tell, and the value + // has to agree with it. + : (TelemetryTooSlowForNegative() ? "unknown" : "no"); _results.AutomaticBlipMethod = _result; MoveTo(Phase.Complete); } @@ -1213,8 +1297,163 @@ private static double DriveRatio(GuidedTelemetrySample sample) return sample.Rpm / Math.Max(sample.SpeedKmh, 1.0); } + private void RecordInterval(GuidedTelemetrySample sample) + { + if (_previousSampleUtc.HasValue) + { + double milliseconds = (sample.TimestampUtc - _previousSampleUtc.Value).TotalMilliseconds; + // A repeated or backward timestamp is not an interval. + if (milliseconds > 0.0) + { + int bucket = (int)Math.Min(Math.Round(milliseconds), _sessionIntervals.Length - 1); + _sessionIntervals[bucket]++; + _sessionIntervalCount++; + } + } + _previousSampleUtc = sample.TimestampUtc; + } + + private void RecordAttemptSample(GuidedTelemetrySample sample) + { + // The gear count is ended by the driver and decided from the + // highest gear alone, so there is nothing in its samples to + // re-examine later. + if (_phase == Phase.GearCount) + { + return; + } + _attemptSamples.Enqueue(sample); + while (_attemptSamples.Count > TraceMaximumSamples + || sample.TimestampUtc - _attemptSamples.Peek().TimestampUtc > TraceWindow) + { + _attemptSamples.Dequeue(); + } + } + + /// + /// Median sample rate over the current attempt, or null when too few + /// samples arrived to measure one. + /// + private double? AttemptSampleRateHz() + { + var intervals = new List(); + DateTime? previous = null; + foreach (GuidedTelemetrySample sample in _attemptSamples) + { + if (previous.HasValue) + { + double milliseconds = (sample.TimestampUtc - previous.Value).TotalMilliseconds; + if (milliseconds > 0.0) + { + intervals.Add(milliseconds); + } + } + previous = sample.TimestampUtc; + } + if (intervals.Count < RateMinimumIntervals) + { + return null; + } + intervals.Sort(); + return 1000.0 / intervals[intervals.Count / 2]; + } + + private double? SessionSampleRateHz() + { + if (_sessionIntervalCount < RateMinimumIntervals) + { + return null; + } + int seen = 0; + for (int milliseconds = 0; milliseconds < _sessionIntervals.Length; milliseconds++) + { + seen += _sessionIntervals[milliseconds]; + if (seen * 2 > _sessionIntervalCount) + { + return 1000.0 / Math.Max(milliseconds, 1); + } + } + return null; + } + + /// + /// Whether this attempt was sampled too slowly for the absence of a + /// brief event to mean anything. An unmeasured rate disqualifies + /// nothing, the same way an unseen resting throttle does not. + /// + private bool TelemetryTooSlowForNegative() + { + double? rate = AttemptSampleRateHz(); + return rate.HasValue && rate.Value < MinimumRateForNegativeHz; + } + + private string SlowTelemetrySummary(string what) + { + double? rate = AttemptSampleRateHz(); + if (!rate.HasValue || rate.Value >= MinimumRateForNegativeHz) + { + return string.Empty; + } + return " Telemetry arrived at about " + + rate.Value.ToString("0", CultureInfo.InvariantCulture) + + " samples a second, which is too slow to rule out " + what + + ": one can pass between two samples. This stays unknown rather than no." + + " A licensed SimHub updates fast enough to settle it."; + } + + private static string TraceName(Phase phase) + { + switch (phase) + { + case Phase.MoveOff: return "move_off"; + case Phase.FullThrottleUpshift: return "full_throttle_upshift"; + case Phase.LiftedUpshift: return "lifted_upshift"; + case Phase.CoastDownshift: return "coast_downshift"; + case Phase.ManualBlipDownshift: return "manual_blip_downshift"; + default: return null; + } + } + + /// + /// Keeps the samples behind a result the driver has just accepted. + /// Retry and Skip discard theirs: only an attempt that was written + /// into the results has anything to be checked against later. + /// + private void KeepAttemptTrace() + { + string name = TraceName(_phase); + if (name == null || _attemptNotTested || _attemptSamples.Count == 0) + { + return; + } + _traces.RemoveAll(trace => string.Equals(trace.Test, name, StringComparison.Ordinal)); + _traces.Add(new GuidedTestTrace + { + Test = name, + Outcome = _attemptAccepted ? "accepted" : "rejected", + SampleRateHz = AttemptSampleRateHz(), + Samples = _attemptSamples.ToArray() + }); + } + + private GuidedDriveTelemetry BuildTelemetry() + { + if (_traces.Count == 0) + { + return null; + } + return new GuidedDriveTelemetry + { + SampleRateHz = SessionSampleRateHz(), + RestingThrottle = _restingThrottleSeen ? (double?)_restingThrottle : null, + RestingClutch = _restingClutchSeen ? (double?)_restingClutch : null, + Traces = _traces.ToArray() + }; + } + private void ResetTrace() { + _attemptSamples.Clear(); _baselineGear = 0; _downshiftCandidateGear = 0; _downshiftCandidateAtUtc = DateTime.MinValue; @@ -1421,8 +1660,12 @@ private string ResultSummary(Phase phase) { return "Downshift detected; automatic blip unavailable in telemetry"; } - return _automaticActionObserved - ? "Downshift and automatic blip detected" + if (_automaticActionObserved) + { + return "Downshift and automatic blip detected"; + } + return TelemetryTooSlowForNegative() + ? "Downshift detected; telemetry too slow to rule out a blip" : "Downshift detected; no automatic blip"; case Phase.ManualBlipDownshift: return _attemptAccepted diff --git a/simhub/AsDriven.Core/VerificationObservation.cs b/simhub/AsDriven.Core/VerificationObservation.cs index 80878f49..a50217d3 100644 --- a/simhub/AsDriven.Core/VerificationObservation.cs +++ b/simhub/AsDriven.Core/VerificationObservation.cs @@ -53,6 +53,11 @@ public sealed class VerificationObservationDraft public string WheelOpenTop { get; set; } public string WheelNotes { get; set; } public string[] EvidenceNotes { get; set; } + /// + /// The samples the guided results were decided from. Null for a draft + /// answered entirely by hand. + /// + public GuidedDriveTelemetry Telemetry { get; set; } } public static class VerificationObservationWriter @@ -124,6 +129,7 @@ public static string WriteRedactedCopy(string originalPath) throw new FileNotFoundException("Observation draft was not found.", fullPath); } JObject payload = JObject.Parse(File.ReadAllText(fullPath)); + CompactTraceRows(payload); payload["observer"] = "Anonymous"; payload.Remove("implementation"); const string disclosure = @@ -326,9 +332,145 @@ public static JObject CreatePayload(VerificationObservationDraft draft) { payload.Add("evidence_notes", new JArray(notes)); } + JObject telemetry = TelemetryPayload(draft.Telemetry); + if (telemetry != null) + { + payload.Add("telemetry", telemetry); + } return payload; } + private static readonly string[] TraceColumns = + { + "t_ms", "gear", "clutch", "throttle", "brake", "rpm", "speed_kmh", + "engine_torque", "engine_started" + }; + + private static readonly HashSet TraceTests = new HashSet( + new[] + { + "move_off", "full_throttle_upshift", "lifted_upshift", + "coast_downshift", "manual_blip_downshift" + }, + StringComparer.Ordinal); + + /// + /// The recorded samples as the draft stores them: channels exactly as + /// SimHub published them, before any resting offset is subtracted, so + /// a later detector can make its own decision about them. + /// + private static JObject TelemetryPayload(GuidedDriveTelemetry telemetry) + { + if (telemetry == null || telemetry.Traces == null) + { + return null; + } + var traces = new JArray(); + foreach (GuidedTestTrace trace in telemetry.Traces) + { + if (trace == null || trace.Samples == null || trace.Samples.Length == 0) + { + continue; + } + RequireChoice(trace.Test, TraceTests, "Telemetry trace test"); + DateTime start = trace.Samples[0].TimestampUtc; + var rows = new JArray(); + foreach (GuidedTelemetrySample sample in trace.Samples) + { + // One sample per line. Indented like the rest of the file, + // a trace would be nine lines a sample and most of the + // draft would be whitespace. + rows.Add(new JRaw("[" + + Number(Math.Round((sample.TimestampUtc - start).TotalMilliseconds), 0) + "," + + sample.Gear.ToString(CultureInfo.InvariantCulture) + "," + + Number(sample.Clutch, 1) + "," + + Number(sample.Throttle, 1) + "," + + Number(sample.Brake, 1) + "," + + Number(sample.Rpm, 0) + "," + + Number(sample.SpeedKmh, 1) + "," + + Number(sample.EngineTorque, 1) + "," + + (sample.EngineStarted ? "1" : "0") + + "]")); + } + traces.Add(new JObject + { + { "test", trace.Test }, + { + "outcome", + string.Equals(trace.Outcome, "accepted", StringComparison.Ordinal) + ? "accepted" + : "rejected" + }, + { "sample_rate_hz", NullableNumber(trace.SampleRateHz) }, + { "columns", new JArray(TraceColumns) }, + { "samples", rows } + }); + } + if (traces.Count == 0) + { + return null; + } + return new JObject + { + { "sample_rate_hz", NullableNumber(telemetry.SampleRateHz) }, + { "resting_throttle", NullableNumber(telemetry.RestingThrottle) }, + { "resting_clutch", NullableNumber(telemetry.RestingClutch) }, + { "traces", traces } + }; + } + + /// + /// JSON has no NaN or infinity, and a simulator that publishes no + /// torque can supply either. Null says the channel had no value. + /// + private static string Number(double value, int decimals) + { + if (double.IsNaN(value) || double.IsInfinity(value)) + { + return "null"; + } + return Math.Round(value, decimals).ToString( + decimals == 0 ? "0" : "0.#", CultureInfo.InvariantCulture); + } + + private static JToken NullableNumber(double? value) + { + if (!value.HasValue || double.IsNaN(value.Value) || double.IsInfinity(value.Value)) + { + return JValue.CreateNull(); + } + return new JValue(Math.Round(value.Value, 1)); + } + + /// + /// Puts each recorded sample back on one line after a draft has been + /// parsed, which otherwise spreads it over nine. + /// + private static void CompactTraceRows(JObject payload) + { + JObject telemetry = payload["telemetry"] as JObject; + JArray traces = telemetry == null ? null : telemetry["traces"] as JArray; + if (traces == null) + { + return; + } + foreach (JToken trace in traces) + { + JArray rows = trace["samples"] as JArray; + if (rows == null) + { + continue; + } + for (int index = 0; index < rows.Count; index++) + { + if (rows[index] is JArray) + { + rows[index] = new JRaw(rows[index].ToString(Formatting.None)); + } + } + } + } + private static string CreateObservationId( string simulator, string telemetryName, diff --git a/simhub/AsDriven.Plugin/VerificationControl.cs b/simhub/AsDriven.Plugin/VerificationControl.cs index 9cf70efd..5daf133f 100644 --- a/simhub/AsDriven.Plugin/VerificationControl.cs +++ b/simhub/AsDriven.Plugin/VerificationControl.cs @@ -27,6 +27,7 @@ internal sealed partial class VerificationControl : Border private string _guidedAutomaticCutMethod = string.Empty; private string _guidedAutomaticBlipMethod = string.Empty; private string _guidedEvidenceNotes = string.Empty; + private GuidedDriveTelemetry _guidedTelemetry; private string _savedDraftPath = string.Empty; // No form control: these are the first stage of a two-stage test, which // the drive either ran or did not. A reviewer editing the second stage @@ -318,6 +319,7 @@ private void ResetForm() _guidedAutomaticCutMethod = string.Empty; _guidedAutomaticBlipMethod = string.Empty; _guidedEvidenceNotes = string.Empty; + _guidedTelemetry = null; _guidedFullThrottleUpshift = "not-tested"; _guidedCoastDownshift = "not-tested"; _evidenceNotes.BorderThickness = new Thickness(1); @@ -441,7 +443,8 @@ private void SaveClicked(object sender, RoutedEventArgs eventArgs) WheelNotes = _wheelNotes.Text, EvidenceNotes = string.IsNullOrWhiteSpace(_evidenceNotes.Text) ? new string[0] - : new[] { _evidenceNotes.Text.Trim() } + : new[] { _evidenceNotes.Text.Trim() }, + Telemetry = _guidedTelemetry }; string path = _plugin.SaveVerificationDraft(draft, _observer.Text); _savedDraftPath = path; @@ -582,6 +585,7 @@ private void ApplyGuidedResults(GuidedDriveResults results) ApplyGuidedChoice(_automaticBlip, results.AutomaticBlip); _guidedFullThrottleUpshift = results.FullThrottleUpshift; _guidedCoastDownshift = results.CoastDownshift; + _guidedTelemetry = results.Telemetry; if (results.ForwardGears.HasValue) { _forwardGears.Text = results.ForwardGears.Value.ToString(CultureInfo.InvariantCulture); diff --git a/tests/test_import_observation.py b/tests/test_import_observation.py index 1f403c05..8ffb9fa4 100644 --- a/tests/test_import_observation.py +++ b/tests/test_import_observation.py @@ -99,6 +99,44 @@ def test_a_driven_implementation_survives_the_import(self) -> None: self.assertIn("no declared version", note) self.assertIn("not the real car", note) + def test_recorded_telemetry_is_evidence_and_changes_nothing_staged(self) -> None: + """A trace lets a later detector re-read a drive; it is not a result. + + The importer stages from the answers the contributor reviewed. If the + samples beside them could change what is staged, attaching evidence + would alter the claim it is evidence for. + """ + observation = _clean_observation() + expected = import_observation(observation, imported_at="2026-11-13") + observation["telemetry"] = { + "sample_rate_hz": 10.0, + "resting_throttle": 0.0, + "resting_clutch": None, + "traces": [ + { + "test": "coast_downshift", + "outcome": "accepted", + "sample_rate_hz": None, + "columns": [ + "t_ms", "gear", "clutch", "throttle", "brake", "rpm", + "speed_kmh", "engine_torque", "engine_started", + ], + "samples": [ + [0, 4, 0, 0, 0, 4500, 80, None, 1], + [100, 3, 0, 0, 0, 6000, 78, 100.5, 1], + ], + } + ], + } + schema = json.loads( + (ROOT / "schema" / "v1" / "verification-observation.schema.json") + .read_text(encoding="utf-8") + ) + self.assertEqual([], validate_instance(observation, schema, "observation")) + self.assertEqual( + expected, import_observation(observation, imported_at="2026-11-13") + ) + def test_gtr2_physics_fingerprint_scope_is_registered(self) -> None: observation = _clean_observation() observation["simulator"] = "gtr2" From b017ec4b822aa8c56758aacf3cfec1d95fc1831d Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Nov 2026 19:24:26 +0000 Subject: [PATCH 2/9] Read the shift tests out of free driving, and skip the scripted ones it settles Every flat upshift and every off-throttle downshift in ordinary driving is already one of the scripted tests, taken without being asked. The gear-count step now asks the driver to drive normally up to top gear and watches each shift on the way, using the scripted detector's own thresholds. Three or more like shifts stand in for the scripted test, so a car that takes flat upshifts and off-throttle downshifts finishes the drive after this step instead of four more, on several shifts' evidence rather than one. The overlay shows the counts as they register and the result line says which tests remain. Free driving can only see what the gearbox accepted; a refusal looks the same as a shift never tried. So it never records that a lift, a blip or the clutch is required, and the scripted two-stage test still runs for anything it did not settle. A shift with clutch input is set aside. A lifted upshift is counted but settles nothing. Flat upshifts without a torque collapse leave the cut unknown, never no. Downshifts that disagree about a throttle spike settle nothing, because a driver who blipped on some looks the same. A lower gear that never took drive is not counted. The slow-sample-rate rule applies here as it does to the scripted test. The counts are stored under telemetry.free_drive and the trace kept for a test settled this way is the most recent shift that counted. Also drops the sentence telling the driver a licensed SimHub is fast enough. It rested on an unlicensed update rate nobody here can observe. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- docs/verification-observations.md | 42 ++- .../v1/verification-observation.schema.json | 44 ++- simhub/AsDriven.Core.Tests/Program.cs | 218 +++++++++++- simhub/AsDriven.Core/AsDriven.Core.csproj | 1 + simhub/AsDriven.Core/FreeDriveObserver.cs | 320 ++++++++++++++++++ .../AsDriven.Core/GuidedVerificationDrive.cs | 206 ++++++++++- .../AsDriven.Core/VerificationObservation.cs | 27 +- tests/test_import_observation.py | 9 + 8 files changed, 839 insertions(+), 28 deletions(-) create mode 100644 simhub/AsDriven.Core/FreeDriveObserver.cs diff --git a/docs/verification-observations.md b/docs/verification-observations.md index df341c00..2bf8da0b 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -33,8 +33,10 @@ the research backlog, or a curation approval. and that creep reads as moving off without a clutch. Held against the brake the engine either stalls or it does not. Time spent stationary under throttle while braking is therefore never counted as a refusal to move. -4. Count accepted forward gears. -5. Test clutchless upshift and downshift acceptance. +4. Drive normally up to top gear. This counts the forward gears and watches + every shift on the way; see "Free driving" below. +5. Test clutchless upshift and downshift acceptance, for whatever free driving + did not already settle. 6. Capture or confirm automatic cut and blip, including the observation method. 7. Record visible cockpit actuators and select the primary mechanism from hardware plus driver animation or reliable documentation. Direct gear inputs @@ -89,7 +91,8 @@ name. ACC's executables expose no useful file version, so the draft records the exact Steam content build from `appmanifest_805550.acf`. A missing or inaccessible manifest remains `unknown` and must be resolved during review rather than guessed. -The overlay starts directly with the move-off maneuver. Each test uses the same +The overlay starts directly with the move-off maneuver, followed by free +driving. Each scripted test uses the same cycle: perform the maneuver, wait for a captured result, then accept it or choose Retry/Skip. The overlay uses a short summary and green capture mark so completion is immediately visible; the longer falsifiable description is retained for form review. @@ -113,6 +116,39 @@ is detected only from a shift-local engine-torque collapse while throttle demand stays high. Imports of older ACC drafts degrade any throttle-derived cut answer to `unknown` because ACC exposes no engine-torque channel through SimHub. +## Free driving + +The second step asks the driver to drive normally up to top gear, shifting up +at full throttle and down off the throttle, without the clutch where the car +allows it. Each of those shifts is one of the scripted tests taken without +being asked, so the step counts them: + +- a clutchless upshift with the throttle held at 70% or more is a full-throttle + upshift, and a torque collapse with the throttle still held is a cut; +- a clutchless downshift on a closed throttle that takes drive is a coast + downshift, and a throttle spike of 15% or more after it is a blip. + +The thresholds are the scripted detector's own. Three or more like shifts stand +in for the scripted test, which is then skipped: a car that takes flat upshifts +and off-throttle downshifts finishes the drive after this step. The overlay +shows the running counts, and the result line says which tests remain. + +Free driving can only see what the gearbox accepted. A refusal looks the same +as a shift the driver never tried, so it never records that a lift, a blip or +the clutch is required; that still takes the scripted two-stage test. For the +same reason: + +- a shift with clutch input is set aside and counted separately; +- a lifted upshift is counted but settles nothing, because lifting having + worked does not show it was needed; +- flat upshifts with no torque collapse leave the cut `unknown`, never `no`; +- downshifts that disagree about a throttle spike settle nothing, because a + driver who blipped on some of them looks the same, and the scripted test + with the pedal held closed is what separates the two. + +The counts are stored under `telemetry.free_drive`, and the trace kept for a +test settled this way is the most recent shift that counted. + ## Recorded telemetry and sample rate A draft keeps the samples each accepted guided result was decided from, under diff --git a/schema/v1/verification-observation.schema.json b/schema/v1/verification-observation.schema.json index 8836c77d..469fada1 100644 --- a/schema/v1/verification-observation.schema.json +++ b/schema/v1/verification-observation.schema.json @@ -303,6 +303,48 @@ "null" ] }, + "free_drive": { + "description": "What the free-driving step counted before the driver ended it. These are shifts the gearbox accepted in ordinary driving; a refusal cannot be seen there, so the counts never establish that a lift, a blip or the clutch is required. Three or more like shifts stand in for the scripted test, and the trace kept for that test is then the most recent of them.", + "type": "object", + "additionalProperties": false, + "properties": { + "full_throttle_upshifts": { + "description": "Clutchless upshifts accepted with the throttle held open.", + "type": "integer", + "minimum": 0 + }, + "cut_measured": { + "description": "Of those, how many published engine torque with the throttle still held through the change.", + "type": "integer", + "minimum": 0 + }, + "with_torque_cut": { + "description": "Of the measured ones, how many showed a torque collapse.", + "type": "integer", + "minimum": 0 + }, + "lifted_upshifts": { + "description": "Clutchless upshifts accepted on a lifted throttle.", + "type": "integer", + "minimum": 0 + }, + "coast_downshifts": { + "description": "Clutchless downshifts on a closed throttle that took drive.", + "type": "integer", + "minimum": 0 + }, + "with_throttle_spike": { + "description": "Of those, how many were followed by a throttle spike of 15% or more.", + "type": "integer", + "minimum": 0 + }, + "shifts_with_clutch": { + "description": "Shifts set aside because clutch input was present.", + "type": "integer", + "minimum": 0 + } + } + }, "traces": { "type": "array", "items": { @@ -329,7 +371,7 @@ }, "$defs": { "telemetryTrace": { - "description": "One guided test's samples, ending at the sample that settled the result. Only an attempt whose result was accepted into the draft is kept; a retried or skipped attempt is not.", + "description": "One guided test's samples. A scripted test's trace ends at the sample that settled the result; a test settled by free driving keeps the samples around the most recent shift that counted. Only a result accepted into the draft has one; a retried or skipped attempt does not.", "type": "object", "additionalProperties": false, "required": [ diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 12f22aad..70ebf829 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1682,6 +1682,149 @@ private static int Main(string[] args) } } + // Ordinary driving already contains the scripted tests: + // every flat upshift and every off-throttle downshift is + // one. A car that takes them needs no scripted test, and + // several shifts are better evidence than one. + { + int clock = 0; + GuidedVerificationDrive lap = FreeDrivingDrive(now); + for (int gear = 1; gear < 6; gear++) + { + FreeUpshift(lap, now, ref clock, 16, gear, 0, 95, true); + } + for (int gear = 6; gear > 3; gear--) + { + FreeDownshift(lap, now, ref clock, 16, gear, 0, false, true); + } + True(lap.GetSnapshot().LiveValues.Contains("Up 5") + && lap.GetSnapshot().LiveValues.Contains("Down 3"), + "shows the driver each free-driving shift as it registers"); + lap.Next(); + True(lap.GetSnapshot().Result.Contains("Highest gear reached: 6"), + "free driving still records the highest gear reached"); + True(lap.GetSnapshot().ResultSummary.Contains("Upshifts and downshifts covered"), + "tells the driver that no scripted shift test remains"); + lap.Next(); + True(lap.GetSnapshot().Completed, + "finishes the drive when free driving settled both shift tests"); + GuidedDriveResults lapResults = lap.GetResults(); + Equal(6, lapResults.ForwardGears.Value, "free driving prefills the gear count"); + Equal("yes", lapResults.FullThrottleUpshift, "free driving establishes full-throttle upshifts"); + Equal("yes", lapResults.ClutchlessUpshift, "free driving establishes clutchless upshifts"); + Equal("yes", lapResults.AutomaticCut, "a torque collapse on every flat upshift is an automatic cut"); + Equal("yes", lapResults.CoastDownshift, "free driving establishes off-throttle downshifts"); + Equal("yes", lapResults.ClutchlessDownshift, "free driving establishes clutchless downshifts"); + Equal("no", lapResults.AutomaticBlip, "no spike on any free-driving downshift is no automatic blip"); + True(lapResults.AutomaticCutMethod.Contains("5 full-throttle clutchless upshifts"), + "says how many upshifts the answer rests on"); + Equal(5, lapResults.Telemetry.FreeDrive.FullThrottleUpshifts, "counts the flat upshifts"); + Equal(5, lapResults.Telemetry.FreeDrive.WithTorqueCut, "counts the torque cuts"); + Equal(3, lapResults.Telemetry.FreeDrive.CoastDownshifts, "counts the off-throttle downshifts"); + Equal( + "full_throttle_upshift,coast_downshift", + string.Join(",", lapResults.Telemetry.Traces.Select(trace => trace.Test)), + "keeps an example of each kind of shift free driving settled"); + verificationDraft.Telemetry = lapResults.Telemetry; + Equal(3, (int)VerificationObservationWriter.CreatePayload(verificationDraft) + ["telemetry"]["free_drive"]["coast_downshifts"], + "stores the free-driving counts in the draft"); + verificationDraft.Telemetry = null; + + // A car that blips itself does so on every downshift. + clock = 0; + GuidedVerificationDrive blipping = FreeDrivingDrive(now); + for (int gear = 6; gear > 2; gear--) + { + FreeDownshift(blipping, now, ref clock, 16, gear, 0, true, true); + } + blipping.Next(); + blipping.Next(); + Equal("yes", blipping.GetResults().AutomaticBlip, + "a spike after every free-driving downshift is recorded as an automatic blip"); + Equal("Full-throttle upshift", blipping.GetSnapshot().Title, + "still asks for the upshift test free driving did not cover"); + blipping.AddSample(GuidedSample(now.AddMilliseconds(clock + 16), 3, 0, 90, 5000, 80, 220, true)); + blipping.AddSample(GuidedSample(now.AddMilliseconds(clock + 32), 4, 0, 90, 4300, 82, 30, true)); + blipping.Next(); + True(blipping.GetSnapshot().Completed, + "does not repeat a downshift test free driving already settled"); + + // Some with a spike and some without is a driver who + // blipped on some of them. Only the scripted test, + // where the pedal stays closed, tells that from the car. + clock = 0; + GuidedVerificationDrive mixedLap = FreeDrivingDrive(now); + for (int gear = 1; gear < 6; gear++) + { + FreeUpshift(mixedLap, now, ref clock, 16, gear, 0, 95, false); + } + for (int gear = 6; gear > 2; gear--) + { + FreeDownshift(mixedLap, now, ref clock, 16, gear, 0, gear % 2 == 0, true); + } + mixedLap.Next(); + mixedLap.Next(); + Equal("Downshift without pedal input", mixedLap.GetSnapshot().Title, + "asks for the scripted downshift when free-driving downshifts disagree"); + Equal("unknown", mixedLap.GetResults().AutomaticCut, + "flat upshifts with no torque collapse leave the cut unknown, never no"); + Equal("not-tested", mixedLap.GetResults().AutomaticBlip, + "disagreeing downshifts record nothing about the blip"); + + // A shift made with the clutch says nothing about + // shifting without it, and a lifted upshift alone does + // not show the lift was needed. + clock = 0; + GuidedVerificationDrive clutched = FreeDrivingDrive(now); + for (int gear = 1; gear < 6; gear++) + { + FreeUpshift(clutched, now, ref clock, 16, gear, gear < 4 ? 80 : 0, gear < 4 ? 95 : 20, false); + } + for (int gear = 6; gear > 2; gear--) + { + FreeDownshift(clutched, now, ref clock, 16, gear, 80, false, true); + } + clutched.Next(); + clutched.Next(); + Equal("Full-throttle upshift", clutched.GetSnapshot().Title, + "clutched and lifted shifts settle nothing, so the scripted tests follow"); + Equal("not-tested", clutched.GetResults().ClutchlessUpshift, + "records nothing about clutchless shifting from shifts made with the clutch"); + Equal(7, clutched.GetResults().Telemetry.FreeDrive.ShiftsWithClutch, + "counts the shifts it set aside"); + Equal(2, clutched.GetResults().Telemetry.FreeDrive.LiftedUpshifts, + "counts lifted upshifts without treating them as a required lift"); + + // The gear index reports what was selected. A lower + // gear that never took drive is not an accepted shift. + clock = 0; + GuidedVerificationDrive slipping = FreeDrivingDrive(now); + for (int gear = 6; gear > 2; gear--) + { + FreeDownshift(slipping, now, ref clock, 16, gear, 0, false, false); + } + slipping.Next(); + slipping.Next(); + Equal("not-tested", slipping.GetResults().ClutchlessDownshift, + "a selected gear that never took drive is not counted as a downshift"); + + // At ten samples a second a blip can pass unseen, so + // its absence is unknown here too. + clock = 0; + GuidedVerificationDrive slowLap = FreeDrivingDrive(now); + for (int gear = 6; gear > 2; gear--) + { + FreeDownshift(slowLap, now, ref clock, 100, gear, 0, false, true); + } + slowLap.Next(); + slowLap.Next(); + Equal("yes", slowLap.GetResults().ClutchlessDownshift, + "slow telemetry still establishes the free-driving downshifts"); + Equal("unknown", slowLap.GetResults().AutomaticBlip, + "an unseen blip in slow free-driving telemetry stays unknown"); + } + // A result-ready phase still receives live telemetry while // it waits for Next. Preserve that current gear when the // coast test opens, or a first-attempt downshift whose blip @@ -2084,12 +2227,12 @@ private static int Main(string[] args) // And the gear count concludes instead of hanging when // no gear was ever engaged. - Equal("Forward gears", waiting.GetSnapshot().Title, "reaches the gear count"); + Equal("Free driving", waiting.GetSnapshot().Title, "reaches the gear count"); waiting.Next(); True(waiting.GetSnapshot().Result.Contains("no gear was ever engaged"), "the gear count concludes rather than refusing to advance"); waiting.Next(); - True(waiting.GetSnapshot().Title != "Forward gears", "the gear count advances"); + True(waiting.GetSnapshot().Title != "Free driving", "the gear count advances"); } // A shift taken at full throttle is not a lifted-throttle @@ -3084,6 +3227,77 @@ private static GuidedVerificationDrive CoastDownshiftDrive( return drive; } + /// A fresh guided session advanced to the free-driving step. + private static GuidedVerificationDrive FreeDrivingDrive(DateTime now) + { + GuidedVerificationDrive drive = new GuidedVerificationDrive(); + drive.Start(6); + for (int guard = 0; guard < 40 + && drive.GetSnapshot().Title != "Free driving"; guard++) + { + drive.AddSample(GuidedSample(now, 1, 0, 0, 1200, 0, 40, true)); + drive.Next(); + } + Equal("Free driving", drive.GetSnapshot().Title, "reaches the free-driving step"); + return drive; + } + + private static void FreeSteady( + GuidedVerificationDrive drive, + DateTime now, + ref int clock, + int interval, + int duration, + int gear, + double clutch, + double throttle, + double rpm, + double speedKmh, + double torque) + { + for (int elapsed = 0; elapsed < duration; elapsed += interval) + { + clock += interval; + drive.AddSample(GuidedSample( + now.AddMilliseconds(clock), gear, clutch, throttle, rpm, speedKmh, torque, true)); + } + } + + /// A second in , then the next gear up. + private static void FreeUpshift( + GuidedVerificationDrive drive, + DateTime now, + ref int clock, + int interval, + int gear, + double clutch, + double throttle, + bool torqueCut) + { + double speed = 60 + gear * 20; + FreeSteady(drive, now, ref clock, interval, 1000, gear, 0, throttle, 7000, speed, 300); + FreeSteady(drive, now, ref clock, interval, 64, gear, clutch, throttle, 7000, speed, torqueCut ? 40 : 300); + FreeSteady(drive, now, ref clock, interval, 800, gear + 1, 0, throttle, 6000, speed, 300); + } + + /// A second coasting in , then the next gear down. + private static void FreeDownshift( + GuidedVerificationDrive drive, + DateTime now, + ref int clock, + int interval, + int gear, + double clutch, + bool blip, + bool takesDrive) + { + FreeSteady(drive, now, ref clock, interval, 1000, gear, 0, 0, 4500, 150, 0); + FreeSteady(drive, now, ref clock, interval, 100, gear - 1, clutch, blip ? 60 : 0, + takesDrive ? 6000 : 4400, 148, 0); + FreeSteady(drive, now, ref clock, interval, 700, gear - 1, 0, 0, + takesDrive ? 6000 : 4400, 148, 0); + } + private static GuidedTelemetrySample GuidedSample( DateTime timestamp, int gear, diff --git a/simhub/AsDriven.Core/AsDriven.Core.csproj b/simhub/AsDriven.Core/AsDriven.Core.csproj index daf6278c..23eda813 100644 --- a/simhub/AsDriven.Core/AsDriven.Core.csproj +++ b/simhub/AsDriven.Core/AsDriven.Core.csproj @@ -54,6 +54,7 @@ + diff --git a/simhub/AsDriven.Core/FreeDriveObserver.cs b/simhub/AsDriven.Core/FreeDriveObserver.cs new file mode 100644 index 00000000..554fad51 --- /dev/null +++ b/simhub/AsDriven.Core/FreeDriveObserver.cs @@ -0,0 +1,320 @@ +using System; +using System.Collections.Generic; + +namespace AsDriven.Core +{ + /// + /// What ordinary driving showed about the gearbox, as counts. + /// + public sealed class FreeDriveSummary + { + /// Clutchless upshifts taken with the throttle held open, and accepted. + public int FullThrottleUpshifts { get; set; } + /// Of those, how many published torque with the throttle still held through the change. + public int CutMeasured { get; set; } + /// Of the measured ones, how many showed the torque collapse of an automatic cut. + public int WithTorqueCut { get; set; } + /// Clutchless upshifts taken on a lifted throttle, and accepted. + public int LiftedUpshifts { get; set; } + /// Clutchless downshifts taken on a closed throttle that took drive. + public int CoastDownshifts { get; set; } + /// Of those, how many were followed by a throttle spike. + public int WithThrottleSpike { get; set; } + /// The largest such spike, as a percentage after the resting offset. + public double PeakThrottleSpike { get; set; } + /// Shifts set aside because clutch input was present. + public int ShiftsWithClutch { get; set; } + /// The samples around the most recent counted full-throttle upshift. + public GuidedTelemetrySample[] UpshiftExample { get; set; } + /// The samples around the most recent counted coast downshift. + public GuidedTelemetrySample[] DownshiftExample { get; set; } + + public FreeDriveSummary Clone() + { + return (FreeDriveSummary)MemberwiseClone(); + } + + public bool HasEvents + { + get + { + return FullThrottleUpshifts > 0 || LiftedUpshifts > 0 + || CoastDownshifts > 0 || ShiftsWithClutch > 0; + } + } + } + + /// + /// Reads the guided tests out of ordinary driving. + /// + /// Every upshift a driver takes flat and every downshift taken off the + /// throttle is already one of the scripted tests, performed without being + /// asked. Watching a lap collects several of each where the scripted test + /// collects one, and a car that takes them all needs no scripted test at + /// all. + /// + /// It can only see what the gearbox accepted. A refusal looks like a shift + /// the driver never tried, so nothing here ever concludes that a lift, a + /// blip or the clutch is required; that still takes the scripted test, + /// where the driver is known to have asked. A shift with clutch input is + /// set aside for the same reason: it says nothing about shifting without. + /// + public sealed class FreeDriveObserver + { + // The thresholds are the guided detector's own, so a shift counted + // here is one the scripted test would also have accepted. + private const double FullThrottle = 70.0; + private const double LiftedThrottle = 45.0; + private const double ClosedThrottle = 10.0; + private const double BlipThrottle = 15.0; + private const double ClutchPresent = 20.0; + private const double EngagementRatioMargin = 1.05; + private const double MinimumSpeedKmh = 20.0; + + // A shift passes through neutral for an instant. Longer than this and + // the gear that follows is a new selection, not the end of a shift. + private const double NeutralPassageMilliseconds = 600.0; + // How long the new gear has to hold before the shift is counted. + private const double ConfirmMilliseconds = 600.0; + private const double BufferMilliseconds = 2500.0; + private const int BufferMaximumSamples = 300; + + private sealed class PendingShift + { + public int From; + public int To; + public DateTime LeftAtUtc; + public DateTime ArrivedAtUtc; + public double BaselineRatio; + } + + private readonly Queue _buffer = new Queue(); + private FreeDriveSummary _summary = new FreeDriveSummary(); + private PendingShift _pending; + private int _engagedGear; + private DateTime _engagedSeenUtc; + private double _engagedRatio; + + /// + /// What the throttle channel reads with the pedal released, which some + /// telemetry readers never report as zero. + /// + public double ThrottleOffset { get; set; } + + public void Reset() + { + _buffer.Clear(); + _summary = new FreeDriveSummary(); + _pending = null; + _engagedGear = 0; + _engagedRatio = 0.0; + } + + public FreeDriveSummary GetSummary() + { + return _summary.Clone(); + } + + public void AddSample(GuidedTelemetrySample sample) + { + if (sample == null) + { + return; + } + _buffer.Enqueue(sample); + while (_buffer.Count > BufferMaximumSamples + || (sample.TimestampUtc - _buffer.Peek().TimestampUtc).TotalMilliseconds + > BufferMilliseconds) + { + _buffer.Dequeue(); + } + if (_pending != null) + { + if (sample.Gear != _pending.To) + { + // The gear did not hold, so whatever this was, it was not + // an accepted shift. + _pending = null; + } + else if ((sample.TimestampUtc - _pending.ArrivedAtUtc).TotalMilliseconds + >= ConfirmMilliseconds) + { + Classify(_pending, sample); + _pending = null; + } + } + if (sample.Gear <= 0) + { + return; + } + if (_pending == null + && _engagedGear > 0 + && Math.Abs(sample.Gear - _engagedGear) == 1 + && sample.SpeedKmh > MinimumSpeedKmh + && (sample.TimestampUtc - _engagedSeenUtc).TotalMilliseconds + <= NeutralPassageMilliseconds) + { + _pending = new PendingShift + { + From = _engagedGear, + To = sample.Gear, + LeftAtUtc = _engagedSeenUtc, + ArrivedAtUtc = sample.TimestampUtc, + BaselineRatio = _engagedRatio + }; + } + _engagedGear = sample.Gear; + _engagedSeenUtc = sample.TimestampUtc; + _engagedRatio = DriveRatio(sample); + } + + private void Classify(PendingShift shift, GuidedTelemetrySample latest) + { + GuidedTelemetrySample[] samples = _buffer.ToArray(); + DateTime left = shift.LeftAtUtc; + DateTime arrived = shift.ArrivedAtUtc; + + if (Maximum(samples, left.AddMilliseconds(-250.0), arrived.AddMilliseconds(300.0), + sample => sample.Clutch) > ClutchPresent) + { + _summary.ShiftsWithClutch++; + return; + } + + if (shift.To > shift.From) + { + // The sample the old gear was last seen in is always inside + // this window, so it is never empty however slow the telemetry. + double before = Minimum(samples, left.AddMilliseconds(-250.0), left, Throttle); + if (before >= FullThrottle) + { + _summary.FullThrottleUpshifts++; + _summary.UpshiftExample = samples; + DateTime from = left.AddMilliseconds(-100.0); + DateTime to = arrived.AddMilliseconds(300.0); + double highest = Maximum( + samples, left.AddMilliseconds(-600.0), to, Torque); + double lowest = Minimum(samples, from, to, Torque); + // A cut is a torque collapse the driver did not ask for. + // With the throttle eased during the change the collapse + // has an ordinary explanation, so that shift measures + // nothing about a cut either way. + if (highest > 0.0 && Minimum(samples, from, to, Throttle) >= FullThrottle) + { + _summary.CutMeasured++; + if (highest > 20.0 && lowest <= highest * 0.40) + { + _summary.WithTorqueCut++; + } + } + } + else if (Throttle(LastAtOrBefore(samples, left)) <= LiftedThrottle) + { + _summary.LiftedUpshifts++; + } + return; + } + + // A blip can lead the reported gear change slightly, so the + // throttle has to be closed before that and anything from there on + // counts as the spike. + DateTime spikeFrom = left.AddMilliseconds(-150.0); + double closedBefore = Maximum( + samples, left.AddMilliseconds(-500.0), spikeFrom, Throttle); + if (double.IsNaN(closedBefore) || closedBefore > ClosedThrottle) + { + return; + } + // The gear index only says what was selected. A lower gear that + // took drive holds the engine at a higher speed for the same road + // speed, which a box left in neutral cannot do. + if (DriveRatio(latest) < shift.BaselineRatio * EngagementRatioMargin) + { + return; + } + _summary.CoastDownshifts++; + _summary.DownshiftExample = samples; + double spike = Maximum( + samples, spikeFrom.AddMilliseconds(1.0), arrived.AddMilliseconds(500.0), Throttle); + if (!double.IsNaN(spike) && spike >= BlipThrottle) + { + _summary.WithThrottleSpike++; + _summary.PeakThrottleSpike = Math.Max(_summary.PeakThrottleSpike, spike); + } + } + + private double Throttle(GuidedTelemetrySample sample) + { + return sample == null ? double.NaN : Math.Max(0.0, sample.Throttle - ThrottleOffset); + } + + private static double Torque(GuidedTelemetrySample sample) + { + return double.IsInfinity(sample.EngineTorque) ? double.NaN : sample.EngineTorque; + } + + private static double DriveRatio(GuidedTelemetrySample sample) + { + return sample.Rpm / Math.Max(sample.SpeedKmh, 1.0); + } + + private static GuidedTelemetrySample LastAtOrBefore( + GuidedTelemetrySample[] samples, DateTime moment) + { + GuidedTelemetrySample found = null; + foreach (GuidedTelemetrySample sample in samples) + { + if (sample.TimestampUtc <= moment) + { + found = sample; + } + } + return found; + } + + /// NaN when no sample in the window carries a value. + private static double Maximum( + GuidedTelemetrySample[] samples, + DateTime from, + DateTime to, + Func channel) + { + double result = double.NaN; + foreach (GuidedTelemetrySample sample in samples) + { + if (sample.TimestampUtc < from || sample.TimestampUtc > to) + { + continue; + } + double value = channel(sample); + if (!double.IsNaN(value) && (double.IsNaN(result) || value > result)) + { + result = value; + } + } + return result; + } + + private static double Minimum( + GuidedTelemetrySample[] samples, + DateTime from, + DateTime to, + Func channel) + { + double result = double.NaN; + foreach (GuidedTelemetrySample sample in samples) + { + if (sample.TimestampUtc < from || sample.TimestampUtc > to) + { + continue; + } + double value = channel(sample); + if (!double.IsNaN(value) && (double.IsNaN(result) || value < result)) + { + result = value; + } + } + return result; + } + } +} diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index 7e48113a..311cee07 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -93,6 +93,8 @@ public sealed class GuidedDriveTelemetry public double? RestingThrottle { get; set; } /// Published clutch at the same moment. public double? RestingClutch { get; set; } + /// What free driving counted, or null when it saw no shifts. + public FreeDriveSummary FreeDrive { get; set; } public GuidedTestTrace[] Traces { get; set; } } @@ -156,6 +158,14 @@ public sealed class GuidedVerificationDrive /// public const double MinimumRateForNegativeHz = 20.0; + /// + /// How many like shifts free driving has to see before it stands in + /// for a scripted test. One could be a fluke of timing; the scripted + /// test itself accepts one, but there the driver was asked to do + /// exactly that and nothing else. + /// + public const int FreeDriveShiftsToSettle = 3; + private enum Phase { Idle, @@ -261,6 +271,10 @@ private enum Phase private readonly int[] _sessionIntervals = new int[1001]; private int _sessionIntervalCount; private DateTime? _previousSampleUtc; + private readonly FreeDriveObserver _freeDrive = new FreeDriveObserver(); + /// What free driving had counted when the driver accepted it. + private FreeDriveSummary _freeDriveAccepted; + private bool _downshiftSettledByFreeDrive; public void Start(int? suggestedForwardGears, string simulator = null) { @@ -281,6 +295,8 @@ public void Start(int? suggestedForwardGears, string simulator = null) _restingThrottle = 0.0; _restingThrottleSeen = false; _traces.Clear(); + _freeDriveAccepted = null; + _downshiftSettledByFreeDrive = false; Array.Clear(_sessionIntervals, 0, _sessionIntervals.Length); _sessionIntervalCount = 0; _previousSampleUtc = null; @@ -317,6 +333,11 @@ public void AddSample(GuidedTelemetrySample sample) return; } RecordAttemptSample(sample); + if (_phase == Phase.GearCount) + { + _freeDrive.ThrottleOffset = ThrottleOffset(); + _freeDrive.AddSample(sample); + } UpdateTrace(sample); DetectResult(sample); } @@ -352,7 +373,8 @@ public void FinishAttempt() { _attemptAccepted = true; _resultReady = true; - _result = "Highest gear reached: " + _maximumGear + "."; + _result = "Highest gear reached: " + _maximumGear + "." + + FreeDriveCountSummary(_freeDrive.GetSummary()); } break; case Phase.FullThrottleUpshift: @@ -911,6 +933,13 @@ private void AcceptResultAndAdvance() case Phase.GearCount: _results.ForwardGears = _maximumGear > 0 ? (int?)_maximumGear : null; _results.DirectGearSelection = "not-tested"; + // Whatever free driving already showed needs no scripted + // test. The rest is asked for one test at a time, as before. + if (ApplyFreeDrive(_freeDrive.GetSummary())) + { + MoveTo(AfterUpshiftTests()); + break; + } // GTR 2 exposes no usable throttle-input channel through // SimHub. Skip the comparison it cannot measure and retain // the lifted attempt only as clutchless-shift evidence; @@ -927,7 +956,7 @@ private void AcceptResultAndAdvance() _results.FullThrottleUpshift = "yes"; _results.AutomaticCut = _automaticActionObserved ? "yes" : "unknown"; _results.AutomaticCutMethod = _result; - MoveTo(Phase.CoastDownshift); + MoveTo(AfterUpshiftTests()); } else { @@ -940,7 +969,7 @@ private void AcceptResultAndAdvance() _results.ClutchlessUpshift = _attemptAccepted ? "yes" : "no"; _results.AutomaticCut = _fullThrottleTestFailed ? "no" : "not-tested"; _results.AutomaticCutMethod = _result; - MoveTo(Phase.CoastDownshift); + MoveTo(AfterUpshiftTests()); break; case Phase.CoastDownshift: if (_attemptAccepted) @@ -1221,7 +1250,7 @@ private void AdvanceAfterSkip() case Phase.MoveOff: MoveTo(Phase.GearCount); break; case Phase.GearCount: MoveTo(Phase.FullThrottleUpshift); break; case Phase.FullThrottleUpshift: MoveTo(Phase.LiftedUpshift); break; - case Phase.LiftedUpshift: MoveTo(Phase.CoastDownshift); break; + case Phase.LiftedUpshift: MoveTo(AfterUpshiftTests()); break; case Phase.CoastDownshift: MoveTo(Phase.ManualBlipDownshift); break; case Phase.ManualBlipDownshift: MoveTo(Phase.Complete); break; } @@ -1335,10 +1364,15 @@ private void RecordAttemptSample(GuidedTelemetrySample sample) /// samples arrived to measure one. /// private double? AttemptSampleRateHz() + { + return MedianSampleRateHz(_attemptSamples); + } + + private static double? MedianSampleRateHz(IEnumerable samples) { var intervals = new List(); DateTime? previous = null; - foreach (GuidedTelemetrySample sample in _attemptSamples) + foreach (GuidedTelemetrySample sample in samples) { if (previous.HasValue) { @@ -1389,7 +1423,11 @@ private bool TelemetryTooSlowForNegative() private string SlowTelemetrySummary(string what) { - double? rate = AttemptSampleRateHz(); + return SlowTelemetrySummary(what, AttemptSampleRateHz()); + } + + private static string SlowTelemetrySummary(string what, double? rate) + { if (!rate.HasValue || rate.Value >= MinimumRateForNegativeHz) { return string.Empty; @@ -1397,8 +1435,7 @@ private string SlowTelemetrySummary(string what) return " Telemetry arrived at about " + rate.Value.ToString("0", CultureInfo.InvariantCulture) + " samples a second, which is too slow to rule out " + what - + ": one can pass between two samples. This stays unknown rather than no." - + " A licensed SimHub updates fast enough to settle it."; + + ": one can pass between two samples. This stays unknown rather than no."; } private static string TraceName(Phase phase) @@ -1436,9 +1473,121 @@ private void KeepAttemptTrace() }); } + private Phase AfterUpshiftTests() + { + return _downshiftSettledByFreeDrive ? Phase.Complete : Phase.CoastDownshift; + } + + private bool FreeDriveSettlesUpshift(FreeDriveSummary summary) + { + return VerificationReviewRules.UpshiftThrottleIsMeasurable(_simulator) + && summary.FullThrottleUpshifts >= FreeDriveShiftsToSettle; + } + + /// + /// Free driving settles the downshift only when its downshifts agree. + /// Some with a throttle spike and some without is a driver who blipped + /// on some of them, and the scripted test - where the pedal is asked to + /// stay closed - is what tells that from the car. + /// + private bool FreeDriveSettlesDownshift(FreeDriveSummary summary) + { + return VerificationReviewRules.UpshiftThrottleIsMeasurable(_simulator) + && summary.CoastDownshifts >= FreeDriveShiftsToSettle + && (summary.WithThrottleSpike == 0 + || summary.WithThrottleSpike == summary.CoastDownshifts); + } + + private static string FreeDriveCountSummary(FreeDriveSummary summary) + { + if (!summary.HasEvents) + { + return string.Empty; + } + return " Free driving saw " + summary.FullThrottleUpshifts + + " full-throttle and " + summary.LiftedUpshifts + + " lifted clutchless upshifts, " + summary.CoastDownshifts + + " off-throttle clutchless downshifts, and " + summary.ShiftsWithClutch + + " shifts with clutch input, which were set aside."; + } + + /// + /// Writes what free driving established into the results. Returns + /// whether the upshift tests are settled; the downshift's answer is + /// kept in because it is + /// consulted later, after any scripted upshift test. + /// + private bool ApplyFreeDrive(FreeDriveSummary summary) + { + _freeDriveAccepted = summary.HasEvents ? summary : null; + double? rate = SessionSampleRateHz(); + bool tooSlow = rate.HasValue && rate.Value < MinimumRateForNegativeHz; + bool upshiftSettled = FreeDriveSettlesUpshift(summary); + if (upshiftSettled) + { + bool cut = summary.CutMeasured >= FreeDriveShiftsToSettle + && summary.WithTorqueCut == summary.CutMeasured; + _results.ClutchlessUpshift = "yes"; + _results.FullThrottleUpshift = "yes"; + _results.AutomaticCut = cut ? "yes" : "unknown"; + _results.AutomaticCutMethod = "Free driving: " + summary.FullThrottleUpshifts + + " full-throttle clutchless upshifts were accepted. " + + (cut + ? "Engine torque collapsed on all " + summary.CutMeasured + + " measured while throttle demand stayed high." + : summary.CutMeasured == 0 + ? "None published engine torque with the throttle held through the" + + " change, so nothing was measured about a cut." + : "Engine torque collapsed on " + summary.WithTorqueCut + " of " + + summary.CutMeasured + " measured, which does not establish a cut." + + SlowTelemetrySummary("an automatic cut", rate)); + KeepFreeDriveTrace("full_throttle_upshift", summary.UpshiftExample); + } + _downshiftSettledByFreeDrive = FreeDriveSettlesDownshift(summary); + if (_downshiftSettledByFreeDrive) + { + bool spike = summary.WithThrottleSpike > 0; + _results.ClutchlessDownshift = "yes"; + _results.CoastDownshift = "yes"; + _results.AutomaticBlip = !AutomaticBlipIsMeasurable() || !ThrottleChannelFollowsPedal() + ? "unknown" + : spike + ? "yes" + : (tooSlow ? "unknown" : "no"); + _results.AutomaticBlipMethod = "Free driving: " + summary.CoastDownshifts + + " off-throttle clutchless downshifts were accepted. " + + (!AutomaticBlipIsMeasurable() + ? UnmeasurableAutomaticBlipSummary(false) + : spike + ? "A throttle spike followed every one, peaking at " + + summary.PeakThrottleSpike.ToString("0", CultureInfo.InvariantCulture) + + "% (the threshold is 15%)." + RestingThrottleSummary() + : "No throttle spike followed any of them." + + SlowTelemetrySummary("an automatic blip", rate)); + KeepFreeDriveTrace("coast_downshift", summary.DownshiftExample); + } + return upshiftSettled; + } + + private void KeepFreeDriveTrace(string name, GuidedTelemetrySample[] samples) + { + if (samples == null || samples.Length == 0) + { + return; + } + _traces.RemoveAll(trace => string.Equals(trace.Test, name, StringComparison.Ordinal)); + _traces.Add(new GuidedTestTrace + { + Test = name, + Outcome = "accepted", + SampleRateHz = MedianSampleRateHz(samples), + Samples = samples + }); + } + private GuidedDriveTelemetry BuildTelemetry() { - if (_traces.Count == 0) + if (_traces.Count == 0 && _freeDriveAccepted == null) { return null; } @@ -1447,6 +1596,7 @@ private GuidedDriveTelemetry BuildTelemetry() SampleRateHz = SessionSampleRateHz(), RestingThrottle = _restingThrottleSeen ? (double?)_restingThrottle : null, RestingClutch = _restingClutchSeen ? (double?)_restingClutch : null, + FreeDrive = _freeDriveAccepted, Traces = _traces.ToArray() }; } @@ -1454,6 +1604,7 @@ private GuidedDriveTelemetry BuildTelemetry() private void ResetTrace() { _attemptSamples.Clear(); + _freeDrive.Reset(); _baselineGear = 0; _downshiftCandidateGear = 0; _downshiftCandidateAtUtc = DateTime.MinValue; @@ -1553,7 +1704,7 @@ private static string Title(Phase phase) { case Phase.Intro: return "Before you begin"; case Phase.MoveOff: return "Move-off clutch test"; - case Phase.GearCount: return "Forward gears"; + case Phase.GearCount: return "Free driving"; case Phase.FullThrottleUpshift: return "Full-throttle upshift"; case Phase.LiftedUpshift: return "Lifted-throttle upshift"; case Phase.CoastDownshift: return "Downshift without pedal input"; @@ -1569,7 +1720,7 @@ private static string Headline(Phase phase) { case Phase.Intro: return "Prepare to drive"; case Phase.MoveOff: return "Release the clutch"; - case Phase.GearCount: return "Find the top gear"; + case Phase.GearCount: return "Drive normally up to top gear"; case Phase.FullThrottleUpshift: return "Upshift without clutch input"; case Phase.LiftedUpshift: return "Upshift without clutch input"; case Phase.CoastDownshift: return "Downshift without clutch input"; @@ -1604,7 +1755,12 @@ private static string PromptLine1(Phase phase) // off without one. Held against the brake there is nowhere for // that creep to go, so the engine either stalls or it does not. case Phase.MoveOff: return "Stopped in 1st, brake on. Release the clutch fully."; - case Phase.GearCount: return "Shift up until the gearbox will not go higher."; + // Every flat upshift and every off-throttle downshift here is + // one of the tests that follow, taken without being asked, so + // a car that accepts them is finished after this step. The + // clutch is not forbidden: a shift made with it is set aside + // and the scripted test asks again. + case Phase.GearCount: return "Shift up at full throttle and down off-throttle. Skip the clutch if the car allows."; case Phase.FullThrottleUpshift: return "Keep throttle above 70% while moving, then shift up."; case Phase.LiftedUpshift: return "While moving, lift off the throttle, then shift up."; case Phase.CoastDownshift: return "At safe RPM, lift and downshift. Stay off the throttle."; @@ -1620,7 +1776,7 @@ private static string PromptLine2(Phase phase) { case Phase.Intro: return "Next accepts; use Retry or Skip when needed."; case Phase.MoveOff: return "If it does not stall, release the brake and add light throttle."; - case Phase.GearCount: return "Then press NEXT. Only you can see the top gear."; + case Phase.GearCount: return "Reach top gear, then press NEXT. A lap of normal driving is plenty."; case Phase.FullThrottleUpshift: return "Keep the clutch fully released throughout the test."; case Phase.LiftedUpshift: return "Keep the clutch fully released throughout the test."; case Phase.CoastDownshift: return "Stay off the throttle and keep the clutch fully released until CAPTURED."; @@ -1643,9 +1799,18 @@ private string ResultSummary(Phase phase) ? "Moved off without the clutch" : "No movement without the clutch; this car needs it"; case Phase.GearCount: - return _maximumGear > 0 - ? "Forward gears recorded: " + _maximumGear - : "No forward gears recorded"; + if (_maximumGear <= 0) + { + return "No forward gears recorded"; + } + FreeDriveSummary seen = _freeDrive.GetSummary(); + bool up = FreeDriveSettlesUpshift(seen); + bool down = FreeDriveSettlesDownshift(seen); + return _maximumGear + " gears. " + + (up && down ? "Upshifts and downshifts covered" + : up ? "Upshifts covered; downshift test next" + : down ? "Downshifts covered; upshift test next" + : "Upshift and downshift tests next"); case Phase.FullThrottleUpshift: return _attemptAccepted ? "Full-throttle upshift detected" @@ -1682,6 +1847,15 @@ private string LiveValues(GuidedTelemetrySample sample) { return "Waiting for live telemetry"; } + if (_phase == Phase.GearCount) + { + // What this step is collecting, so the driver can see a shift + // register and knows when there are enough. + FreeDriveSummary seen = _freeDrive.GetSummary(); + return "Top gear " + (_maximumGear > 0 ? _maximumGear.ToString() : "-") + + " | Up " + seen.FullThrottleUpshifts + + " | Down " + seen.CoastDownshifts; + } return "G " + (sample.Gear == 0 ? "N" : sample.Gear.ToString()) + " | C " + Math.Round(sample.Clutch) + "%" + " | T " + Math.Round(EffectiveThrottle(sample.Throttle)) + "%" diff --git a/simhub/AsDriven.Core/VerificationObservation.cs b/simhub/AsDriven.Core/VerificationObservation.cs index a50217d3..ee883c50 100644 --- a/simhub/AsDriven.Core/VerificationObservation.cs +++ b/simhub/AsDriven.Core/VerificationObservation.cs @@ -361,12 +361,12 @@ public static JObject CreatePayload(VerificationObservationDraft draft) /// private static JObject TelemetryPayload(GuidedDriveTelemetry telemetry) { - if (telemetry == null || telemetry.Traces == null) + if (telemetry == null) { return null; } var traces = new JArray(); - foreach (GuidedTestTrace trace in telemetry.Traces) + foreach (GuidedTestTrace trace in telemetry.Traces ?? new GuidedTestTrace[0]) { if (trace == null || trace.Samples == null || trace.Samples.Length == 0) { @@ -406,17 +406,32 @@ private static JObject TelemetryPayload(GuidedDriveTelemetry telemetry) { "samples", rows } }); } - if (traces.Count == 0) + FreeDriveSummary freeDrive = telemetry.FreeDrive; + if (traces.Count == 0 && freeDrive == null) { return null; } - return new JObject + var payload = new JObject { { "sample_rate_hz", NullableNumber(telemetry.SampleRateHz) }, { "resting_throttle", NullableNumber(telemetry.RestingThrottle) }, - { "resting_clutch", NullableNumber(telemetry.RestingClutch) }, - { "traces", traces } + { "resting_clutch", NullableNumber(telemetry.RestingClutch) } }; + if (freeDrive != null) + { + payload.Add("free_drive", new JObject + { + { "full_throttle_upshifts", freeDrive.FullThrottleUpshifts }, + { "cut_measured", freeDrive.CutMeasured }, + { "with_torque_cut", freeDrive.WithTorqueCut }, + { "lifted_upshifts", freeDrive.LiftedUpshifts }, + { "coast_downshifts", freeDrive.CoastDownshifts }, + { "with_throttle_spike", freeDrive.WithThrottleSpike }, + { "shifts_with_clutch", freeDrive.ShiftsWithClutch } + }); + } + payload.Add("traces", traces); + return payload; } /// diff --git a/tests/test_import_observation.py b/tests/test_import_observation.py index 8ffb9fa4..b5fbac4d 100644 --- a/tests/test_import_observation.py +++ b/tests/test_import_observation.py @@ -112,6 +112,15 @@ def test_recorded_telemetry_is_evidence_and_changes_nothing_staged(self) -> None "sample_rate_hz": 10.0, "resting_throttle": 0.0, "resting_clutch": None, + "free_drive": { + "full_throttle_upshifts": 5, + "cut_measured": 5, + "with_torque_cut": 5, + "lifted_upshifts": 0, + "coast_downshifts": 3, + "with_throttle_spike": 0, + "shifts_with_clutch": 1, + }, "traces": [ { "test": "coast_downshift", From 8f895a611985cdb46092d8b5e19817a1bb9be67c Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 13 Nov 2026 19:24:38 +0000 Subject: [PATCH 3/9] Drop the unverified unlicensed-rate remark from the sample-rate comment Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- simhub/AsDriven.Core/GuidedVerificationDrive.cs | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index 311cee07..ec780a17 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -151,9 +151,8 @@ public sealed class GuidedVerificationDrive /// /// The slowest telemetry at which "nothing was seen" still means /// nothing happened. An automatic blip or an ignition cut lasts on the - /// order of a tenth of a second, so at ten samples a second - which is - /// what an unlicensed SimHub delivers - one can pass entirely between - /// two samples. A spike that was seen is still a spike at any rate; + /// order of a tenth of a second, so at ten samples a second one can + /// pass entirely between two samples. A spike that was seen is still a spike at any rate; /// only its absence needs the rate to vouch for it. /// public const double MinimumRateForNegativeHz = 20.0; From c3cf63c7b0e300a048028a1cc7e5a67cc8dc3c3a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 00:01:02 +0000 Subject: [PATCH 4/9] Count free-driving shifts on a channel that reports the car's own clutch The first live drive of free driving, a TS050 in Project Motor Racing 2.3.0.3, settled nothing and fell through to the scripted tests. Its draft showed why: six shifts set aside as clutched and none counted. PMR's clutch channel reads 100% with the car stopped and 100% again through every downshift, as the car works its own clutch, so every downshift was filed as the driver's. The scripted test accepts such a shift with a caution; free driving now does the same when the channel already read above 20% at rest, stores how many counted shifts moved the channel, and attaches that caution to the result. The upshifts matched nothing at all. PMR's published throttle drops to 80% for the last few samples before the gear index changes, which after the simulator's resting offset fell under the 70% the detector required right up to the change. The driver's throttle is now read from before the last 150 ms, and those last moments only have to not be a lift. The downshift's engagement baseline is taken from the same earlier point, before a blip raises it. Shifts that are none of the tests are now counted as other_shifts, so a draft says when shifts were seen and why they settled nothing. A scripted trace ended on the sample that settled it, which for an upshift is the gear change itself. It now runs up to half a second past it. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- docs/verification-observations.md | 16 +++- .../v1/verification-observation.schema.json | 12 ++- simhub/AsDriven.Core.Tests/Program.cs | 75 +++++++++++++++++ simhub/AsDriven.Core/FreeDriveObserver.cs | 84 ++++++++++++++----- .../AsDriven.Core/GuidedVerificationDrive.cs | 44 +++++++++- .../AsDriven.Core/VerificationObservation.cs | 4 +- tests/test_import_observation.py | 2 + 7 files changed, 208 insertions(+), 29 deletions(-) diff --git a/docs/verification-observations.md b/docs/verification-observations.md index 2bf8da0b..73b4182b 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -128,7 +128,10 @@ being asked, so the step counts them: - a clutchless downshift on a closed throttle that takes drive is a coast downshift, and a throttle spike of 15% or more after it is a blip. -The thresholds are the scripted detector's own. Three or more like shifts stand +The driver's throttle is read from before the last 150 ms, because the car's +own cut or blip can show on the published throttle just ahead of the gear +change; those last moments only have to not be a lift. The thresholds are the +scripted detector's own. Three or more like shifts stand in for the scripted test, which is then skipped: a car that takes flat upshifts and off-throttle downshifts finishes the drive after this step. The overlay shows the running counts, and the result line says which tests remain. @@ -138,7 +141,12 @@ as a shift the driver never tried, so it never records that a lift, a blip or the clutch is required; that still takes the scripted two-stage test. For the same reason: -- a shift with clutch input is set aside and counted separately; +- a shift with clutch input is set aside and counted separately, unless the + clutch channel already read above 20% with the car stopped. Such a channel + reports the car's own clutch (Project Motor Racing reads 100% at rest and + through every downshift), so the shift is counted, the count of shifts where + the channel moved is stored, and the result carries the scripted test's + caution to confirm the pedal was untouched; - a lifted upshift is counted but settles nothing, because lifting having worked does not show it was needed; - flat upshifts with no torque collapse leave the cut `unknown`, never `no`; @@ -153,8 +161,8 @@ test settled this way is the most recent shift that counted. A draft keeps the samples each accepted guided result was decided from, under the optional `telemetry` block. Each trace holds at most the last six seconds -or 300 samples of one test, ending at the sample that settled it, with the -channels exactly as SimHub published them. The gear count has no trace because +or 300 samples of one test up to the sample that settled it, then up to half a +second more, with the channels exactly as SimHub published them. The gear count has no trace because the driver ends it. A retried or skipped attempt is not kept, and a draft answered by hand has no `telemetry` block at all. diff --git a/schema/v1/verification-observation.schema.json b/schema/v1/verification-observation.schema.json index 469fada1..a9b71aff 100644 --- a/schema/v1/verification-observation.schema.json +++ b/schema/v1/verification-observation.schema.json @@ -339,7 +339,17 @@ "minimum": 0 }, "shifts_with_clutch": { - "description": "Shifts set aside because clutch input was present.", + "description": "Shifts set aside because clutch input was present on a channel that follows the pedal.", + "type": "integer", + "minimum": 0 + }, + "counted_with_vehicle_clutch": { + "description": "Counted shifts during which the clutch channel moved, where resting_clutch shows the channel reports the car's own clutch rather than the pedal.", + "type": "integer", + "minimum": 0 + }, + "other_shifts": { + "description": "Shifts that were none of the tests: taken on part throttle, or a lower gear that never took drive.", "type": "integer", "minimum": 0 } diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 70ebf829..b35d1fe6 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1809,6 +1809,81 @@ private static int Main(string[] args) Equal("not-tested", slipping.GetResults().ClutchlessDownshift, "a selected gear that never took drive is not counted as a downshift"); + // Project Motor Racing reports the car's own clutch on + // the clutch channel: 100% with the car stopped, and + // 100% again through every downshift, together with + // the car's blip. Its published throttle also dips to + // 80% just before an upshift. The first live drive of + // free driving, in a TS050, set every downshift aside + // as the driver's clutch and matched no upshift at all, + // so nothing was settled. These are that drive's shapes. + clock = 0; + GuidedVerificationDrive pmrLap = new GuidedVerificationDrive(); + pmrLap.Start(7, "pmr"); + pmrLap.AddSample(GuidedSample(now, 0, 100, 0, 1500, 0, -0.4, true, 50)); + for (int guard = 0; guard < 40 + && pmrLap.GetSnapshot().Title != "Free driving"; guard++) + { + pmrLap.Next(); + } + for (int gear = 1; gear < 7; gear++) + { + double speed = 100 + gear * 20; + FreeSteady(pmrLap, now, ref clock, 16, 1000, gear, 0, 100, 8200, speed, 410); + FreeSteady(pmrLap, now, ref clock, 16, 100, gear, 0, 100, 8200, speed, 300); + FreeSteady(pmrLap, now, ref clock, 16, 100, gear, 0, 100, 8200, speed, 190); + FreeSteady(pmrLap, now, ref clock, 16, 64, gear, 0, 80, 8200, speed, 140); + FreeSteady(pmrLap, now, ref clock, 16, 800, gear + 1, 0, 100, 8000, speed, 300); + } + for (int gear = 7; gear > 3; gear--) + { + FreeSteady(pmrLap, now, ref clock, 16, 1000, gear, 0, 0, 5700, 155, -95); + FreeSteady(pmrLap, now, ref clock, 16, 100, gear, 100, 100, 5800, 153, -80); + FreeSteady(pmrLap, now, ref clock, 16, 32, gear - 1, 80, 0, 6100, 152, -57); + FreeSteady(pmrLap, now, ref clock, 16, 700, gear - 1, 0, 0, 6750, 149, -90); + } + pmrLap.Next(); + pmrLap.Next(); + GuidedDriveResults pmrResults = pmrLap.GetResults(); + Equal(6, pmrResults.Telemetry.FreeDrive.FullThrottleUpshifts, + "counts a flat upshift whose published throttle dips as the car cuts"); + Equal(4, pmrResults.Telemetry.FreeDrive.CoastDownshifts, + "counts a downshift where the channel shows the car's own clutch"); + Equal(0, pmrResults.Telemetry.FreeDrive.ShiftsWithClutch, + "sets no shift aside as the driver's clutch on a channel that reads 100% at rest"); + Equal(4, pmrResults.Telemetry.FreeDrive.CountedWithVehicleClutch, + "records how many counted shifts moved the clutch channel"); + True(pmrLap.GetSnapshot().Completed, + "free driving settles both tests on a car that works its own clutch"); + Equal("yes", pmrResults.FullThrottleUpshift, "the live drive's upshifts are settled"); + Equal("yes", pmrResults.AutomaticCut, "the live drive's torque collapse is read as a cut"); + Equal("yes", pmrResults.CoastDownshift, "the live drive's downshifts are settled"); + True(pmrResults.AutomaticBlipMethod.Contains("car's own clutch"), + "attaches the scripted test's clutch caution to the free-driving result"); + + // The scripted trace used to end on the sample that + // settled it, which for an upshift is the gear change + // itself, leaving nothing after the shift to look at. + GuidedVerificationDrive tail = new GuidedVerificationDrive(); + tail.Start(6); + for (int guard = 0; guard < 40 + && tail.GetSnapshot().Title != "Full-throttle upshift"; guard++) + { + tail.AddSample(GuidedSample(now, 3, 0, 0, 4000, 80, 100, true)); + tail.Next(); + } + for (int tick = 1; tick <= 120; tick++) + { + tail.AddSample(GuidedSample( + now.AddMilliseconds(tick * 16), tick < 30 ? 3 : 4, 0, 95, 6000, 120, 300, true)); + } + tail.Next(); + GuidedTelemetrySample[] tailSamples = tail.GetResults().Telemetry.Traces + .Single(trace => trace.Test == "full_throttle_upshift").Samples; + int afterShift = tailSamples.Count(sample => sample.Gear == 4); + True(afterShift > 20 && afterShift < 45, + "keeps about half a second of samples after the result, and no more"); + // At ten samples a second a blip can pass unseen, so // its absence is unknown here too. clock = 0; diff --git a/simhub/AsDriven.Core/FreeDriveObserver.cs b/simhub/AsDriven.Core/FreeDriveObserver.cs index 554fad51..184e2923 100644 --- a/simhub/AsDriven.Core/FreeDriveObserver.cs +++ b/simhub/AsDriven.Core/FreeDriveObserver.cs @@ -24,6 +24,16 @@ public sealed class FreeDriveSummary public double PeakThrottleSpike { get; set; } /// Shifts set aside because clutch input was present. public int ShiftsWithClutch { get; set; } + /// + /// Counted shifts during which the clutch channel moved, on a channel + /// already known to report the car's own clutch rather than the pedal. + /// + public int CountedWithVehicleClutch { get; set; } + /// + /// Shifts that were none of the tests: taken on part throttle, or a + /// lower gear that never took drive. + /// + public int OtherShifts { get; set; } /// The samples around the most recent counted full-throttle upshift. public GuidedTelemetrySample[] UpshiftExample { get; set; } /// The samples around the most recent counted coast downshift. @@ -39,7 +49,7 @@ public bool HasEvents get { return FullThrottleUpshifts > 0 || LiftedUpshifts > 0 - || CoastDownshifts > 0 || ShiftsWithClutch > 0; + || CoastDownshifts > 0 || ShiftsWithClutch > 0 || OtherShifts > 0; } } } @@ -101,6 +111,17 @@ private sealed class PendingShift /// public double ThrottleOffset { get; set; } + /// + /// True when the clutch channel is known not to follow the pedal: it + /// already read as depressed with the car stopped and the driver doing + /// nothing. Project Motor Racing reads 100% at rest and again through + /// every downshift, as the car works its own clutch. Setting those + /// shifts aside as the driver's left nothing to count at all, so on + /// such a channel a shift is counted and the clutch movement recorded + /// beside it, which is how the scripted test already treats it. + /// + public bool ClutchChannelReportsVehicle { get; set; } + public void Reset() { _buffer.Clear(); @@ -174,32 +195,49 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) DateTime left = shift.LeftAtUtc; DateTime arrived = shift.ArrivedAtUtc; - if (Maximum(samples, left.AddMilliseconds(-250.0), arrived.AddMilliseconds(300.0), - sample => sample.Clutch) > ClutchPresent) + bool clutchMoved = Maximum( + samples, left.AddMilliseconds(-250.0), arrived.AddMilliseconds(300.0), + sample => sample.Clutch) > ClutchPresent; + if (clutchMoved && !ClutchChannelReportsVehicle) { _summary.ShiftsWithClutch++; return; } + // The car's own shift can show on the pedal channels just before + // the gear index changes: a cut pulls the published throttle down + // and a blip pushes it up. What the driver was doing is therefore + // read from before that, and the last moments only have to not + // contradict it. + DateTime settled = left.AddMilliseconds(-150.0); + GuidedTelemetrySample beforeShift = LastAtOrBefore(samples, settled) + ?? LastAtOrBefore(samples, left); + if (shift.To > shift.From) { - // The sample the old gear was last seen in is always inside - // this window, so it is never empty however slow the telemetry. - double before = Minimum(samples, left.AddMilliseconds(-250.0), left, Throttle); - if (before >= FullThrottle) + double held = Minimum(samples, left.AddMilliseconds(-600.0), settled, Throttle); + if (double.IsNaN(held)) + { + held = Throttle(beforeShift); + } + double atChange = Minimum(samples, settled, left, Throttle); + if (held >= FullThrottle && atChange > LiftedThrottle) { _summary.FullThrottleUpshifts++; _summary.UpshiftExample = samples; - DateTime from = left.AddMilliseconds(-100.0); + if (clutchMoved) + { + _summary.CountedWithVehicleClutch++; + } DateTime to = arrived.AddMilliseconds(300.0); double highest = Maximum( samples, left.AddMilliseconds(-600.0), to, Torque); - double lowest = Minimum(samples, from, to, Torque); + double lowest = Minimum(samples, settled, to, Torque); // A cut is a torque collapse the driver did not ask for. - // With the throttle eased during the change the collapse + // With the throttle lifted during the change the collapse // has an ordinary explanation, so that shift measures // nothing about a cut either way. - if (highest > 0.0 && Minimum(samples, from, to, Throttle) >= FullThrottle) + if (highest > 0.0 && Minimum(samples, settled, to, Throttle) > LiftedThrottle) { _summary.CutMeasured++; if (highest > 20.0 && lowest <= highest * 0.40) @@ -208,31 +246,39 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) } } } - else if (Throttle(LastAtOrBefore(samples, left)) <= LiftedThrottle) + else if (atChange <= LiftedThrottle) { _summary.LiftedUpshifts++; } + else + { + _summary.OtherShifts++; + } return; } // A blip can lead the reported gear change slightly, so the // throttle has to be closed before that and anything from there on // counts as the spike. - DateTime spikeFrom = left.AddMilliseconds(-150.0); + DateTime spikeFrom = settled; double closedBefore = Maximum( samples, left.AddMilliseconds(-500.0), spikeFrom, Throttle); - if (double.IsNaN(closedBefore) || closedBefore > ClosedThrottle) - { - return; - } // The gear index only says what was selected. A lower gear that // took drive holds the engine at a higher speed for the same road - // speed, which a box left in neutral cannot do. - if (DriveRatio(latest) < shift.BaselineRatio * EngagementRatioMargin) + // speed, which a box left in neutral cannot do. The comparison is + // with the engine before any blip began raising it. + double baseline = beforeShift == null ? shift.BaselineRatio : DriveRatio(beforeShift); + if (double.IsNaN(closedBefore) || closedBefore > ClosedThrottle + || DriveRatio(latest) < baseline * EngagementRatioMargin) { + _summary.OtherShifts++; return; } _summary.CoastDownshifts++; + if (clutchMoved) + { + _summary.CountedWithVehicleClutch++; + } _summary.DownshiftExample = samples; double spike = Maximum( samples, spikeFrom.AddMilliseconds(1.0), arrived.AddMilliseconds(500.0), Throttle); diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index ec780a17..4c6f5705 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -143,6 +143,8 @@ public sealed class GuidedVerificationDrive // intake limit of 256 KB. private static readonly TimeSpan TraceWindow = TimeSpan.FromSeconds(6.0); private const int TraceMaximumSamples = 300; + private static readonly TimeSpan TraceAfterResult = TimeSpan.FromMilliseconds(500.0); + private const int TraceAfterResultMaximumSamples = 40; // A rate is only reported from enough intervals to take a median of. // Fewer than this is a handful of samples, not a measurement. @@ -270,6 +272,7 @@ private enum Phase private readonly int[] _sessionIntervals = new int[1001]; private int _sessionIntervalCount; private DateTime? _previousSampleUtc; + private DateTime? _resultReadyAtUtc; private readonly FreeDriveObserver _freeDrive = new FreeDriveObserver(); /// What free driving had counted when the driver accepted it. private FreeDriveSummary _freeDriveAccepted; @@ -326,6 +329,15 @@ public void AddSample(GuidedTelemetrySample sample) { _lastSample = sample; RecordInterval(sample); + if (IsTestPhase(_phase) && _resultReady && _phase != Phase.GearCount + && _resultReadyAtUtc.HasValue + && sample.TimestampUtc - _resultReadyAtUtc.Value <= TraceAfterResult + && _attemptSamples.Count < TraceMaximumSamples + TraceAfterResultMaximumSamples) + { + // The result is settled at the gear change, which is the + // one moment a later reader most wants to see past. + _attemptSamples.Enqueue(sample); + } if (!IsTestPhase(_phase) || (_resultReady && _phase != Phase.GearCount)) { @@ -335,6 +347,8 @@ public void AddSample(GuidedTelemetrySample sample) if (_phase == Phase.GearCount) { _freeDrive.ThrottleOffset = ThrottleOffset(); + _freeDrive.ClutchChannelReportsVehicle = + _restingClutchSeen && _restingClutch > 20.0; _freeDrive.AddSample(sample); } UpdateTrace(sample); @@ -909,6 +923,7 @@ private void SetResult(bool accepted, bool automaticAction, string message) _attemptAccepted = accepted; _automaticActionObserved = automaticAction; _resultReady = true; + _resultReadyAtUtc = _lastSample == null ? (DateTime?)null : _lastSample.TimestampUtc; _result = message; } @@ -1506,8 +1521,9 @@ private static string FreeDriveCountSummary(FreeDriveSummary summary) return " Free driving saw " + summary.FullThrottleUpshifts + " full-throttle and " + summary.LiftedUpshifts + " lifted clutchless upshifts, " + summary.CoastDownshifts - + " off-throttle clutchless downshifts, and " + summary.ShiftsWithClutch - + " shifts with clutch input, which were set aside."; + + " off-throttle clutchless downshifts, " + summary.ShiftsWithClutch + + " shifts with clutch input, which were set aside, and " + summary.OtherShifts + + " that were none of these."; } /// @@ -1539,7 +1555,8 @@ private bool ApplyFreeDrive(FreeDriveSummary summary) + " change, so nothing was measured about a cut." : "Engine torque collapsed on " + summary.WithTorqueCut + " of " + summary.CutMeasured + " measured, which does not establish a cut." - + SlowTelemetrySummary("an automatic cut", rate)); + + SlowTelemetrySummary("an automatic cut", rate)) + + FreeDriveClutchSummary(summary); KeepFreeDriveTrace("full_throttle_upshift", summary.UpshiftExample); } _downshiftSettledByFreeDrive = FreeDriveSettlesDownshift(summary); @@ -1562,12 +1579,30 @@ private bool ApplyFreeDrive(FreeDriveSummary summary) + summary.PeakThrottleSpike.ToString("0", CultureInfo.InvariantCulture) + "% (the threshold is 15%)." + RestingThrottleSummary() : "No throttle spike followed any of them." - + SlowTelemetrySummary("an automatic blip", rate)); + + SlowTelemetrySummary("an automatic blip", rate)) + + FreeDriveClutchSummary(summary); KeepFreeDriveTrace("coast_downshift", summary.DownshiftExample); } return upshiftSettled; } + /// + /// The same caution the scripted tests attach, for the same reason: + /// on a channel that reports the car's clutch, a driver's foot on the + /// pedal cannot be told from the car working it. + /// + private string FreeDriveClutchSummary(FreeDriveSummary summary) + { + if (summary.CountedWithVehicleClutch == 0) + { + return string.Empty; + } + return " The clutch channel moved during " + summary.CountedWithVehicleClutch + + " of the counted shifts. It already read " + Math.Round(_restingClutch) + + "% with the car stopped before any test began, so this simulator reports the" + + " car's own clutch here; confirm the pedal was untouched."; + } + private void KeepFreeDriveTrace(string name, GuidedTelemetrySample[] samples) { if (samples == null || samples.Length == 0) @@ -1603,6 +1638,7 @@ private GuidedDriveTelemetry BuildTelemetry() private void ResetTrace() { _attemptSamples.Clear(); + _resultReadyAtUtc = null; _freeDrive.Reset(); _baselineGear = 0; _downshiftCandidateGear = 0; diff --git a/simhub/AsDriven.Core/VerificationObservation.cs b/simhub/AsDriven.Core/VerificationObservation.cs index ee883c50..dee3bce1 100644 --- a/simhub/AsDriven.Core/VerificationObservation.cs +++ b/simhub/AsDriven.Core/VerificationObservation.cs @@ -427,7 +427,9 @@ private static JObject TelemetryPayload(GuidedDriveTelemetry telemetry) { "lifted_upshifts", freeDrive.LiftedUpshifts }, { "coast_downshifts", freeDrive.CoastDownshifts }, { "with_throttle_spike", freeDrive.WithThrottleSpike }, - { "shifts_with_clutch", freeDrive.ShiftsWithClutch } + { "shifts_with_clutch", freeDrive.ShiftsWithClutch }, + { "counted_with_vehicle_clutch", freeDrive.CountedWithVehicleClutch }, + { "other_shifts", freeDrive.OtherShifts } }); } payload.Add("traces", traces); diff --git a/tests/test_import_observation.py b/tests/test_import_observation.py index b5fbac4d..a26b3e1e 100644 --- a/tests/test_import_observation.py +++ b/tests/test_import_observation.py @@ -120,6 +120,8 @@ def test_recorded_telemetry_is_evidence_and_changes_nothing_staged(self) -> None "coast_downshifts": 3, "with_throttle_spike": 0, "shifts_with_clutch": 1, + "counted_with_vehicle_clutch": 0, + "other_shifts": 2, }, "traces": [ { From b7eda0bc75afea78365a17994cefeeeb21e5c51b Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 00:11:18 +0000 Subject: [PATCH 5/9] Let a measured resting throttle override PMR's fallback offset PMR's 15% idle offset was applied whenever the resting reading was not above 5%, which included a reading of zero. A TS050 in PMR 2.3.0.3 read 0% at rest in four drafts, so full throttle read 85% and the channel was treated as not following the pedal. Free driving then saw a throttle spike on 13 of 13 downshifts and recorded the blip as unknown. A reading taken with the car stopped now always decides the offset. PMR's 15% remains the fallback only for a drive that never saw the car at rest, which the existing rolling-start test still covers. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- docs/verification-observations.md | 7 +++++++ simhub/AsDriven.Core.Tests/Program.cs | 8 ++++++++ simhub/AsDriven.Core/GuidedVerificationDrive.cs | 15 ++++++++++----- 3 files changed, 25 insertions(+), 5 deletions(-) diff --git a/docs/verification-observations.md b/docs/verification-observations.md index 73b4182b..73952050 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -174,6 +174,13 @@ detector can instead be run over the stored samples. `resting_throttle` and `resting_clutch` record what those channels read with the car stopped, which is what decides whether a later reading can be attributed to the pedal. +A throttle reading taken with the car stopped is the offset the drive uses: a +channel that rests above 5% has that much subtracted and cannot establish an +automatic blip, and one that rests at or near zero follows the pedal. Project +Motor Racing rested at 15% on the drives that first measured it and at 0% in a +TS050 on 2.3.0.3, so its 15% figure is applied only when a drive never saw the +car at rest. + The draft also records the median sample rate, for the drive and for each attempt. An automatic blip or an ignition cut lasts about a tenth of a second, so at ten samples a second one can pass between two samples unseen. Below 20 diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index b35d1fe6..4e20ee09 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1858,6 +1858,14 @@ private static int Main(string[] args) Equal("yes", pmrResults.FullThrottleUpshift, "the live drive's upshifts are settled"); Equal("yes", pmrResults.AutomaticCut, "the live drive's torque collapse is read as a cut"); Equal("yes", pmrResults.CoastDownshift, "the live drive's downshifts are settled"); + // That drive read 0% throttle with the car stopped, so + // the channel follows the pedal and PMR's older 15% + // idle figure does not apply to it. A spike after every + // downshift is then a blip, not an unknown. + Equal("yes", pmrResults.AutomaticBlip, + "a measured zero resting throttle overrides PMR's fallback offset"); + True(pmrResults.AutomaticBlipMethod.Contains("peaking at 100%"), + "reports the spike without subtracting an offset the drive did not measure"); True(pmrResults.AutomaticBlipMethod.Contains("car's own clutch"), "attaches the scripted test's clutch caution to the free-driving result"); diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index 4c6f5705..d93bffe5 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -1092,9 +1092,14 @@ private bool ThrottleChannelFollowsPedal() /// /// The guided tests are about the driver's pedal. Some readers retain /// a small idle offset even with the pedal fully released. PMR has - /// consistently reported 15%, including when a drive begins after the - /// stationary move-off sample was missed. Preserve a captured offset - /// when available, otherwise apply PMR's established value. A + /// reported 15% on the drives that established this, including when a + /// drive begins after the stationary move-off sample was missed. A + /// reading taken with the car stopped always wins over that figure: + /// PMR 2.3.0.3 read 0% at rest in a TS050, and subtracting 15% anyway + /// made full throttle read 85% and marked a channel that plainly + /// followed the pedal as one that did not, so a blip seen on every + /// downshift was recorded as unknown. PMR's established value is only + /// the fallback for a drive that never saw the car at rest. A /// non-pedal channel remains ineligible to establish an automatic blip /// through . /// @@ -1105,9 +1110,9 @@ private double EffectiveThrottle(double publishedThrottle) private double ThrottleOffset() { - if (_restingThrottleSeen && _restingThrottle > 5.0) + if (_restingThrottleSeen) { - return _restingThrottle; + return _restingThrottle > 5.0 ? _restingThrottle : 0.0; } return string.Equals(_simulator, "pmr", StringComparison.Ordinal) ? 15.0 From 2ee6b77c28b835f351181bead25c09e272cce6af Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 00:25:50 +0000 Subject: [PATCH 6/9] Count quick consecutive shifts in free driving A gear had to hold 600 ms before its shift counted, and a shift that was followed sooner was dropped outright, so a quick run down the gears counted only its last step. A gear held 300 ms before the next shift began is now judged on what it showed in that time, with the clutch, torque and blip windows ending there so the next shift's blip is not attributed to it. In a run of coast downshifts the throttle only has to have closed again after the previous blip. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- docs/verification-observations.md | 6 ++- simhub/AsDriven.Core.Tests/Program.cs | 62 +++++++++++++++++++++++ simhub/AsDriven.Core/FreeDriveObserver.cs | 60 +++++++++++++++++++--- 3 files changed, 119 insertions(+), 9 deletions(-) diff --git a/docs/verification-observations.md b/docs/verification-observations.md index 73952050..4017b1d5 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -131,7 +131,11 @@ being asked, so the step counts them: The driver's throttle is read from before the last 150 ms, because the car's own cut or blip can show on the published throttle just ahead of the gear change; those last moments only have to not be a lift. The thresholds are the -scripted detector's own. Three or more like shifts stand +scripted detector's own. A shift counts once its gear has held for 600 ms, or +for 300 ms when the next shift follows straight on, as it does going down the +gears for a corner; only what happened before that next shift is read. A gear +held for less is not counted. In a run of downshifts the throttle only has to +have closed again after the previous blip. Three or more like shifts stand in for the scripted test, which is then skipped: a car that takes flat upshifts and off-throttle downshifts finishes the drive after this step. The overlay shows the running counts, and the result line says which tests remain. diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 4e20ee09..50160f2c 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1892,6 +1892,53 @@ private static int Main(string[] args) True(afterShift > 20 && afterShift < 45, "keeps about half a second of samples after the result, and no more"); + // Going down the gears for a corner, a driver takes + // the next gear well inside the time one is normally + // confirmed over. A live drive lost most of its quick + // downshifts that way. Each gear here is held 350 ms, + // with the car's blip on every one. + clock = 0; + GuidedVerificationDrive quick = FreeDrivingDrive(now); + FreeSteady(quick, now, ref clock, 16, 1000, 6, 0, 0, 4500, 220, 0); + for (int gear = 5; gear >= 2; gear--) + { + double quickSpeed = 100 + gear * 20; + FreeSteady(quick, now, ref clock, 16, 100, gear, 0, 60, 6000, quickSpeed, 0); + FreeSteady(quick, now, ref clock, 16, 250, gear, 0, 0, 6500, quickSpeed, 0); + } + FreeSteady(quick, now, ref clock, 16, 700, 2, 0, 0, 6500, 140, 0); + Equal(4, QuickCount(quick, false), + "counts each of a run of quick downshifts"); + Equal(4, QuickCount(quick, true), + "attributes one blip to each quick downshift, not its neighbour's"); + + // Short gears on the way up are taken just as fast. + clock = 0; + GuidedVerificationDrive quickUp = FreeDrivingDrive(now); + FreeSteady(quickUp, now, ref clock, 16, 1000, 1, 0, 100, 7000, 60, 300); + for (int gear = 2; gear <= 5; gear++) + { + FreeSteady(quickUp, now, ref clock, 16, 400, gear, 0, 100, 7000, 40 + gear * 25, 300); + } + FreeSteady(quickUp, now, ref clock, 16, 700, 6, 0, 100, 7000, 200, 300); + quickUp.Next(); + quickUp.Next(); + Equal(5, quickUp.GetResults().Telemetry.FreeDrive.FullThrottleUpshifts, + "counts each of a run of quick full-throttle upshifts"); + + // A gear touched for an instant on the way past is + // still not an accepted shift. + clock = 0; + GuidedVerificationDrive passing = FreeDrivingDrive(now); + FreeSteady(passing, now, ref clock, 16, 1000, 6, 0, 0, 4500, 220, 0); + for (int gear = 5; gear >= 2; gear--) + { + FreeSteady(passing, now, ref clock, 16, 120, gear, 0, 0, 6500, 100 + gear * 20, 0); + } + FreeSteady(passing, now, ref clock, 16, 700, 2, 0, 0, 6500, 140, 0); + Equal(1, QuickCount(passing, false), + "does not count gears passed through in an instant, only the one that held"); + // At ten samples a second a blip can pass unseen, so // its absence is unknown here too. clock = 0; @@ -3310,6 +3357,21 @@ private static GuidedVerificationDrive CoastDownshiftDrive( return drive; } + /// + /// Ends the free-driving step and returns how many coast downshifts it + /// counted, or how many of them carried a throttle spike. + /// + private static int QuickCount(GuidedVerificationDrive drive, bool withSpike) + { + if (!drive.GetSnapshot().ResultReady) + { + drive.Next(); + drive.Next(); + } + FreeDriveSummary counted = drive.GetResults().Telemetry.FreeDrive; + return withSpike ? counted.WithThrottleSpike : counted.CoastDownshifts; + } + /// A fresh guided session advanced to the free-driving step. private static GuidedVerificationDrive FreeDrivingDrive(DateTime now) { diff --git a/simhub/AsDriven.Core/FreeDriveObserver.cs b/simhub/AsDriven.Core/FreeDriveObserver.cs index 184e2923..57ccb06a 100644 --- a/simhub/AsDriven.Core/FreeDriveObserver.cs +++ b/simhub/AsDriven.Core/FreeDriveObserver.cs @@ -86,6 +86,11 @@ public sealed class FreeDriveObserver private const double NeutralPassageMilliseconds = 600.0; // How long the new gear has to hold before the shift is counted. private const double ConfirmMilliseconds = 600.0; + // A driver going down through the gears for a corner does not wait + // that long between shifts. A gear held at least this long before the + // next shift began is judged on what it showed in that time, rather + // than being dropped because it was not held for longer. + private const double QuickShiftMilliseconds = 300.0; private const double BufferMilliseconds = 2500.0; private const int BufferMaximumSamples = 300; @@ -96,11 +101,14 @@ private sealed class PendingShift public DateTime LeftAtUtc; public DateTime ArrivedAtUtc; public double BaselineRatio; + /// The most recent sample still in the new gear. + public GuidedTelemetrySample Latest; } private readonly Queue _buffer = new Queue(); private FreeDriveSummary _summary = new FreeDriveSummary(); private PendingShift _pending; + private DateTime _coastDownshiftArrivedUtc = DateTime.MinValue; private int _engagedGear; private DateTime _engagedSeenUtc; private double _engagedRatio; @@ -127,6 +135,7 @@ public void Reset() _buffer.Clear(); _summary = new FreeDriveSummary(); _pending = null; + _coastDownshiftArrivedUtc = DateTime.MinValue; _engagedGear = 0; _engagedRatio = 0.0; } @@ -153,8 +162,15 @@ public void AddSample(GuidedTelemetrySample sample) { if (sample.Gear != _pending.To) { - // The gear did not hold, so whatever this was, it was not - // an accepted shift. + // Left again before the full confirmation. Held long + // enough, it was a shift followed quickly by another; + // otherwise the gear did not hold and it was not an + // accepted shift at all. + if ((_pending.Latest.TimestampUtc - _pending.ArrivedAtUtc).TotalMilliseconds + >= QuickShiftMilliseconds) + { + Classify(_pending, _pending.Latest); + } _pending = null; } else if ((sample.TimestampUtc - _pending.ArrivedAtUtc).TotalMilliseconds @@ -163,6 +179,10 @@ public void AddSample(GuidedTelemetrySample sample) Classify(_pending, sample); _pending = null; } + else + { + _pending.Latest = sample; + } } if (sample.Gear <= 0) { @@ -181,7 +201,8 @@ public void AddSample(GuidedTelemetrySample sample) To = sample.Gear, LeftAtUtc = _engagedSeenUtc, ArrivedAtUtc = sample.TimestampUtc, - BaselineRatio = _engagedRatio + BaselineRatio = _engagedRatio, + Latest = sample }; } _engagedGear = sample.Gear; @@ -194,9 +215,13 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) GuidedTelemetrySample[] samples = _buffer.ToArray(); DateTime left = shift.LeftAtUtc; DateTime arrived = shift.ArrivedAtUtc; + // Nothing after the last sample in this gear belongs to this + // shift. When another followed quickly, what comes next is that + // shift's cut or blip, not this one's. + DateTime until = latest.TimestampUtc; bool clutchMoved = Maximum( - samples, left.AddMilliseconds(-250.0), arrived.AddMilliseconds(300.0), + samples, left.AddMilliseconds(-250.0), Earlier(arrived.AddMilliseconds(300.0), until), sample => sample.Clutch) > ClutchPresent; if (clutchMoved && !ClutchChannelReportsVehicle) { @@ -229,7 +254,7 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) { _summary.CountedWithVehicleClutch++; } - DateTime to = arrived.AddMilliseconds(300.0); + DateTime to = Earlier(arrived.AddMilliseconds(300.0), until); double highest = Maximum( samples, left.AddMilliseconds(-600.0), to, Torque); double lowest = Minimum(samples, settled, to, Torque); @@ -261,8 +286,20 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) // throttle has to be closed before that and anything from there on // counts as the spike. DateTime spikeFrom = settled; - double closedBefore = Maximum( - samples, left.AddMilliseconds(-500.0), spikeFrom, Throttle); + double closedBefore; + if ((left - _coastDownshiftArrivedUtc).TotalMilliseconds < ConfirmMilliseconds) + { + // This follows straight on from a downshift already counted + // as taken off the throttle, and that one's blip is still + // inside the usual window. A driver on the throttle would + // still be on it now, so it only has to have closed again. + closedBefore = Throttle(beforeShift); + } + else + { + closedBefore = Maximum( + samples, left.AddMilliseconds(-500.0), spikeFrom, Throttle); + } // The gear index only says what was selected. A lower gear that // took drive holds the engine at a higher speed for the same road // speed, which a box left in neutral cannot do. The comparison is @@ -275,13 +312,15 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) return; } _summary.CoastDownshifts++; + _coastDownshiftArrivedUtc = arrived; if (clutchMoved) { _summary.CountedWithVehicleClutch++; } _summary.DownshiftExample = samples; double spike = Maximum( - samples, spikeFrom.AddMilliseconds(1.0), arrived.AddMilliseconds(500.0), Throttle); + samples, spikeFrom.AddMilliseconds(1.0), + Earlier(arrived.AddMilliseconds(500.0), until), Throttle); if (!double.IsNaN(spike) && spike >= BlipThrottle) { _summary.WithThrottleSpike++; @@ -299,6 +338,11 @@ private static double Torque(GuidedTelemetrySample sample) return double.IsInfinity(sample.EngineTorque) ? double.NaN : sample.EngineTorque; } + private static DateTime Earlier(DateTime first, DateTime second) + { + return first <= second ? first : second; + } + private static double DriveRatio(GuidedTelemetrySample sample) { return sample.Rpm / Math.Max(sample.SpeedKmh, 1.0); From a4c3a5e629611f774a2f0a8fdda3bf793e4c3e2a Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 00:44:34 +0000 Subject: [PATCH 7/9] Do not read an unloaded engine's falling torque as a cut A flat clutchless upshift on an H-pattern box leaves the engine with no gear driving it. It runs up to the limiter and its torque falls away, then returns once the next gear takes drive. Free driving counted that as an automatic cut on every upshift of a live March GTP drive. A collapse during which engine speed climbed is now recorded as engine_ran_free and measures nothing about a cut, so the cut stays unknown. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- docs/verification-observations.md | 5 +- .../v1/verification-observation.schema.json | 5 ++ simhub/AsDriven.Core.Tests/Program.cs | 27 ++++++++++ simhub/AsDriven.Core/FreeDriveObserver.cs | 50 +++++++++++++++++-- .../AsDriven.Core/GuidedVerificationDrive.cs | 5 ++ .../AsDriven.Core/VerificationObservation.cs | 1 + tests/test_import_observation.py | 1 + 7 files changed, 90 insertions(+), 4 deletions(-) diff --git a/docs/verification-observations.md b/docs/verification-observations.md index 4017b1d5..29da0c9d 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -131,7 +131,10 @@ being asked, so the step counts them: The driver's throttle is read from before the last 150 ms, because the car's own cut or blip can show on the published throttle just ahead of the gear change; those last moments only have to not be a lift. The thresholds are the -scripted detector's own. A shift counts once its gear has held for 600 ms, or +scripted detector's own. A torque collapse on a flat upshift is not counted as +a cut when engine speed climbed while it fell: that is an engine running free +between the gears of an H-pattern box, and it measures nothing about a cut +(`engine_ran_free`). A shift counts once its gear has held for 600 ms, or for 300 ms when the next shift follows straight on, as it does going down the gears for a corner; only what happened before that next shift is read. A gear held for less is not counted. In a run of downshifts the throttle only has to diff --git a/schema/v1/verification-observation.schema.json b/schema/v1/verification-observation.schema.json index a9b71aff..6cc049c1 100644 --- a/schema/v1/verification-observation.schema.json +++ b/schema/v1/verification-observation.schema.json @@ -338,6 +338,11 @@ "type": "integer", "minimum": 0 }, + "engine_ran_free": { + "description": "Full-throttle upshifts whose torque fell while engine speed climbed, as an unloaded engine does between the gears of an H-pattern box. They measure nothing about a cut and are not in cut_measured.", + "type": "integer", + "minimum": 0 + }, "shifts_with_clutch": { "description": "Shifts set aside because clutch input was present on a channel that follows the pedal.", "type": "integer", diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 50160f2c..6ec35314 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1926,6 +1926,33 @@ private static int Main(string[] args) Equal(5, quickUp.GetResults().Telemetry.FreeDrive.FullThrottleUpshifts, "counts each of a run of quick full-throttle upshifts"); + // An H-pattern box taken flat without the clutch + // leaves the engine unloaded between gears: it runs + // up to the limiter and its torque falls away, then + // returns once the next gear takes drive. A live + // drive of such a car recorded a cut on every one. + clock = 0; + GuidedVerificationDrive hPattern = FreeDrivingDrive(now); + FreeSteady(hPattern, now, ref clock, 16, 1000, 2, 0, 100, 8600, 150, 600); + for (int gear = 3; gear <= 6; gear++) + { + double hSpeed = 60 + gear * 30; + FreeSteady(hPattern, now, ref clock, 16, 64, 0, 0, 100, 8800, hSpeed, 520); + FreeSteady(hPattern, now, ref clock, 16, 150, gear, 0, 100, 9450, hSpeed, 180); + FreeSteady(hPattern, now, ref clock, 16, 800, gear, 0, 100, 8600, hSpeed + 5, 600); + } + hPattern.Next(); + hPattern.Next(); + GuidedDriveResults hResults = hPattern.GetResults(); + Equal(4, hResults.Telemetry.FreeDrive.FullThrottleUpshifts, + "counts flat upshifts through neutral on an H-pattern box"); + Equal(4, hResults.Telemetry.FreeDrive.EngineRanFree, + "sees the engine running free between gears"); + Equal(0, hResults.Telemetry.FreeDrive.WithTorqueCut, + "does not read an unloaded engine's falling torque as a cut"); + Equal("unknown", hResults.AutomaticCut, + "leaves the cut unknown when the engine only ran free"); + // A gear touched for an instant on the way past is // still not an accepted shift. clock = 0; diff --git a/simhub/AsDriven.Core/FreeDriveObserver.cs b/simhub/AsDriven.Core/FreeDriveObserver.cs index 57ccb06a..1378ff07 100644 --- a/simhub/AsDriven.Core/FreeDriveObserver.cs +++ b/simhub/AsDriven.Core/FreeDriveObserver.cs @@ -14,6 +14,11 @@ public sealed class FreeDriveSummary public int CutMeasured { get; set; } /// Of the measured ones, how many showed the torque collapse of an automatic cut. public int WithTorqueCut { get; set; } + /// + /// Full-throttle upshifts whose torque fell because the engine ran + /// free between gears, which measures nothing about a cut. + /// + public int EngineRanFree { get; set; } /// Clutchless upshifts taken on a lifted throttle, and accepted. public int LiftedUpshifts { get; set; } /// Clutchless downshifts taken on a closed throttle that took drive. @@ -91,6 +96,9 @@ public sealed class FreeDriveObserver // next shift began is judged on what it showed in that time, rather // than being dropped because it was not held for longer. private const double QuickShiftMilliseconds = 300.0; + // Engine speed at the torque minimum, relative to when the gear was + // left, above which the engine is taken to have run free. + private const double FreeRevvingMargin = 1.03; private const double BufferMilliseconds = 2500.0; private const int BufferMaximumSamples = 300; @@ -264,10 +272,27 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) // nothing about a cut either way. if (highest > 0.0 && Minimum(samples, settled, to, Throttle) > LiftedThrottle) { - _summary.CutMeasured++; - if (highest > 20.0 && lowest <= highest * 0.40) + bool collapsed = highest > 20.0 && lowest <= highest * 0.40; + // An engine held flat with no gear driving it runs + // away, and its torque falls as it climbs to the + // limiter. That is what an H-pattern box does between + // gears, and it is not a cut: a cut takes torque away + // while the engine holds its speed or slows. A live + // drive of such a car read as a cut on every upshift. + GuidedTelemetrySample leaving = LastAtOrBefore(samples, left); + GuidedTelemetrySample weakest = LowestTorque(samples, settled, to); + if (collapsed && leaving != null && weakest != null + && weakest.Rpm > leaving.Rpm * FreeRevvingMargin) { - _summary.WithTorqueCut++; + _summary.EngineRanFree++; + } + else + { + _summary.CutMeasured++; + if (collapsed) + { + _summary.WithTorqueCut++; + } } } } @@ -385,6 +410,25 @@ private static double Maximum( return result; } + private static GuidedTelemetrySample LowestTorque( + GuidedTelemetrySample[] samples, DateTime from, DateTime to) + { + GuidedTelemetrySample result = null; + foreach (GuidedTelemetrySample sample in samples) + { + if (sample.TimestampUtc < from || sample.TimestampUtc > to) + { + continue; + } + double value = Torque(sample); + if (!double.IsNaN(value) && (result == null || value < Torque(result))) + { + result = sample; + } + } + return result; + } + private static double Minimum( GuidedTelemetrySample[] samples, DateTime from, diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index d93bffe5..c754d887 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -1555,6 +1555,11 @@ private bool ApplyFreeDrive(FreeDriveSummary summary) + (cut ? "Engine torque collapsed on all " + summary.CutMeasured + " measured while throttle demand stayed high." + : summary.CutMeasured == 0 && summary.EngineRanFree > 0 + ? "Engine torque fell on " + summary.EngineRanFree + + " while engine speed climbed with no gear driving it, which is" + + " an unloaded engine and not a cut, so nothing was measured" + + " about a cut." : summary.CutMeasured == 0 ? "None published engine torque with the throttle held through the" + " change, so nothing was measured about a cut." diff --git a/simhub/AsDriven.Core/VerificationObservation.cs b/simhub/AsDriven.Core/VerificationObservation.cs index dee3bce1..3729ab9c 100644 --- a/simhub/AsDriven.Core/VerificationObservation.cs +++ b/simhub/AsDriven.Core/VerificationObservation.cs @@ -424,6 +424,7 @@ private static JObject TelemetryPayload(GuidedDriveTelemetry telemetry) { "full_throttle_upshifts", freeDrive.FullThrottleUpshifts }, { "cut_measured", freeDrive.CutMeasured }, { "with_torque_cut", freeDrive.WithTorqueCut }, + { "engine_ran_free", freeDrive.EngineRanFree }, { "lifted_upshifts", freeDrive.LiftedUpshifts }, { "coast_downshifts", freeDrive.CoastDownshifts }, { "with_throttle_spike", freeDrive.WithThrottleSpike }, diff --git a/tests/test_import_observation.py b/tests/test_import_observation.py index a26b3e1e..1fb2bc93 100644 --- a/tests/test_import_observation.py +++ b/tests/test_import_observation.py @@ -122,6 +122,7 @@ def test_recorded_telemetry_is_evidence_and_changes_nothing_staged(self) -> None "shifts_with_clutch": 1, "counted_with_vehicle_clutch": 0, "other_shifts": 2, + "engine_ran_free": 1, }, "traces": [ { From 1bf7be934a2269b9ce7788193d6c0313a5584830 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 00:57:44 +0000 Subject: [PATCH 8/9] Judge a downshift against the settled ratio of the gear left The coast and manual-blip tests confirm a downshift when the lower gear holds the engine at a higher speed than the gear before it. When the driver shifted up to set the test up, that baseline was taken from the first sample in the higher gear, with the engine still falling from the gear below, so it read as high as the gear the driver then went back down to. A live drive had a downshift that plainly took drive go unconfirmed and recorded as refused. The baseline is now the lowest ratio seen in the gear, which is the one it actually holds. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- simhub/AsDriven.Core.Tests/Program.cs | 24 +++++++++++++++++++ .../AsDriven.Core/GuidedVerificationDrive.cs | 11 +++++++++ 2 files changed, 35 insertions(+) diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 6ec35314..6d2bd61a 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1926,6 +1926,30 @@ private static int Main(string[] args) Equal(5, quickUp.GetResults().Telemetry.FreeDrive.FullThrottleUpshifts, "counts each of a run of quick full-throttle upshifts"); + // The driver arrives at the coast test off the + // throttle, shifts up to set it up, and the engine is + // still coming down when the higher gear first shows. + // That first sample must not become the ratio the + // downshift is judged against. + GuidedVerificationDrive setUp = new GuidedVerificationDrive(); + setUp.Start(6); + for (int guard = 0; guard < 40 + && setUp.GetSnapshot().Title != "Downshift without pedal input"; guard++) + { + setUp.AddSample(GuidedSample(now, 4, 0, 0, 4000, 80, 100, true)); + setUp.Next(); + } + clock = 0; + FreeSteady(setUp, now, ref clock, 16, 100, 5, 0, 0, 4000, 80, 0); + FreeSteady(setUp, now, ref clock, 16, 1000, 5, 0, 80, 3200, 80, 100); + FreeSteady(setUp, now, ref clock, 16, 500, 5, 0, 0, 3200, 80, 0); + FreeSteady(setUp, now, ref clock, 16, 1000, 4, 0, 0, 4000, 80, 0); + True(setUp.GetSnapshot().ResultReady, + "confirms a downshift after an upshift made to set it up"); + setUp.Next(); + Equal("yes", setUp.GetResults().CoastDownshift, + "judges the downshift against the settled ratio of the gear left"); + // An H-pattern box taken flat without the clutch // leaves the engine unloaded between gears: it runs // up to the limiter and its torque falls away, then diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index c754d887..6bf4ff11 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -779,6 +779,17 @@ private void DetectDownshift(GuidedTelemetrySample sample, bool manualBlip) _baselineDriveRatio = DriveRatio(sample); _downshiftArmedThrottle = 0.0; } + // The ratio a gear holds the engine at is the lowest it shows: + // slip, a blip or an engine still falling from the previous gear + // can only read higher. A baseline taken from the first sample in + // a gear the driver had just shifted up into caught the engine + // before it had come down, read as high as the gear below, and so + // a live downshift that plainly took drive was never confirmed. + if (_downshiftCandidateGear == 0 && sample.Gear == _baselineGear + && sample.SpeedKmh > 5.0) + { + _baselineDriveRatio = Math.Min(_baselineDriveRatio, DriveRatio(sample)); + } // Arming already required a closed throttle, so anything above it // from here is the car blipping or a pedal the driver was asked not // to touch. Either way it is not throttle carried in from before. From 6d87c3c9a8fe843dd14a3cb0c19d24da88ca6986 Mon Sep 17 00:00:00 2001 From: Claude Date: Sat, 14 Nov 2026 01:06:28 +0000 Subject: [PATCH 9/9] Do not read the limiter in neutral as a cut either Shifted at the limiter, an unloaded engine has nowhere left to climb, so the rising-speed check missed it and a live H-pattern drive still showed two upshifts with a torque cut. A collapse while the box passed through neutral and the engine held its speed is now set aside the same way. The method text also says when too few upshifts were measured, rather than that a collapse on all of them does not establish a cut. Co-Authored-By: Claude Fable 5.2 Claude-Session: https://claude.ai/code/session_0114SHKuGF14miYjDXrsrGx5 --- docs/verification-observations.md | 4 +++- simhub/AsDriven.Core.Tests/Program.cs | 20 +++++++++++++++++++ simhub/AsDriven.Core/FreeDriveObserver.cs | 12 ++++++++++- .../AsDriven.Core/GuidedVerificationDrive.cs | 5 +++++ 4 files changed, 39 insertions(+), 2 deletions(-) diff --git a/docs/verification-observations.md b/docs/verification-observations.md index 29da0c9d..7de59281 100644 --- a/docs/verification-observations.md +++ b/docs/verification-observations.md @@ -133,7 +133,9 @@ own cut or blip can show on the published throttle just ahead of the gear change; those last moments only have to not be a lift. The thresholds are the scripted detector's own. A torque collapse on a flat upshift is not counted as a cut when engine speed climbed while it fell: that is an engine running free -between the gears of an H-pattern box, and it measures nothing about a cut +between the gears of an H-pattern box, and it measures nothing about a cut. +The same holds when the box passed through neutral and the engine held its +speed, as it does when shifted at the limiter (`engine_ran_free`). A shift counts once its gear has held for 600 ms, or for 300 ms when the next shift follows straight on, as it does going down the gears for a corner; only what happened before that next shift is read. A gear diff --git a/simhub/AsDriven.Core.Tests/Program.cs b/simhub/AsDriven.Core.Tests/Program.cs index 6d2bd61a..6be0429e 100644 --- a/simhub/AsDriven.Core.Tests/Program.cs +++ b/simhub/AsDriven.Core.Tests/Program.cs @@ -1977,6 +1977,26 @@ private static int Main(string[] args) Equal("unknown", hResults.AutomaticCut, "leaves the cut unknown when the engine only ran free"); + // The same box shifted at the limiter: the engine + // cannot climb any further, and holds its speed + // through neutral while the torque falls away. + clock = 0; + GuidedVerificationDrive limiter = FreeDrivingDrive(now); + FreeSteady(limiter, now, ref clock, 16, 1000, 2, 0, 100, 9450, 150, 600); + for (int gear = 3; gear <= 6; gear++) + { + double limitSpeed = 60 + gear * 30; + FreeSteady(limiter, now, ref clock, 16, 96, 0, 0, 100, 9450, limitSpeed, 100); + FreeSteady(limiter, now, ref clock, 16, 100, gear, 0, 100, 9450, limitSpeed, 100); + FreeSteady(limiter, now, ref clock, 16, 800, gear, 0, 100, 9450, limitSpeed + 5, 600); + } + limiter.Next(); + limiter.Next(); + Equal(0, limiter.GetResults().Telemetry.FreeDrive.WithTorqueCut, + "does not read the limiter in neutral as a cut"); + Equal("unknown", limiter.GetResults().AutomaticCut, + "leaves the cut unknown for shifts taken at the limiter through neutral"); + // A gear touched for an instant on the way past is // still not an accepted shift. clock = 0; diff --git a/simhub/AsDriven.Core/FreeDriveObserver.cs b/simhub/AsDriven.Core/FreeDriveObserver.cs index 1378ff07..2766f088 100644 --- a/simhub/AsDriven.Core/FreeDriveObserver.cs +++ b/simhub/AsDriven.Core/FreeDriveObserver.cs @@ -99,6 +99,9 @@ public sealed class FreeDriveObserver // Engine speed at the torque minimum, relative to when the gear was // left, above which the engine is taken to have run free. private const double FreeRevvingMargin = 1.03; + // Time between the last sample in one gear and the first in the next + // beyond which the box is taken to have passed through neutral. + private const double NeutralSeenMilliseconds = 40.0; private const double BufferMilliseconds = 2500.0; private const int BufferMaximumSamples = 300; @@ -281,8 +284,15 @@ private void Classify(PendingShift shift, GuidedTelemetrySample latest) // drive of such a car read as a cut on every upshift. GuidedTelemetrySample leaving = LastAtOrBefore(samples, left); GuidedTelemetrySample weakest = LowestTorque(samples, settled, to); + // Shifted at the limiter the engine has nowhere left + // to climb, so through neutral it only has to have + // held its speed: a cut with nothing driving the + // engine would let it fall. + bool throughNeutral = (arrived - left).TotalMilliseconds + >= NeutralSeenMilliseconds; if (collapsed && leaving != null && weakest != null - && weakest.Rpm > leaving.Rpm * FreeRevvingMargin) + && (weakest.Rpm > leaving.Rpm * FreeRevvingMargin + || (throughNeutral && weakest.Rpm >= leaving.Rpm * 0.99))) { _summary.EngineRanFree++; } diff --git a/simhub/AsDriven.Core/GuidedVerificationDrive.cs b/simhub/AsDriven.Core/GuidedVerificationDrive.cs index 6bf4ff11..7d7ba68c 100644 --- a/simhub/AsDriven.Core/GuidedVerificationDrive.cs +++ b/simhub/AsDriven.Core/GuidedVerificationDrive.cs @@ -1574,6 +1574,11 @@ private bool ApplyFreeDrive(FreeDriveSummary summary) : summary.CutMeasured == 0 ? "None published engine torque with the throttle held through the" + " change, so nothing was measured about a cut." + : summary.CutMeasured < FreeDriveShiftsToSettle + ? "Only " + summary.CutMeasured + " could be measured for a cut, of which " + + summary.WithTorqueCut + " showed a torque collapse; " + + FreeDriveShiftsToSettle + " are needed to establish one." + + SlowTelemetrySummary("an automatic cut", rate) : "Engine torque collapsed on " + summary.WithTorqueCut + " of " + summary.CutMeasured + " measured, which does not establish a cut." + SlowTelemetrySummary("an automatic cut", rate))