Repository navigation
AP_Mount: fix lat/lon precision loss in Xacti - #34925
Conversation
| xacti_gnss_status_msg.utc_seconds = sec; | ||
| xacti_gnss_status_msg.latitude = loc.lat * 1e-7; | ||
| xacti_gnss_status_msg.longitude = loc.lng * 1e-7; | ||
| xacti_gnss_status_msg.latitude = (double)loc.lat / 1.0e7; |
There was a problem hiding this comment.
thanks very much for this. I guess the fix is the cast to double?
It's fine but I guess we could stick with the, "* 1e-7" part instead of changing to a divide.
There was a problem hiding this comment.
1e-7 is treated as a float (due to -fsingle-precision-constant) and becomes 1.0000000117e-7.
I built and tested each value using Claude and compared the results.
| Expression | Longitude example (expected 139.7671250) | Issue |
|---|---|---|
loc.lng * 1e-7 |
139.767135620 | Rounded to steps of about 1.4 m |
(double)loc.lng * 1e-7 |
139.767126633 | The steps are gone, but there is always an offset of about 0.15 m to the east |
(double)loc.lng / 1.0e7 |
139.767125000 | No error |
Global Size Summary (Total Flash delta in bytes)Built from 415e61c rebased onto 7246115
|
|
Automated review note — AI-generated (Claude+Codex), validated against the live diff. Please sanity-check before acting. Reviewed at head This two-line fix looks good to merge: dividing the int32 coordinates by 1.0e7 in double precision gives the Xacti float64 lat/lon fields full 1e-7 deg precision, where the old float multiply lost up to about 1.8 m. The author's reply to the cast-only suggestion checks out too, since keeping Note
Not fully reviewed: behaviour on a real Xacti camera or DroneCAN bus, and builds for single-precision-FPU boards, were not exercised. |
peterbarker
left a comment
There was a problem hiding this comment.
LGTM.
This is only one of many, many places that would require the same fix.
-fsingle-precision-constant is a mistake IMO. Not when we introduced it, but now with new tooling and whatnot I think we can do better.
Summary
Fix loss of precision in the latitude/longitude sent to Xacti cameras in the GnssStatus DroneCAN message.
Classification & Testing (check all that apply and add your own)
Took several photos at the same spot while the GPS position was jittering slightly:
Description
com.xacti.GnssStatusdefines latitude and longitude asfloat64, but they were calculated asloc.lat * 1e-7.This rounds the longitude to steps of about 1.5e-5° (~1.4 m).
This PR casts to double before dividing, so the full precision of the int32 location is kept.