Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
Open CAFD Map gets better with every name someone adds. There are three ways to contribute:

- **A few lines in a `names\<ECU>.tsv` file.** Best for names you found yourself, or to fix or fill in anything the untrimmed names lack. These files win over every other source; the format is below.
- **Untrimmed dumps.** If you have untrimmed CAFD definitions (XML with every function, like the dumps in `definitions\untrimmed\` were made from), run `python ocm\untrimmed.py pack-dumps <folder>`. It merges them into `definitions\untrimmed\<CAFD id>.tsv.gz` by layout and never removes existing names. Open a pull request with the changed `.tsv.gz` files.
- **Untrimmed dumps.** If you have untrimmed CAFD definitions (XML with every function, like the dumps in `definitions\untrimmed\` were made from), run `python ocm\untrimmed.py pack-dumps <folder>`. For decrypted CAFDs from a psdzdata old enough to still carry its names, run `python ocm\untrimmed.py pack-caf <folder>` instead. Both merge into `definitions\untrimmed\<CAFD id>.tsv.gz` by layout and never remove existing names. Open a pull request with the changed `.tsv.gz` files.
- **Whole files in `definitions\`**: definition XMLs (`<CafdDefinition>`) or FDL cheat sheets (`<FDL>`) you already have. See [definitions/README.md](definitions/README.md). Their names are merged automatically. A definition XML also covers people whose psdzdata lacks that CAFD version.

## The format
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Current psdzdata ships CAFDs with every FDL name trimmed to `.`. E-Sys then show
- **Value labels from BMW's own coding rules.** Every value still carries the rule that selects it, and it's turned into readable text. For example `wert_02 [SA5AC]` means "value 2, coded when option 5AC is fitted". `[n/a here]` means the rule never applies to your reference vehicle class.
- **Names from the community.** `names\*.tsv` holds group, function and value names people have found: BimmerCode, screenshots, coding tables, backup diffs. Each name is keyed by CAFD id + datagroup + byte + mask, so it applies to every software version that has that function.
- **Contributed definition XMLs and cheat sheets.** `definitions\` holds definition files and cheat sheets people have shared, including the public community cheat sheets (GPL-3.0). `names\cheatsheet\` adds BMW names from a cheat sheet whose file can't be shipped. Their names are merged in the same way. A contributed definition is also used directly when your psdzdata doesn't have that CAFD version.
- **BMW's untrimmed names.** `definitions\untrimmed\` holds BMW's own group, function and value names (including the `aktiv` / `nicht_aktiv` labels of on/off options) from untrimmed CAFD dumps: about 149,000 functions, 274,000 values and 5,400 groups across 164 CAFD ids, covering G- and F-chassis ECUs. For G020 and F030, about 89% of functions across every CAFD version carry these verified names. They're keyed by each datagroup's exact function layout, so they apply to every software version with that layout and never land where bits moved. They're used twice: built into your definition DB (Backup Editor and E-Sys), and applied by the E-Sys agent at runtime, so even CAFD versions that were never exported get named when you open them.
- **BMW's untrimmed names.** `definitions\untrimmed\` holds BMW's own group, function and value names (including the `aktiv` / `nicht_aktiv` labels of on/off options) from untrimmed CAFD dumps: about 249,000 functions, 530,000 values and 9,500 groups across 313 CAFD ids, covering G- and F-chassis ECUs. For G020 and F030, about 89% of functions across every CAFD version carry these verified names. They're keyed by each datagroup's exact function layout, so they apply to every software version with that layout and never land where bits moved. They're used twice: built into your definition DB (Backup Editor and E-Sys), and applied by the E-Sys agent at runtime, so even CAFD versions that were never exported get named when you open them.
- **Option titles.** FP criteria show the factory option's title (`SA5AC High-beam assistant`), from `data\sa_titles.tsv` or your own decrypted ISTA database.

- **BMW-style placeholders** for everything not named yet: `<Group>_<start>_<end>_<mask>`, e.g. `PfFesMaster_9_9_255`.
Expand Down
Binary file added definitions/untrimmed/0000000F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000014.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000018.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000021.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000028.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000044.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000004C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000052.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000053.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000069.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000076.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000007E.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000008A.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000092.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000098.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000009C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000009D.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000A6.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000000AA.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000000AE.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000000B5.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000000B6.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000B7.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000B8.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000C9.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000D0.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000D8.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000DD.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000DE.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000000E6.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000EC.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000ED.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000000F5.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000000F9.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000122.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000012B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000012F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000138.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000147.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000160.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000163.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000168.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000171.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000186.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000001A7.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000001EE.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000001F6.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000001F9.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000001FD.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000201.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000205.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000219.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000223.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000023F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000251.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000025F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000026C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000286.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000028B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000029B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000002A5.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000002AA.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000002AF.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000002B9.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000002C9.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000324.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000033D.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000375.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000552.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000553.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000005B6.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000005B7.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000629.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000067B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000068D.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000694.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000006C6.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000006DE.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000721.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000760.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000794.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000007A1.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000007B2.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000007C3.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000007C8.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000819.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000832.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000893.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000896.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000089A.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000089D.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000008A2.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000008A6.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000008E5.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000008E8.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000909.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000090A.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000090B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000911.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000912.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000913.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000918.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000985.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000009C8.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000009EF.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000A07.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000A08.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000A3F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000A48.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000A4B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000B0A.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000B4B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000B52.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000B82.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000BCE.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000BD5.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000BF3.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000BFC.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000C18.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000C1E.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000C77.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000CE2.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000D37.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000D38.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000DBD.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000DDC.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000DDD.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000DED.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000E69.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000E74.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00000E75.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000ED9.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000F9B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00000FA2.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001016.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001043.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001060.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000106D.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001082.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001083.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000010BA.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001148.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000114B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001162.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000011AB.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000011AC.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000011AD.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000011C5.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000011D7.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000011FC.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000120C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000121C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000123C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000124F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000013D8.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000013F5.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000013FE.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001412.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001413.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000141F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000146B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000014A8.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000014CA.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001543.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000157F.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000163C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000169A.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000016BF.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000016EE.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000016F5.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000017BC.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000017BD.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000017BE.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000180B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000184A.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001872.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000019CC.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000019D7.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001A33.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001A4E.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001A79.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001AB7.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001AE4.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001B29.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001B2A.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001B2B.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001B2C.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001B2D.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001B43.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001B4E.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001B52.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001B6D.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001C61.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001C6C.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001C7A.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001C9D.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001D4A.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001D75.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001D92.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001D93.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001DF7.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001DF8.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001EBF.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001EF6.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001EFA.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001F3F.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00001F61.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001F86.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001FBF.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001FDA.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001FF2.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00001FFB.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002046.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002060.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002066.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000206D.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002081.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002089.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002098.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000211B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000222D.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002259.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/0000225A.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000225B.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000225C.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000022D3.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000022E0.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002458.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000249A.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000024C3.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000025A3.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002660.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000269F.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000026F7.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002742.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002781.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002795.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/0000297A.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002984.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/000029B7.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002AA4.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00002ABF.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002ACA.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/00002FAC.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000030FF.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000033E2.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00003860.tsv.gz
Binary file not shown.
Binary file modified definitions/untrimmed/00004579.tsv.gz
Binary file not shown.
Binary file added definitions/untrimmed/000047D6.tsv.gz
Binary file not shown.
51 changes: 51 additions & 0 deletions ocm/untrimmed.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
Packing (merges into existing files, never drops entries):
python ocm\untrimmed.py pack-dumps <folder> XML dumps (CafdDefinition with every function, e.g. Agent3353)
python ocm\untrimmed.py pack-caf <folder> decrypted psdzdata CAFDs (<caf> XML) that still carry their names
python ocm\untrimmed.py pack-db <sqlite> [src..] names already in a definition DB (default sources: untrimmed fwl)
python ocm\untrimmed.py stats
"""
Expand Down Expand Up @@ -142,6 +143,54 @@ def pack_dumps(folder):
print(f"{len(per)} CAFD ids from {folder} -> {PACK_DIR} ({total} rows)")


CAF_NS = {"c": "http://bmw.com/2005/psdz.data.caf"}
CAF_FILE = re.compile(r"(?:cafd_)?([0-9A-Fa-f]{8})[._]")


def pack_caf(folder):
"""Decrypted psdzdata CAFDs (<caf> XML: cafd_<id>.caf.<ver>.xml or <id>_<ver>.xml) from a psdzdata old enough to
still carry its names. Only the names are read; the files themselves must never be committed."""
per = defaultdict(lambda: defaultdict(lambda: {"G": None, "F": {}, "V": {}}))
clean = lambda n: re.sub(r"\s+", " ", n or "").strip()
n = 0
for p in sorted(glob.glob(os.path.join(folder, "*.xml"))):
m = CAF_FILE.match(os.path.basename(p))
if not m:
continue
try:
data = ET.parse(p).getroot().find("c:data", CAF_NS)
except ET.ParseError as e:
print(f" skipped {os.path.basename(p)}: {e}", file=sys.stderr)
continue
n += 1
for dg in data.findall("c:datagroup", CAF_NS) if data is not None else ():
fns = []
for fn in dg.findall("c:function", CAF_NS):
si = fn.find("c:storageInfo", CAF_NS)
st = int(si.findtext("c:start", "0", CAF_NS))
fns.append((st, int(si.findtext("c:end", str(st), CAF_NS)),
(si.findtext("c:mask", "", CAF_NS) or "").strip(), fn))
e = per[m.group(1).upper()][((dg.get("address") or "").upper(), layout_hash([f[:3] for f in fns]))]
if not placeholder(clean(dg.get("name"))):
e["G"] = clean(dg.get("name"))
for st, en, mk, fn in fns:
k = (st, en, norm_mask(mk))
if not placeholder(clean(fn.get("name"))):
e["F"][k] = clean(fn.get("name"))
seen = set()
for cp in fn.findall("c:condParam", CAF_NS):
uvs = cp.find("c:values", CAF_NS)
uvs = [] if uvs is None else sorted(uvs.findall("c:unitVal", CAF_NS),
key=lambda u: int(u.get("position", "0")))
hx = "".join((u.text or "").strip().upper() for u in uvs)
name = clean(cp.findtext("c:paramName", "", CAF_NS))
if hx and hx not in seen and not placeholder(name): # first name wins when values share a code
seen.add(hx)
e["V"][k + (hx,)] = name
total = sum(merge(cid, found) for cid, found in per.items())
print(f"{n} CAFDs, {len(per)} CAFD ids from {folder} -> {PACK_DIR} ({total} rows)")


def pack_db(db, sources=("untrimmed", "fwl")):
con = sqlite3.connect(f"file:{db}?mode=ro", uri=True)
src = "(" + ",".join("?" * len(sources)) + ")"
Expand Down Expand Up @@ -188,6 +237,8 @@ def stats():
cmd = sys.argv[1] if len(sys.argv) > 1 else ""
if cmd == "pack-dumps" and len(sys.argv) > 2:
pack_dumps(sys.argv[2])
elif cmd == "pack-caf" and len(sys.argv) > 2:
pack_caf(sys.argv[2])
elif cmd == "pack-db" and len(sys.argv) > 2:
pack_db(sys.argv[2], tuple(sys.argv[3:]) or ("untrimmed", "fwl"))
elif cmd == "stats":
Expand Down