Skip to content

generator: discover items from XML when base.ini has no entry#48

Open
denis-coach wants to merge 3 commits into
Osiris-DevWorks:release/1.4.2from
denis-coach:feature/Read-items-from-XML-to-update-ini
Open

generator: discover items from XML when base.ini has no entry#48
denis-coach wants to merge 3 commits into
Osiris-DevWorks:release/1.4.2from
denis-coach:feature/Read-items-from-XML-to-update-ini

Conversation

@denis-coach
Copy link
Copy Markdown

Reverse the enhancement generator's skip logic: instead of silently discarding DataForge XML entities whose loc key is absent from base.ini, synthesize a description from XML attributes and emit them into the enhancement INI files. These entries flow through the existing "enhancements" source injection and get status "New" in the table.

Changes:

  • Add _synthesize_description() helper that builds rich descriptions from XML (career/role/crew/length for ships, item type for components, tracking type for missiles, filename stem as fallback)
  • scan_entity_dir() and scan_spaceships() no longer skip missing keys; they synthesize descriptions and emit discovered items
  • Mission title/desc assembly synthesizes from contract debug names when loc keys are missing from base.ini
  • _component_name_tag() falls back to XML ItemComponentParams for size/type when the description text lacks Size:/Grade:/Class: lines
  • Add shield/cooler/powerplant/qdrive/radar to _ITEM_TYPE_ABBREV so the tagger's fallback path works for all component types
  • Name tag generation synthesizes name values from XML file stems for discovered items whose name key is not in loc
  • append_enhancements() guards against None existing_value

Covers all category scanners: ships, components, weapons, missiles, FPS weapons, and missions. Crafting blueprints deferred (different discovery pattern).

17 new tests in test_discovered_items.py. Full suite: 578 passed, 4 pre-existing failures in test_pak_extraction.py.

h0useRus and others added 3 commits May 22, 2026 15:26
Reverse the enhancement generator's skip logic: instead of silently
discarding DataForge XML entities whose loc key is absent from base.ini,
synthesize a description from XML attributes and emit them into the
enhancement INI files. These entries flow through the existing
"enhancements" source injection and get status "New" in the table.

Changes:
- Add _synthesize_description() helper that builds rich descriptions
  from XML (career/role/crew/length for ships, item type for components,
  tracking type for missiles, filename stem as fallback)
- scan_entity_dir() and scan_spaceships() no longer skip missing keys;
  they synthesize descriptions and emit discovered items
- Mission title/desc assembly synthesizes from contract debug names
  when loc keys are missing from base.ini
- _component_name_tag() falls back to XML ItemComponentParams for
  size/type when the description text lacks Size:/Grade:/Class: lines
- Add shield/cooler/powerplant/qdrive/radar to _ITEM_TYPE_ABBREV so
  the tagger's fallback path works for all component types
- Name tag generation synthesizes name values from XML file stems
  for discovered items whose name key is not in loc
- append_enhancements() guards against None existing_value

Covers all category scanners: ships, components, weapons, missiles,
FPS weapons, and missions. Crafting blueprints deferred (different
discovery pattern).

17 new tests in test_discovered_items.py. Full suite: 578 passed,
4 pre-existing failures in test_pak_extraction.py.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Change the "New" status color from orange (#FF9800) to green (#66BB6A)
so discovered items are visually distinct from user modifications.

Add an "Include new lines" checkbox to the Config tab (Appearance group).
When unchecked (default), items with status "New" that have no user
override are excluded from the applied global.ini. When checked, they
flow into the output alongside enhanced and modified entries.

Setting persisted via AppSettings.get/set_include_new_lines().

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Status determination: enhancement-sourced keys missing from the global
source (base.ini) now correctly receive status "New" instead of falling
through to "Enhanced". The merge hierarchy includes "enhancements", so
all enhancement keys appeared in base_merged - the fix checks whether
the key exists in the base source before classifying.

Bomb compartments: add scan_entity_dir coverage for ships/bombcompartments/
(10 Eclipse/Retaliator/Gladiator/BEHR bomb racks). New enhancements_bomb_rack()
extracts Size, Grade, Bomb Slots, and Health from AttachDef +
SCItemMissileRackParams. _component_name_tag gains an AttachDef fallback
so numeric grades (1->A) and SubType ("BombRack" -> "BRK") feed into the
bracket annotation, producing tags like [BRK-S3-A].

UI: "Include new lines" checkbox moved from Appearance to Tools panel;
"New" status color changed to orange (#FF9800).
@Osiris-DevWorks Osiris-DevWorks self-requested a review May 22, 2026 20:21
@Osiris-DevWorks Osiris-DevWorks changed the base branch from main to release/1.4.2 May 22, 2026 20:22
Copy link
Copy Markdown
Owner

@Osiris-DevWorks Osiris-DevWorks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Walking through this PR. Leaving one note on the parser change. More may follow as I review the rest.

Comment thread src/parser/ini_parser.py
status = _determine_status_from_source(source, base_source)
# Discovered from DataForge XML — key only exists in enhancements,
# not in the global (base.ini) source.
if source == 'enhancements' and key not in base_sources.get(base_source, {}):
Copy link
Copy Markdown
Owner

@Osiris-DevWorks Osiris-DevWorks May 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New is being set in two places now — line 188 (if key not in base_merged) and this new branch.

src/parser/CLAUDE.md names _determine_status_from_source as the single source of truth for status mapping but it never actually returns 'New' since the signature can't tell whether the key is in stock base — the call site has to know.

Could we consolidate? Something like _determine_status_from_source(source_name, base_source, *, key_in_base: bool) with both New branches moved inside. Line 188 already split it before this PR so this isn't new debt, just a natural moment to clean up if you want.

@Osiris-DevWorks Osiris-DevWorks self-requested a review May 22, 2026 20:48
Comment thread src/gui/config_tab.py
tools_desc.setWordWrap(True)
tools_layout.addWidget(tools_desc)

self.include_new_cb = QCheckBox("Include new lines")
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Include new lines" is ambiguous. Let's use something a bit more descriptive like:

  • "Include discovered items"
  • "Include 'New' status entries"
  • "Apply XML-discovered items"

raw_type = icp.get("itemType", "")
# Map CamelCase XML itemType to the space-separated form
# used in _ITEM_TYPE_ABBREV (e.g. "ShieldGenerator" → "Shield Generator")
_ITEM_TYPE_XML_MAP = {
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This literal map is rebuilt on every call to _component_name_tag and duplicates the CamelCase-splitting logic that attach_class_name already does:

attach_class_name = re.sub(r"([a-z])([A-Z])", r"\1 \2", subtype)

Same approach would work here:

if raw_type:
    xml_item_type = re.sub(r"([a-z])([A-Z])", r"\1 \2", raw_type)

That removes the literal map, deduplicates the CamelCase split, and makes the path generic — any new XML itemType would work without an entry being added.

If the explicit map is intentional as a controlled allowlist (only let these five types through the fallback), worth a comment saying so. Otherwise the regex is cleaner. Minor.

gen_module.scan_entity_dir(
tmp_path, dummy_enhancement_fn, loc=loc, capture_all=False,
)
# No direct way to check the counter (it's local), but the function
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test has no assertion — the comment says "no direct way to check the counter (it's local)" and then just runs the function. As-is it passes regardless of whether discovery actually works.

Two ways out:

  1. Delete the test (it doesn't verify anything the other tests don't already cover).
  2. Rewrite to assert on the observable output:
result = gen_module.scan_entity_dir(
    tmp_path, dummy_enhancement_fn, loc=loc, capture_all=False,
)
assert "item_DescSHLD_A" in result  # discovered
assert "item_DescSHLD_B" in result  # in loc

That checks the function emits both even when the local counter itself is unreachable. Major.

class TestDiscoveredItemsInParser:
"""Covers that discovered items get 'New' status through the parser."""

def test_enhancement_only_key_gets_new_status(self, tmp_path):
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test imports load_source_files but never calls it. The assertions only check that the two INIs were parsed — they don't exercise the new branch in ini_parser.py:196 (the if source == 'enhancements' and key not in base_sources...: status = 'New' path), which is the actual change this PR makes to the parser.

To verify the New-status classification, something like:

entries = load_source_files(
    sources_dict={"global": base, "enhancements": enhancement},
    hierarchy=["global", "enhancements"],
    user_overrides=None,
)
discovered_b = next(e for e in entries if e.key == "item_DescSHLD_B")
assert discovered_b.status == "New"

existing_a = next(e for e in entries if e.key == "item_DescSHLD_A")
assert existing_a.status == "Enhanced"  # in base AND enhancements → Enhanced, not New

As-is, the parser change has no test coverage exercising the new branch. Major.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants