[bugfix]dynemb export supports input_tile=3#495
Conversation
| # so DynamicEmbLoad raises "can't find path to load" for | ||
| # the user-side ones. Symlink the missing twin paths. | ||
| input_tile = os.environ.get("INPUT_TILE", "") | ||
| if input_tile.startswith("3"): |
There was a problem hiding this comment.
Consider reusing is_input_tile_emb() from tzrec/acc/utils.py (already used in tzrec/utils/export_util.py) instead of re-deriving the INPUT_TILE=3 check inline. Keeps the env-var convention (input_tile[0] == "3") in one place and avoids drift if it changes.
| for i, seg in enumerate(segs): | ||
| if seg not in _INPUT_TILE_USER_SEGMENTS: | ||
| continue | ||
| user_segs = list(segs) | ||
| user_segs[i] = f"{seg}_user" | ||
| user_entry = ".".join(user_segs) | ||
| user_path = os.path.join(dynamicemb_path, user_entry) | ||
| if os.path.lexists(user_path): | ||
| continue | ||
| try: | ||
| os.symlink(entry, user_path) | ||
| logger.info( | ||
| f"created INPUT_TILE=3 dynamicemb symlink {user_entry} -> {entry}" | ||
| ) | ||
| except OSError as e: | ||
| logger.warning(f"failed to create dynamicemb symlink {user_entry}: {e}") |
There was a problem hiding this comment.
Minor defensive nit: if a single dotted entry ever contained more than one segment in _INPUT_TILE_USER_SEGMENTS, this loop would create N separate symlinks (each renaming a different segment). In practice the construction in EmbeddingGroupImpl / SequenceEmbeddingGroupImpl produces only one such segment per path, so this is theoretical — but adding a break after a successful (or skipped-because-exists) os.symlink would make the intent explicit and match the construction-side rename.
Code review summaryTargeted bugfix for the INPUT_TILE=3 export path: the checkpoint only stores non- Noteworthy:
Verified accurate:
|
No description provided.