Skip to content

[MAINTENANCE] Python CommonLib: add package structure, fix imports, and expand Python test suite #130

@yunlishao-vibe

Description

@yunlishao-vibe

Task Summary

The Python CommonLib (ConfigHelper.py, MsgHelper.py, SocketHelper.py, VehDataMsgDefs.py) lacks a proper package structure, has inconsistent import patterns, and the Python test suite under tests/Python/ needs expansion to cover core message and socket functionality.

Why is this needed?

Currently:

  • No __init__.py in CommonLib/ — it's not a proper Python package, imports rely on sys.path.insert() hacks
  • Both tests/Python/SimpleEchoClient/simple_echo_client.py and tests/Python/SimpleTrafficLight/echo_client.py duplicate the sys.path.insert pattern manually
  • SocketHelper.py has a # TODO: add traffic light and detector data comment
  • Python tests only validate socket echo behavior — no unit tests for message packing/unpacking
  • MsgHelper.py at 28K lines is likely the largest file in the Python codebase and has never been unit tested
  • requirements.txt references outdated SUMO version numbers in comments and is inconsistent with environment.yml
  • Two separate conda environment definitions (environment.yml and requirements.txt) with different package lists

Scope of Work

1. Add Python package structure

  • Add CommonLib/__init__.py exposing key classes
  • Update all test files to use clean from CommonLib.X import Y imports
  • Remove sys.path hacks from test scripts

2. Fix requirements files

  • Reconcile requirements.txt and environment.yml — one should be authoritative
  • Remove outdated version comments in requirements.txt (e.g., vissim==11.0, carmaker==10.0.1)
  • Ensure environment.yml includes all packages needed to run Python tests without SUMO/VISSIM/CarMaker installed

3. Implement SocketHelper.py TODO

  • The # TODO: add traffic light and detector data in SocketHelper.py — implement or convert to proper issue

4. Expand Python unit tests

Add tests/Python/unit/ directory with:

  • test_msg_helper.py — test message packing/unpacking with known byte sequences
  • test_config_helper.py — test YAML config parsing with sample config files
  • test_socket_helper.py — test message header parsing (no live socket needed, use mock data)

5. Add CI-runnable test runner

  • Add tests/Python/run_unit_tests.py (or pytest config) that runs all unit tests without requiring any simulator
  • These should be runnable with: python -m pytest tests/Python/unit/

Files Involved

  • CommonLib/__init__.py (new)
  • CommonLib/SocketHelper.py
  • CommonLib/MsgHelper.py
  • CommonLib/ConfigHelper.py
  • CommonLib/VehDataMsgDefs.py
  • requirements.txt
  • environment.yml
  • tests/Python/ (all)
  • tests/Python/unit/ (new)

Acceptance Criteria

  • CommonLib/__init__.py exists and package imports work cleanly
  • No sys.path.insert hacks in test files
  • requirements.txt and environment.yml are consistent
  • tests/Python/unit/test_msg_helper.py passes with pytest
  • tests/Python/unit/test_config_helper.py passes with pytest
  • SocketHelper.py TODO resolved or tracked as issue
  • All unit tests runnable without any simulator installed

Branch and PR Target

  • All PRs for this issue should target the dev branch (not main)
  • First ensure main is merged into dev before starting work

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions