Skip to content

Releases: nauticalab/orcapod-python

v0.1.0rc1

08 Apr 18:09
0473621

Choose a tag to compare

What's Changed

  • IDE setup by @guzman-raphael in #1
  • Add auto-tracking for automatic computation graph generation by @eywalker in #2
  • Improve hashing implementation, add convenience methods and add tutorial on Tracker usage by @eywalker in #4
  • DJ integration by @eywalker in #5
  • Use updated hashing logic by @eywalker in #6
  • Add improved tag and packet key inference, store logic, and content-based hashing by @eywalker in #8
  • Fix identity structure for query operations and add wip safe storage by @eywalker in #9
  • Refactor and add comprehensive testing for hashing by @eywalker in #14
  • Ruff formatting and turn hashing into a subpackage by @eywalker in #17
  • Refactor hashing and project structure by @eywalker in #18
  • Expanded tags and packets to accept any Python data type and use Arrow-based datastore by @eywalker in #22
  • Rename orcabridge to orcapod by @eywalker in #25
  • Add working pipeline system implementation by @eywalker in #27
  • Complete refactor of streams and datagram logic and use of protocols throughout by @eywalker in #34
  • Prepare for 0.1.0alpha release by @eywalker in #55
  • Enhance function_pod decorator to attach FunctionPod as callable attribute by @eywalker in #57
  • Add execution engine options to streams and nodes by @eywalker in #60
  • Fix bug to pass resource parameters to ray task options() by @brian-arnold in #62
  • Enhance pipeline functionality and access to pods by @eywalker in #63
  • feat(core): add packet function executor system and async channel execution by @eywalker in #67
  • Integrate TODO audit findings into DESIGN_ISSUES.md with priorities by @eywalker in #68
  • Add concrete asynchronous orchestrator by @eywalker in #71
  • Add native async streaming overrides for core operators by @eywalker in #70
  • Add async function support and fix concurrent packet execution by @eywalker in #72
  • Add comprehensive specification-derived test suite by @eywalker in #73
  • Fix hash_file return type and wire file_hasher through converters by @eywalker in #74
  • Claude/fix utc tzdata timestamp [ENG-225] by @brian-arnold in #75
  • Added GenericAliasHandler class [ENG-226] by @brian-arnold in #76
  • Integrate Ray execution engine and enable passing of options [ENG-231] by @brian-arnold in #77
  • Refactor source caching out of pipeline and add support for general caching for all sources by @eywalker in #80
  • feat: pipeline serialization and node simplification by @eywalker in #81
  • feat: add documentation site and clean up public API namespace by @eywalker in #82
  • refactor: major cleanup and simplification of protocols by @eywalker in #83
  • refactor(executors): improve function execution chain by @eywalker in #84
  • feat: add sync pipeline orchestrator with per-packet observability by @eywalker in #85
  • refactor: slim node protocols, observer injection, async orchestrator alignment (PLT-922) by @eywalker in #86
  • feat: add PacketFunctionProxy for read-only pipeline loading (PLT-931) by @eywalker in #87
  • Add infrastructure to capture and store logs: stdout, stderr, python logs, and tracebacks [ENG-264] by @brian-arnold in #89
  • feat(hashing): integrate starfix-python for Arrow schema and data hashing by @kurodo3[bot] in #90
  • feat: DBConnectorProtocol + ConnectorArrowDatabase + DBTableSource (PLT-1078) by @kurodo3[bot] in #91
  • feat(status): add StatusOberver [ENG-288] by @brian-arnold in #92
  • feat(observability): thread pipeline_path and pipeline_snapshot_hash through on_run_start by @kurodo3[bot] in #93
  • Add Upath support [ENG-235] by @brian-arnold in #94
  • Union type by @brian-arnold in #95
  • fix(ray): pass runtime_env to ray.init() instead of ray.remote() [ENG-294] by @brian-arnold in #97
  • feat(databases): implement SQLiteConnector — DBConnectorProtocol for SQLite (PLT-1076) by @kurodo3[bot] in #98
  • fix(function_node): restore within-node concurrent packet execution by @brian-arnold in #99
  • docs(logging_observer): fix and strengthen create_packet_logger Warning by @kurodo3[bot] in #102
  • feat(join): unify async_execute into N-way streaming MJoin [ENG-301] by @brian-arnold in #101
  • Update test docs for join operator by @brian-arnold in #103
  • fix(PLT-1156): add CACHE_ONLY mode to FunctionNode for unavailable upstream sources by @kurodo3[bot] in #104
  • fix(PLT-1158): uncached operator with UNAVAILABLE parents resolves to UNAVAILABLE by @kurodo3[bot] in #105
  • docs(spiraldb): add pyspiral optional dep and SpiralDB exploration findings by @kurodo3[bot] in #106
  • feat(spiraldb): implement SpiralDBConnector (PLT-1074) by @kurodo3[bot] in #109
  • feat(sources): implement SQLiteTableSource with ROWID fallback (PLT-1077) by @kurodo3[bot] in #110
  • feat(sources): SpiralDBTableSource with PK as default tag columns by @kurodo3[bot] in #111
  • feat(sources): PostgreSQLTableSource + executor/channel fixes (PLT-1072) by @kurodo3[bot] in #113
  • Restore Pipeline.show_graph, fix graph rendering, add tests by @eywalker in #114
  • fix(operator_node): enforce non-active node semantics (PLT-1182) by @kurodo3[bot] in #115
  • fix(types): Schema.repr shows clean type names instead of <class '...'> wrappers by @kurodo3[bot] in #116
  • fix: narrow Stream.flow() return type from Collection to Sequence/list by @kurodo3[bot] in #118
  • fix(ray): restore ray client functionality [ENG-339] by @brian-arnold in #119
  • perf: reduce import orcapod startup time from >1200ms to ~150ms (PLT-1178) by @kurodo3[bot] in #117
  • Observability reader [ENG-347] by @brian-arnold in #121
  • feat(ENG-258): add MinIO/S3 and UPath support for DeltaTableDatabase by @kurodo3[bot] in #122
  • feat(pipeline): normalized save format with DB deduplication and save levels (ENG-256/ENG-320/ENG-342) by @kurodo3[bot] in #120
  • feat(databases): add at() and base_path for sub-scoped database views (ENG-341) by @kurodo3[bot] in #123
  • feat(pipeline): add auto_save_path parameter to Pipeline for automatic pipeline saving by @brian-arnold in #124
  • refactor: decouple pipeline_path from nodes + observer contextualization (ENG-340 / ENG-349) by @kurodo3[bot] in #125
  • PLT-923: Schema ↔ Arrow nullability correspondence + logical equality tests by @kurodo3[bot] in #126
  • fix(ray): propagate original function name to Ray metrics instead of generic _capture [ENG-358] by @brian-arnold in #128
  • feat(ENG-373): add configurable table_scope to FunctionNode and OperatorNode by @kurodo3[bot] in #129
  • fix: restore Arrow nullable flags lost in Polars round-trips (ENG-375) by @kurodo3[bot] in https://github.com/n...
Read more