Improve execution performance in RequestManager validation and Execution.has_ended

[pedroigorjs]

Summary

This PR improves rule execution performance by reducing overhead in two hot paths:

• replaces pandera-based DataFrame validation in RequestManager.validate() with a lightweight pandas-based validation flow
• optimizes Execution.has_ended() by switching from a full null-count scan to a short-circuit null check

What Changed

• RequestManager
  • precomputes a lightweight schema as a plain dict instead of a pandera.DataFrameSchema
  • validates required columns manually
  • handles null/default substitution with vectorized pandas operations
  • coerces validated input columns to str
• Execution
  • changes has_ended() from .isna().sum() == 0 to not .isna().any()
• Tests
  • updates the RequestManager test to match the new lightweight schema representation

Motivation

The goal is to reduce validation overhead during rule execution, especially in smaller and latency-sensitive workloads where fixed framework costs dominate total runtime.

Local performance notes in the repository indicate that input validation was one of the main execution bottlenecks, and that has_ended() is called repeatedly during execution. These changes target both paths directly.

Testing

• poetry run pytest tests/test_engine/test_request_manager.py

Risks

• This PR changes validation semantics for DataFrame inputs:
  • "", "null" and "None" are now treated as null-like values during DataFrame validation
  • previously, under pandera, those values were accepted as valid strings
• If existing rules or clients rely on those literal string values, this change may be behavior-breaking

Notes

This PR is intentionally small and focused on committed performance changes only:

• lightweight RequestManager validation
• has_ended() short-circuit optimization

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
retrack/engine
base.py	70	5	93%	48, 62–64, 152
executor.py	131	10	92%	62, 74–75, 153, 225, 271, 289, 293, 305, 313
request_manager.py	80	11	86%	16, 35, 54, 69–70, 149, 152, 171–176
rule.py	50	3	94%	87, 109, 116
retrack/nodes
check.py	41	3	93%	23, 26, 89
constants.py	93	3	97%	123, 126, 171
math.py	43	1	98%	75
retrack/nodes/dynamic
base.py	16	1	94%	26
csv_table.py	44	2	95%	60, 63
flow.py	44	1	98%	18
flow_connector.py	25	12	52%	24–48
retrack/utils
component_registry.py	105	18	83%	77–78, 81–82, 85–86, 91–92, 101, 112–123
exceptions.py	28	5	82%	17–18, 106–127
graph.py	60	7	88%	35, 46, 59, 61, 63, 84, 86
registry.py	34	6	82%	23–26, 38, 43, 51
transformers.py	156	16	90%	17, 37–38, 52, 57–58, 63, 66, 69, 72, 75, 109, 169, 178, 227, 353
retrack/validators
base.py	4	1	75%	14
node_exists.py	15	2	87%	36, 38
node_validator.py	33	3	91%	31, 56–57
TOTAL	1636	110	93%

Tests	Skipped	Failures	Errors	Time
92	0 💤	0 ❌	0 🔥	4.206s ⏱️

[github-actions]

Coverage Report

File	Stmts	Miss	Cover	Missing
retrack/engine
base.py	70	5	93%	48, 62–64, 152
executor.py	131	10	92%	62, 74–75, 153, 225, 271, 289, 293, 305, 313
request_manager.py	80	11	86%	16, 35, 54, 69–70, 149, 152, 171–176
rule.py	50	3	94%	87, 109, 116
retrack/nodes
check.py	41	3	93%	23, 26, 89
constants.py	93	3	97%	123, 126, 171
math.py	43	1	98%	75
retrack/nodes/dynamic
base.py	16	1	94%	26
csv_table.py	44	2	95%	60, 63
flow.py	44	1	98%	18
flow_connector.py	25	12	52%	24–48
retrack/utils
component_registry.py	105	18	83%	77–78, 81–82, 85–86, 91–92, 101, 112–123
exceptions.py	28	3	89%	106–127
graph.py	60	7	88%	35, 46, 59, 61, 63, 84, 86
registry.py	34	6	82%	23–26, 38, 43, 51
transformers.py	156	16	90%	17, 37–38, 52, 57–58, 63, 66, 69, 72, 75, 109, 169, 178, 227, 353
retrack/validators
base.py	4	1	75%	14
node_exists.py	15	2	87%	36, 38
node_validator.py	33	3	91%	31, 56–57
TOTAL	1636	108	93%

Tests	Skipped	Failures	Errors	Time
92	0 💤	0 ❌	0 🔥	3.810s ⏱️