Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/en/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,32 @@ pt-snap query --template-use leak_detection --params '{"min_size": 1024}'
`min_size` is the minimum candidate size in bytes and defaults to `0`. Select
the target device with the command-level `--device` option, not inside `--params`.

## Parameter Validation

`--params` is validated against the template before any SQL is rendered:

- Every key must be a parameter the template declares. Unknown keys are
rejected instead of ignored, so a misspelled filter such as `min_sze` fails
with `Unknown parameter(s) for template 'leak_detection': min_sze (accepted:
min_size, limit)` rather than silently returning unfiltered results.
- Values are converted to the declared type (`int`, `float`, `str`, `bool`).
- Parameters that are rendered into SQL as identifiers or keywords, such as
`order_by` and `order_dir`, accept only the values listed under
`[choices: ...]` in `--template-info`. String choices match
case-insensitively and render with the declared spelling (`desc` becomes
`DESC`); anything else is rejected before it reaches the database.

```bash
pt-snap query --template-info allocation
# order_by: str (optional) [choices: id, allocated, active, reserved] [default: id]
# order_dir: str (optional) [choices: ASC, DESC] [default: ASC]

pt-snap query --template-use allocation --params '{"order_by": "reserved", "order_dir": "desc"}' -n 5
```

The same rules apply to `SnapshotAnalyzer.execute_query()` and the MCP
`execute_query` tool, which raise `TemplateRenderError` with the same message.

## Peak Memory Attribution Workflow

These additions productize the common "find the peak, then explain what was live at that moment" workflow.
Expand Down Expand Up @@ -202,6 +228,7 @@ hexadecimal address strings are required.
Query templates are defined in YAML format with:
- `version`: Template version
- `queries`: Query definitions with description, supported devices, parameters, SQL (Jinja2 templated), and output schema
- Each parameter declares `type`, `default`, `required`, `description`, and optionally `choices`, a closed list of accepted values that is mandatory for parameters rendered as SQL identifiers or keywords

When passed explicitly to `ResultMapper`, recognized mapping types are `int`,
`float`, `str`, `bool`, `hex`, and `datetime`; `datetime` is currently a
Expand Down
26 changes: 26 additions & 0 deletions docs/zh/querying.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,31 @@ pt-snap query --template-use leak_detection --params '{"min_size": 1024}'
`min_size` 表示候选泄漏的最小字节数,默认值为 `0`。目标设备应通过命令级
`--device` 选项指定,而不是放进 `--params`。

## 参数校验

`--params` 会在渲染任何 SQL 之前按模板定义进行校验:

- 每个键都必须是模板声明的参数。未声明的键会被拒绝而不是忽略,因此把
`min_size` 拼成 `min_sze` 会得到 `Unknown parameter(s) for template
'leak_detection': min_sze (accepted: min_size, limit)`,而不是悄悄返回一份
未过滤的结果。
- 值会按声明的类型(`int`、`float`、`str`、`bool`)转换。
- 会被当作 SQL 标识符或关键字渲染的参数(如 `order_by`、`order_dir`)只接受
`--template-info` 中 `[choices: ...]` 列出的取值。字符串取值不区分大小写,
并按声明的写法渲染(`desc` 会变成 `DESC`);其他任何值都会在到达数据库之前
被拒绝。

```bash
pt-snap query --template-info allocation
# order_by: str (optional) [choices: id, allocated, active, reserved] [default: id]
# order_dir: str (optional) [choices: ASC, DESC] [default: ASC]

pt-snap query --template-use allocation --params '{"order_by": "reserved", "order_dir": "desc"}' -n 5
```

`SnapshotAnalyzer.execute_query()` 和 MCP 的 `execute_query` 工具遵循同样的规则,
并以相同的错误信息抛出 `TemplateRenderError`。

## 峰值内存归因工作流

这些新增能力把“先找到峰值,再解释峰值时刻哪些内存仍然活跃”的手工分析流程产品化了。
Expand Down Expand Up @@ -194,6 +219,7 @@ CLI、Python API 和 MCP 的查询结果包含原始 SQLite 值。模板的 `out
查询模板使用 YAML 格式定义,包含:
- `version`: 模板版本
- `queries`: 查询定义,包含描述、支持的设备、参数、SQL(Jinja2 模板语法)和输出 schema
- 每个参数声明 `type`、`default`、`required`、`description`,以及可选的 `choices`(允许取值的封闭列表;会被渲染为 SQL 标识符或关键字的参数必须声明它)

显式传给 `ResultMapper` 时,可识别的映射类型包括 `int`、`float`、`str`、
`bool`、`hex` 和 `datetime`;当前 `datetime` 只是透传声明,不执行解析。
Expand Down
1 change: 1 addition & 0 deletions src/pt_snap_cli/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -41,18 +41,18 @@
*,
context_cache: ContextCache | None = None,
) -> None:
self._config = Config()

Check warning on line 44 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_config` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)
self._db_path = db_path

Check warning on line 45 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_db_path` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)
self._device_id = device_id

Check warning on line 46 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_device_id` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)
self._focus_service = FocusService(self._config)

Check warning on line 47 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_focus_service` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)
# Share one context cache across the analyzer so that long-lived
# analyzers (e.g. the MCP server singleton) reuse a single
# SQLite connection and skip schema validation on every query.
# Explicit ``is not None`` because an empty cache is falsy via
# ``__len__`` and would be silently replaced otherwise.
self._context_cache = context_cache if context_cache is not None else ContextCache()

Check warning on line 53 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_context_cache` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)
self._query_service = QueryService(self._focus_service, context_cache=self._context_cache)

Check warning on line 54 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_query_service` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)
self._metadata_service = ImportMetadataService()

Check warning on line 55 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type annotation for attribute `_metadata_service` is required because this class is not decorated with `@final` (reportUnannotatedClassAttribute)

@property
def context_cache(self) -> ContextCache:
Expand Down Expand Up @@ -92,7 +92,7 @@

if candidate_db is not None:
try:
self._focus_service.validate_session_db(candidate_db, candidate_device)

Check warning on line 95 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Result of call expression is of type "FocusState" and is not used; assign to variable "_" if this is intentional (reportUnusedCallResult)
except DatabaseMissingError as exc:
raise FileNotFoundError(str(exc)) from exc
except DatabaseSchemaError as exc:
Expand All @@ -105,7 +105,7 @@
self._device_id = device_id
return self.get_focus()

def list_templates(self, category: str | None = None) -> list[dict[str, Any]]:

Check warning on line 108 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type `Any` is not allowed (reportExplicitAny)
return [
{
"name": template.name,
Expand All @@ -115,7 +115,7 @@
for template in self._query_service.list_templates(category)
]

def get_template_info(self, name: str) -> dict[str, Any] | None:

Check warning on line 118 in src/pt_snap_cli/api.py

View workflow job for this annotation

GitHub Actions / Lint & Format

Type `Any` is not allowed (reportExplicitAny)
try:
info = self._query_service.get_template_info(name)
except TemplateNotFoundError:
Expand All @@ -132,6 +132,7 @@
"default": param.default,
"required": param.required,
"description": param.description,
"choices": param.choices,
}
for param_name, param in info.parameters.items()
},
Expand Down
16 changes: 12 additions & 4 deletions src/pt_snap_cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -425,13 +425,18 @@ def query_database(
if info.parameters:
for param_name, param_details in info.parameters.items():
required_str = " (required)" if param_details.required else " (optional)"
choices_str = (
f" [choices: {', '.join(str(choice) for choice in param_details.choices)}]"
Comment thread
codeasier marked this conversation as resolved.
Resolved
if param_details.choices
else ""
)
default_str = (
f" [default: {param_details.default}]"
if param_details.default is not None
else ""
)
typer.secho(
f" {param_name}: {param_details.type}{required_str}{default_str}",
f" {param_name}: {param_details.type}{required_str}{choices_str}{default_str}",
fg=typer.colors.YELLOW,
)
typer.echo(f" {param_details.description}")
Expand All @@ -457,9 +462,12 @@ def query_database(
param_details.default if param_details.default is not None else 0.0
)
elif param_details.type == "str":
example_params[param_name] = (
param_details.default if param_details.default is not None else "example"
)
if param_details.default is not None:
example_params[param_name] = param_details.default
elif param_details.choices:
example_params[param_name] = param_details.choices[0]
else:
example_params[param_name] = "example"
elif param_details.type == "bool":
example_params[param_name] = True
if example_params:
Expand Down
1 change: 1 addition & 0 deletions src/pt_snap_cli/core/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ class TemplateParameter:
default: object | None
required: bool
description: str
choices: list[object] | None = None


@dataclass(frozen=True)
Expand Down
1 change: 1 addition & 0 deletions src/pt_snap_cli/core/query_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,7 @@ def get_template_info(self, name: str) -> TemplateInfo:
default=param_details["default"],
required=param_details["required"],
description=param_details["description"],
choices=param_details.get("choices"),
)
for param_name, param_details in info["parameters"].items()
},
Expand Down
1 change: 1 addition & 0 deletions src/pt_snap_cli/query/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@
- Variant SQL is selected from `Context.callstack_layout`; unknown or conflicting
layouts must fail variant templates with a layout-specific error rather than a
generic re-import message. Non-variant templates keep working.
- `QueryTemplate.validate_params()` rejects undeclared parameter names and enforces `QueryParameter.choices`; extra render-context variables (`device_id`, table names, the pushed-down `limit`) are injected by `QueryExecutor.render()` after validation, not passed through `params`.

### Testing Requirements
- Run `pytest tests/query` for query subsystem changes; real SQLite template semantics live in `test_peak_memory_templates.py`, `test_callstack_schema_compat.py`, and `test_query_max_rows_pushdown.py`.
Expand Down
65 changes: 57 additions & 8 deletions src/pt_snap_cli/query/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,35 @@ class QueryParameter:
default: Any = None
required: bool = False
description: str = ""
# Closed set of accepted values. Required for parameters that are rendered
# into SQL as identifiers or keywords (e.g. ORDER BY column / direction),
# where the type alone cannot bound what reaches the database.
choices: list[Any] | None = None

def __post_init__(self) -> None:
if self.choices is None:
return
if not isinstance(self.choices, list) or not self.choices:
raise ValueError(f"Parameter '{self.name}' choices must be a non-empty list")
if self.default is None:
return
matched = self._match_choice(self.default)
if matched is None:
raise ValueError(
Comment thread
codeasier marked this conversation as resolved.
Resolved
f"Parameter '{self.name}' default {self.default!r} is not one of its choices"
)
self.default = matched

def _match_choice(self, value: Any) -> Any | None:
"""Return the canonical choice equal to ``value``, or None when not allowed."""
if self.choices is None:
return value
if self.type == "str" and isinstance(value, str):
for choice in self.choices:
if isinstance(choice, str) and choice.lower() == value.lower():
return choice
return None
return value if value in self.choices else None

def validate(self, value: Any) -> Any:
"""Validate and convert parameter value.
Expand All @@ -28,11 +57,14 @@ def validate(self, value: Any) -> Any:
value: Value to validate.

Returns:
Converted value.
Converted value. For ``str`` parameters with ``choices`` the
canonical spelling from ``choices`` is returned, so case-insensitive
input (``"desc"``) renders as declared (``"DESC"``).

Raises:
TypeError: If value cannot be converted to expected type.
ValueError: If required parameter is missing.
ValueError: If required parameter is missing or the value is not one
of the declared choices.
"""
if value is None:
if self.required:
Expand All @@ -48,12 +80,20 @@ def validate(self, value: Any) -> Any:

converter = type_converters.get(self.type, str)
try:
return converter(value)
converted = converter(value)
except (ValueError, TypeError) as e:
raise TypeError(
f"Parameter '{self.name}' cannot be converted to {self.type}: {e}"
) from e

if self.choices is None:
return converted
matched = self._match_choice(converted)
if matched is None:
allowed = ", ".join(str(choice) for choice in self.choices)
raise ValueError(f"Parameter '{self.name}' must be one of: {allowed} (got {value!r})")
return matched


@dataclass
class QueryTemplate:
Expand Down Expand Up @@ -91,18 +131,26 @@ def validate_params(self, params: dict[str, Any]) -> dict[str, Any]:
Validated and converted parameters.

Raises:
ValueError: If required parameter is missing.
ValueError: If a required parameter is missing, a value is not one
of the parameter's declared choices, or ``params`` contains a
name the template does not declare. Unknown names are rejected
rather than ignored so that a misspelled filter cannot silently
widen a query.
TypeError: If parameter type is invalid.
"""
unknown = sorted(name for name in params if name not in self.parameters)
if unknown:
accepted = ", ".join(self.parameters) if self.parameters else "none"
raise ValueError(
f"Unknown parameter(s) for template '{self.name}': {', '.join(unknown)} "
f"(accepted: {accepted})"
)

validated = {}
for name, param_def in self.parameters.items():
value = params.get(name)
validated[name] = param_def.validate(value)

for name, value in params.items():
if name not in self.parameters:
validated[name] = value

return validated

@classmethod
Expand All @@ -125,6 +173,7 @@ def from_dict(cls, data: dict[str, Any], default_category: str | None = None) ->
default=param_data.get("default"),
required=param_data.get("required", False),
description=param_data.get("description", ""),
choices=param_data.get("choices"),
)

template_name = data.get("name", "")
Expand Down
1 change: 1 addition & 0 deletions src/pt_snap_cli/query/registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,6 +204,7 @@ def get_template_info(name: str) -> dict[str, Any] | None:
"default": param.default,
"required": param.required,
"description": param.description,
"choices": param.choices,
}
for param_name, param in template.parameters.items()
},
Expand Down
2 changes: 2 additions & 0 deletions src/pt_snap_cli/query/templates/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@
### Working In This Directory
- Keep each YAML template's `queries` entry name aligned with the file/topic and documented template name.
- Include accurate parameter defaults, required flags, descriptions, and output schema entries.
- Any parameter rendered as a SQL identifier or keyword (`order_by`, `order_dir`, or similar) must declare `choices`; `order_by` choices must be columns present in `output_schema`. `tests/query/test_registry.py` enforces this for packaged templates. Empty `choices`, a non-list `choices`, or a default outside the list raise at `QueryParameter` construction; `_load_yaml_templates` warns and skips that YAML file, so the template is absent rather than listed.
- Callers cannot pass undeclared parameters; `QueryTemplate.validate_params()` rejects unknown names, so declare every input the SQL reads.
- Use Jinja variables provided by `QueryExecutor`, especially device-specific table names.
- Make ranked/grouped results deterministic with explicit tie-breakers and keep every selected result column in `output_schema`.

Expand Down
2 changes: 2 additions & 0 deletions src/pt_snap_cli/query/templates/basic/allocation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,13 @@ queries:
type: str
default: "id"
required: false
choices: [id, allocated, active, reserved]
description: Column to order by
order_dir:
type: str
default: "ASC"
required: false
choices: [ASC, DESC]
description: Sort direction (ASC or DESC)
limit:
type: int
Expand Down
2 changes: 2 additions & 0 deletions src/pt_snap_cli/query/templates/basic/block.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -69,11 +69,13 @@ queries:
type: str
default: "size"
required: false
choices: [id, address, size, requestedSize, state, allocEventId, freeEventId]
description: Column to order by
order_dir:
type: str
default: "DESC"
required: false
choices: [ASC, DESC]
description: Sort direction (ASC or DESC)
limit:
type: int
Expand Down
2 changes: 2 additions & 0 deletions src/pt_snap_cli/query/templates/basic/event.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,13 @@ queries:
type: str
default: "id"
required: false
choices: [id, action, address, size, stream, allocated, active, reserved, callstack]
description: Column to order by
order_dir:
type: str
default: "ASC"
required: false
choices: [ASC, DESC]
description: Sort direction (ASC or DESC)
limit:
type: int
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,13 @@ queries:
type: str
default: "size"
required: false
choices: [size, requestedSize, allocEventId, freeEventId, id]
description: Sort key (size, requestedSize, allocEventId, freeEventId, id)
order_dir:
type: str
default: "DESC"
required: false
choices: [ASC, DESC]
description: Sort direction (ASC or DESC)
limit:
type: int
Expand Down
Loading
Loading