-
Notifications
You must be signed in to change notification settings - Fork 19
feat: add JSON datasource #222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
|
|
@@ -38,16 +38,15 @@ AIMBAT is a seismological tool for automated and interactive measurement of body | |||||
| ``` | ||||||
| src/aimbat/ | ||||||
| ├── app.py # Cyclopts CLI root — registers all subcommands | ||||||
| ├── cli/ # CLI command definitions (thin layer, delegates to core/) | ||||||
| ├── _cli/ # CLI command definitions (thin layer, delegates to core/) | ||||||
| ├── core/ # Business logic: ICCS/MCCC algorithms, event/seismogram ops | ||||||
| │ ├── _active_event.py # Manages the single active event constraint | ||||||
| │ ├── _data.py # SAC ingestion entry point | ||||||
| │ ├── _iccs.py # ICCS alignment (wraps pysmo.tools.iccs) | ||||||
| │ └── _snapshot.py # Parameter state capture for rollback/comparison | ||||||
| ├── models/ # SQLModel ORM definitions (Events, Seismograms, Stations, etc.) | ||||||
| │ └── _sqlalchemy.py # SAPandasTimestamp / SAPandasTimedelta type decorators | ||||||
| ├── aimbat_types/ # Custom Pydantic types (PydanticTimestamp, enums for parameters) | ||||||
| ├── io/ # File I/O — _base.py defines abstract base; _sac.py implements SAC via pysmo | ||||||
| ├── _types/ # Custom Pydantic types (PydanticTimestamp, enums for parameters) | ||||||
|
smlloyd marked this conversation as resolved.
|
||||||
| ├── io/ # File I/O — _base.py defines abstract base; sac.py implements SAC via pysmo | ||||||
| ├── utils/ # Shared helpers (JSON→table, UUID truncation, styling, sample data) | ||||||
| ├── _config.py # Global Settings (pydantic-settings, env prefix AIMBAT_) | ||||||
| ├── _lib/ # Internal mixins (EventParametersValidatorMixin) | ||||||
|
|
@@ -89,11 +88,11 @@ Settings live in `_config.py` as a `pydantic-settings` class. All settings can b | |||||
|
|
||||||
| ### CLI Pattern | ||||||
|
|
||||||
| Each CLI module in `cli/` creates a Cyclopts `App` instance and registers it with the root app in `app.py`. CLI functions are thin wrappers that open a `Session` from `aimbat.db.engine` and delegate to `core/` functions. | ||||||
| Each CLI module in `_cli/` creates a Cyclopts `App` instance and registers it with the root app in `app.py`. CLI functions are thin wrappers that open a `Session` from `aimbat.db.engine` and delegate to `core/` functions. | ||||||
|
|
||||||
| ### Custom Types | ||||||
|
|
||||||
| - Use `PydanticTimestamp` / `PydanticTimedelta` (from `aimbat.aimbat_types`) for pandas-compatible time fields in models | ||||||
| - Use `PydanticTimestamp` / `PydanticTimedelta` (from `aimbat._types`) for pandas-compatible time fields in models | ||||||
| - Use `PydanticNegativeTimedelta` / `PydanticPositiveTimedelta` for constrained sign validation | ||||||
| - Use `SAPandasTimestamp` / `SAPandasTimedelta` (from `aimbat.models._sqlalchemy`) as the `sa_type` in SQLModel fields | ||||||
|
||||||
| - Use `SAPandasTimestamp` / `SAPandasTimedelta` (from `aimbat.models._sqlalchemy`) as the `sa_type` in SQLModel fields | |
| - Use `SAPandasTimestamp` / `SAPandasTimedelta` (from `aimbat._types`) as the `sa_type` in SQLModel fields |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,43 @@ | ||
| # Core concepts | ||
|
|
||
| ## Motivation | ||
|
|
||
| Precise phase arrival picks are the foundation of travel time tomography — | ||
| the accuracy of the resulting images of Earth's interior depends directly on | ||
| the quality of these measurements. Obtaining them requires picking the phase | ||
| arrival and assessing data quality for every seismogram, across every event | ||
| in the dataset. With modern seismic arrays recording each earthquake on | ||
| increasingly large numbers of seismometers, doing this seismogram by seismogram | ||
| quickly becomes impractical. | ||
|
|
||
| AIMBAT addresses this by shifting the focus from individual seismograms to the | ||
| dataset as a whole. Rather than assessing and processing each trace in | ||
| isolation, the focus is at the array level — where data quality and phase | ||
| arrivals can be judged in the context of all seismograms at once. Decisions | ||
| about filter settings, time windows, and which seismograms to include apply to | ||
| the entire dataset, and picks are refined across all traces simultaneously. | ||
| Everything is processed in bulk. | ||
|
|
||
| ## Semi-automatic | ||
|
|
||
| This bulk processing happens in a semi-automatic way, whereby initial picks | ||
| surrounded by large time windows are iteratively refined into accurate phase | ||
| arrival picks with narrow time windows. Selecting high quality seismograms and | ||
| updating picks (for all stations simultaneously) are either performed manually, | ||
| or automatically by the ICCS algorithm. The automatically refined picks depend | ||
| on user-adjustable parameters, which are typically tuned between iterations to | ||
| achieve the best results. Once satisfied with the picks and parameter settings, | ||
| MCCC is run to produce the final relative arrival time measurements. | ||
|
|
||
| ## Snapshots and rollback | ||
|
|
||
| The iterative nature of the workflow means exploring different parameter | ||
| combinations is central to the process. This is safe to do because the | ||
| seismogram data themselves are never modified — AIMBAT only stores and updates | ||
| processing parameters separately from the data. | ||
|
|
||
| To support this further, snapshots of the current parameter state can be saved | ||
| at any point during processing — including before any changes are made. | ||
| Rolling back to a snapshot restores the parameters exactly as they were, but | ||
| does not delete any other snapshots, so it is possible to switch freely between | ||
| saved states. |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The comment describes
core/_data.pyas "SAC ingestion entry point" but after this PR it handles generic data ingestion for all data types (SAC, JSON_STATION, JSON_EVENT). Update to something like: "Data ingestion entry point for all data types"