Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 28 additions & 5 deletions include/internal/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -245,14 +245,37 @@ edit strategies symmetric.

Current structural edit policy:

- Row insert/erase: mutate `rows`, `keys_`, and sparse row-overlay slots
directly because rows are the native storage unit.
- Column insert: write the current visible table, including sparse overlay
edits, through the CSV writer and reparse it into fresh row storage. This
bakes visible edits into the rebuilt rows and clears sparse overlays.
- Clean row/cell access: `data_frame::DataFrameStorage` centrally selects a
concrete `CleanDataFrame` or `DirtyDataFrame` through an atomic tagged pointer.
Calls remain direct and inlineable, with no vtable or C++17 variant dependency.
Clean handling owns physical rows/names; dirty handling owns visible names,
sparse overlays, and logical-to-physical mapping. Facade features never inspect
the active state themselves.
- First-edit promotion: construct all dirty state before release publication and
retain the clean backing object for readers that captured the previous backend.
Mutable cells bind to the stable selector. Structural changes and moves are
exclusive; read operations do not acquire a transition lock.
- Read-only column views: `data_frame::RowViewAccessor` reuses `CSVRow`'s decoded,
trimmed field extraction without constructing a scalar-aware `CSVField`.
Both backends use it for parsed values; dirty overlays retain their own lookup.
The measurements and rejected physical-column caching alternative are retained
in [data_frame/PERFORMANCE.md](data_frame/PERFORMANCE.md).
- Row insert/erase: mutate `rows` and `keys_` directly and update the handler's
row bindings and overlay slots because rows are the native storage unit.
- Column insert: delegate validation, schema/key-column remapping, and rebuilding
to `DirtyDataFrame`, including for a clean input frame. Construct fresh shared
backing chunks from visible values,
including sparse edits, without CSV serialization/reparsing. This preserves
empty fields and zero-column rows and returns the frame to clean handling.
Stored keys remain stable row identities even after key-column cell edits.
View keys are retargeted into private owned key-byte storage before their
parsed backing is released; owning-key types require no additional work.
- Column erase: soft-delete the visible column by removing it from the visible
column-name list and logical-to-physical column map. Underlying `CSVRow`
storage is intentionally left unchanged.
- Selection: share immutable parsed rows and independently snapshot editing
state and visible-column mapping. Row proxies resolve current overlays when
creating cells; raw underlying rows intentionally exclude logical edits.

This asymmetry is intentional. The consistent principle is to use the cheapest
reliable operation that preserves visible semantics and keeps ordinary row
Expand Down
4 changes: 0 additions & 4 deletions include/internal/col_names.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -48,10 +48,6 @@ namespace csv {
return this->_policy;
}

CSV_INLINE size_t ColNames::size() const noexcept {
return this->col_names.size();
}

CSV_INLINE const std::string& ColNames::operator[](size_t i) const {
if (i >= this->col_names.size())
throw_column_index_out_of_bounds();
Expand Down
2 changes: 1 addition & 1 deletion include/internal/col_names.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ namespace csv {
csv::ColumnNamePolicy get_policy() const noexcept;

bool empty() const noexcept { return this->col_names.empty(); }
size_t size() const noexcept;
inline size_t size() const noexcept { return this->col_names.size(); }

/** Retrieve column name by index. Throws if index is out of bounds. */
const std::string& operator[](size_t i) const;
Expand Down
3 changes: 3 additions & 0 deletions include/internal/csv_row.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ namespace csv {
struct CSVRowFragment;
}

namespace data_frame { struct RowViewAccessor; }

static const std::string ERROR_NAN = "Not a number.";
static const std::string ERROR_OVERFLOW = "Overflow error.";
static const std::string ERROR_FLOAT_TO_INT =
Expand Down Expand Up @@ -546,6 +548,7 @@ namespace csv {
template<typename RowSink, typename ParsePolicy, typename FieldPolicy, typename RowPolicy>
friend class internals::CSVParserCore;
friend struct internals::CSVRowRowPolicy;
friend struct internals::data_frame::RowViewAccessor;
friend internals::parser::CSVParserDriverBase;
friend struct internals::speculative::CSVRowFragment;

Expand Down
47 changes: 39 additions & 8 deletions include/internal/data_frame/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,14 @@ namespace `csv`; the folder split is for maintainability, not a namespace move.

## Storage Model

`DataFrame` is row-backed. Its primary storage is `std::vector<CSVRow>` plus
per-row sparse edit overlays. Do not turn normal row/cell access into a
`DataFrame` delegates storage operations through private
`internals::data_frame::DataFrameStorage`. Its C++11 tagged pointer selects
concrete `CleanDataFrame` or `DirtyDataFrame` implementations without virtual
dispatch. State checks belong only in this selector, never in facade features.
`CleanDataFrame` owns the native `std::vector<CSVRow>` and physical column names.
`DirtyDataFrame` owns visible names, sparse overlays, and column mapping.
Clean reads access parsed rows directly, without overlay slots or mapping.
Do not turn normal row/cell access into a
columnar abstraction just to make structural edit implementations symmetric.

The guiding rule is:
Expand All @@ -16,11 +22,17 @@ The guiding rule is:

Current structural edit strategy:

- Row insert/erase mutates row storage, keyed metadata, and sparse overlay slots
directly.
- Column insert materializes the current visible table through `CSVWriter`,
reparses it into fresh row storage, and clears sparse overlays because visible
edits are baked into the rebuilt rows.
- Cell assignment activates dirty handling lazily. Taking mutable row/cell
proxies does not allocate overlays or activate the flag.
- Row insert/erase delegates to the active concrete backend. Dirty handling
shifts overlay slots alongside native rows; row and slot capacity is prepared
before committing keyed insertion metadata.
- Column insert builds fresh owned row storage from visible values, without CSV
serialization or reparsing. Shared backing chunks avoid allocation per row
and preserve empty values and zero-column row cardinality. Successful
materialization returns the selector to clean handling. `DirtyDataFrame`
owns schema validation, key-column remapping, and row rebuilding, including
structural insertion into an otherwise clean frame.
- Column erase is a soft delete: visible column names and the
logical-to-physical column map change, while underlying `CSVRow` storage stays
intact.
Expand All @@ -41,13 +53,32 @@ compaction/materialization API over adding hot-path indirection for all access.
- Sparse overlays are keyed by physical column index. Any feature that changes
physical row storage or logical-to-physical mapping must account for existing
overlays.
- Stored keys identify rows independently of key-column cell edits. Structural
materialization must preserve those keys, including custom-function keys.
- `csv::string_view` keys must be copied into private owned key storage before
replacing parsed rows, and the cached key index must be invalidated when views
are retargeted. Owning-key types use a compile-time no-op storage policy.
- Row proxies resolve current editing state when constructing cells; never
capture overlay allocation history as the authority for later row access.
- Selection shares parsed rows but snapshots overlays independently and keeps
the visible column mapping. Never reconstruct a logical frame solely from
`get_underlying_row()`: it omits edits and includes hidden columns.
- First-edit promotion fully constructs the dirty backend and slot array before
release publication. Retain the clean backing object so readers that loaded
the old backend can finish without per-read locking. Bind mutable cells to the
stable selector, not to whichever concrete backend was active at capture.
- Structural operations and moves require exclusive access. Successful column
materialization may retire dirty state; ordinary cell edits retain row locks.
- DataFrame iterators should follow the library's cached-proxy convention:
store the current proxy inside the iterator and expose `operator*` /
`operator->` reference-like access, as `CSVReader` and `CSVRow` do.

## Test Expectations

Put DataFrame behavior tests in `tests/test_data_frame.cpp`. For writer
Put general DataFrame behavior tests in `tests/test_data_frame.cpp`; put
clean/dirty transitions and edit-preservation regressions in
`tests/test_data_frame_dirty.cpp`. Both belong to the `data_frame_test` target.
For writer
compatibility, also check `tests/test_write_csv.cpp` when row-like output or
`to_sv_range()` behavior changes.

Expand Down
Loading
Loading