Skip to content

Perf: optimize Tablet write with columnar string storage and lazy DeviceID construction (~10x throughput)#748

Open
ColinLeeo wants to merge 4 commits intodevelopfrom
tablet_refine
Open

Perf: optimize Tablet write with columnar string storage and lazy DeviceID construction (~10x throughput)#748
ColinLeeo wants to merge 4 commits intodevelopfrom
tablet_refine

Conversation

@ColinLeeo
Copy link
Contributor

@ColinLeeo ColinLeeo commented Mar 19, 2026

Write 200 x 50000 rows: 15039 ms
Throughput: 664938 rows/s

截屏2026-03-20 00 47 02

Write 200 x 50000 rows: 1578 ms
Throughput: 6337140 rows/s

截屏2026-03-20 00 51 04 [collapsed.zip](https://github.com/user-attachments/files/26122101/collapsed.zip)

@codecov-commenter
Copy link

codecov-commenter commented Mar 19, 2026

Codecov Report

❌ Patch coverage is 78.65169% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.87%. Comparing base (ce16cb2) to head (dbf86df).

Files with missing lines Patch % Lines
cpp/src/writer/tsfile_writer.cc 57.69% 8 Missing and 3 partials ⚠️
cpp/src/common/tablet.h 79.31% 2 Missing and 4 partials ⚠️
cpp/src/common/tablet.cc 94.11% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #748      +/-   ##
===========================================
+ Coverage    61.85%   61.87%   +0.02%     
===========================================
  Files          704      704              
  Lines        41276    41347      +71     
  Branches      5929     5948      +19     
===========================================
+ Hits         25531    25585      +54     
- Misses       14905    14915      +10     
- Partials       840      847       +7     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ColinLeeo ColinLeeo changed the title refine tabet writing. Perf: optimize Tablet write with columnar string storage and lazy DeviceID construction (~10x throughput) Mar 20, 2026
@hongzhi-gao
Copy link
Contributor

LGTM

@ColinLeeo ColinLeeo requested a review from jt2594838 March 24, 2026 09:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants