Skip to content

fix: keep inbox sampling cohort stable#4

Open
mawxcodehub wants to merge 2 commits into
sonic-mast:mainfrom
mawxcodehub:codex/stable-inbox-sample-cohort
Open

fix: keep inbox sampling cohort stable#4
mawxcodehub wants to merge 2 commits into
sonic-mast:mainfrom
mawxcodehub:codex/stable-inbox-sample-cohort

Conversation

@mawxcodehub

Copy link
Copy Markdown

Summary

  • Select inbox samples from a stable BTC-address cohort instead of API result order.
  • Persist the sampled cohort in data/sample-cohort.json and backfill missing addresses deterministically.
  • Add node:test coverage proving API result reordering does not change the sampled inboxes.

Closes #3

Validation

  • node --test scripts/collect-metrics.test.js
  • node -e 'const m=require("./data/metrics.json"); const drops=[]; for(let i=1;i<m.length;i++) if(m[i].total_messages<m[i-1].total_messages) drops.push(${m[i-1].date} ${m[i-1].total_messages}->${m[i].date} ${m[i].total_messages}); console.log(JSON.stringify({drops:drops.length, examples:drops.slice(-5)}, null, 2))'
  • git diff --check

Signed-off-by: MAWXHUB <130845891+mawxcodehub@users.noreply.github.com>

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a stable sampling mechanism for agent inboxes in scripts/collect-metrics.js by reading and writing a cohort of addresses to a local JSON file, and adds unit tests to verify this behavior. Feedback suggests defining the hardcoded sample size as a constant for better maintainability, and wrapping the JSON parsing logic in a try-catch block to handle potentially corrupted cohort files gracefully.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.


const AIBTC_API = 'https://aibtc.com/api';
const DATA_DIR = path.join(__dirname, '..', 'data');
const COHORT_FILE = path.join(DATA_DIR, 'sample-cohort.json');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To improve maintainability, it's a good practice to define the sample size as a constant here, instead of hardcoding the value 20 in the main function. You can then use this constant in sampleInboxMetrics and writeSampleCohort calls.

Suggested change
const COHORT_FILE = path.join(DATA_DIR, 'sample-cohort.json');
const COHORT_FILE = path.join(DATA_DIR, 'sample-cohort.json');
const SAMPLE_SIZE = 20;

Comment thread scripts/collect-metrics.js Outdated
Comment on lines +72 to +73
const cohort = JSON.parse(fs.readFileSync(COHORT_FILE, 'utf8'));
return Array.isArray(cohort.btcAddresses) ? cohort.btcAddresses : [];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The readSampleCohort function could be more robust. If sample-cohort.json exists but is empty or contains invalid JSON, JSON.parse will throw an error, causing the script to exit. It would be better to handle this case gracefully by catching the error and returning an empty array. This would make the script self-healing in case of a corrupted cohort file, allowing it to generate a new one on the next run.

  try {
    const fileContent = fs.readFileSync(COHORT_FILE, 'utf8');
    if (!fileContent) {
      return [];
    }
    const cohort = JSON.parse(fileContent);
    return Array.isArray(cohort.btcAddresses) ? cohort.btcAddresses : [];
  } catch (error) {
    console.error(`Error reading or parsing ${COHORT_FILE}: ${error.message}`);
    return [];
  }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cumulative total_messages can decrease when the sampled agent cohort changes

2 participants