Skip to content

Pid tuning experiments#12

Open
DavidAlex17 wants to merge 10 commits into
dmitrykoro:mainfrom
demcclain:pid-tuning-experiments
Open

Pid tuning experiments#12
DavidAlex17 wants to merge 10 commits into
dmitrykoro:mainfrom
demcclain:pid-tuning-experiments

Conversation

@DavidAlex17
Copy link
Copy Markdown

This branch contains all completed work for the federated learning defense analysis:

Experiments Completed:

  • IID distribution: Multi-Krum (k=7,8,9) and PID (α=1.0σ,1.5σ,2.5σ + optimized)
  • NIID distribution: Multi-Krum (k=7,8,9) and PID (α=2.5σ,3.0σ,3.5σ + optimized)

Key Findings:

  • IID: Multi-Krum k=8 (76.06%), PID optimized (75.75%)
  • NIID: PID optimized (28.18%), Multi-Krum k=9 (23.49%) - 4.69pp improvement
  • NIID requires 2× higher thresholds (3.0σ vs 1.5σ) to avoid false positives

Deliverables Ready:

  • All simulation results with ADR/FPR metrics calculated
  • PowerPoint presentation with graphs and analysis
  • Report sections: Defense Methods, Experimental Design, Discussion (LaTeX formatted)

demcclain and others added 10 commits October 31, 2025 04:05
- Keep Multi-Krum k parameter sensitivity: k=7,8,9
- Keep PID threshold sensitivity: 1.0σ, 1.5σ, 2.5σ
- Keep PID optimized combo (early removal + tighter threshold)
- Remove Trust, Trimmed Mean, and other PID variations
- Change preserve_dataset to false for proper cleanup
- Limit to 7 strategies for clear graph visualization
- Create femnist_pid_tuning_niid.json with same 7 strategies as IID
- Change dataset_keyword from femnist_iid to femnist_niid
- Update simulation_runner.py to use NIID config
- Increase thresholds: 1.5σ, 2.0σ, 3.0σ (was 1.0σ, 1.5σ, 2.5σ)
- NIID data has higher variance, needs more conservative thresholds
- Fixes ZeroDivisionError from removing all clients in round 5
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants