Weight distributed attribution aggregation by samples (#1925) - #1925
Open
craymichael wants to merge 6 commits into
Open
Weight distributed attribution aggregation by samples (#1925)#1925craymichael wants to merge 6 commits into
craymichael wants to merge 6 commits into
Conversation
Contributor
|
@craymichael has exported this pull request. If you are a Meta employee, you can view the originating Diff in D117601324. |
Summary: Summary Feature Ablation and Shapley formatted baselines but did not validate tuple arity or tensor shape before running the model. `_tensorize_baseline` also paired inputs and baselines with `zip`, silently discarding extra baselines. Problem For two inputs, a one-element baseline tuple reached the model with a missing argument, while a three-element tuple silently ignored its final baseline. A baseline pool with shape `[2, 1]` for a three-row input was neither a per-example baseline nor a singleton baseline and failed later through opaque broadcasting errors. Fix * Validate baselines in both synchronous and future Feature Ablation and Shapley entry points. * Defensively reject arity mismatches in `_tensorize_baseline`. * Require Shapley tensor baselines to match the input or use a singleton leading dimension with matching trailing dimensions. * Preserve Feature Ablation’s documented support for any tensor shape that broadcasts exactly to the input, including `[F]` and 0-D tensors. Test Plan Before: the new regressions failed through late `IndexError`, missing-forward-argument, and broadcast errors; the extra-baseline case did not raise at all. After: * `buck test fbcode//pytorch/captum/tests/attr:test_common fbcode//pytorch/captum/tests/attr:test_feature_ablation fbcode//pytorch/captum/tests/attr:test_shapley` — Pass 131, Fail 0. * `arc lint -a` on changed Python files — no source lint issues; focused autodeps updates applied. * `arc lint -a --engine extra --take CITRINEAGENT` on changed Captum implementation files — no issues. * `arc pyre check-owning-targets` on changed files — no type errors. Differential Revision: D117601314
Summary: Summary Perturbation paths combined original and replacement values as `old * (1 - mask) + new * mask`. IEEE arithmetic makes `NaN * 0` and `Inf * 0` non-finite, so values outside the selected feature could corrupt model inputs and attributions. Counterexample For input `[1, 2]`, baseline `[0, NaN]`, and mask `[0, 1]`, ablating feature 0 should evaluate `[0, 2]`. Arithmetic masking instead produced `[0, NaN]`, so feature 0 incorrectly received a `NaN` attribution. Fix * Use `torch.where` for Feature Ablation, Feature Permutation, Shapley feature updates, and attribution-mask accumulation. * Apply the same selection semantics to within-group baseline construction, add-back, and permutation helpers. * Mirror the fix in the legacy UFO implementations so alternate call paths cannot retain the corruption. * Keep masks and donor indices on the destination tensor device. Test Plan Before: six focused regressions failed across core and within-group paths (`Pass 159, Fail 6`). After: * `buck test fbcode//pytorch/captum/tests/attr:test_feature_ablation fbcode//pytorch/captum/tests/attr:test_feature_permutation fbcode//pytorch/captum/tests/attr:test_shapley fbcode//pytorch/captum/tests/attr/fb:test_within_groups_utils fbcode//pytorch/captum/tests/attr/fb:test_shapley_value_permutation` — Pass 178, Fail 0. * Regressions cover `NaN`, `+Inf`, and `-Inf` in selected and inactive positions. * `arc lint -a` on all changed Python files — no new lint issues; existing legacy UFO line-length advice remains unchanged. * `arc lint -a --engine extra --take CITRINEAGENT` on implementation files — no issues. * `arc pyre check-owning-targets` on changed files — no type errors. Differential Revision: D117601315
…1922) Summary: Summary Perturbation methods accepted feature-mask tuples with the wrong arity. Feature Ablation could then return zero attribution for an omitted input, while Shapley failed later or enumerated an invalid feature space. Shapley also accepted fractional, negative, non-finite, and complex group IDs even though its permutation loop requires non-negative integer IDs. Counterexample With two model inputs and a one-element feature-mask tuple, Feature Ablation perturbed only the first input and silently left the second input’s attribution at zero. With Shapley mask `[0, 0.5]`, the integer feature loop omitted group `0.5`. Fix * Require explicit feature-mask tuples to have one tensor per input in the shared formatter. * Validate Shapley group IDs before feature enumeration. * Accept integral-valued float masks and bool masks for compatibility. * Keep value-domain validation out of Feature Ablation/Permutation so Greedy Feature Selection’s internal `-1` sentinel and existing float-typed masks continue to work. Test Plan Before: four focused regression methods failed (`Pass 132, Fail 4`), including silent acceptance of short and long mask tuples and invalid Shapley IDs. After: * Broad perturbation and wrapper suite — Pass 422, Fail 0, covering Feature Ablation, Feature Permutation, Shapley, DataLoaderAttribution, internal wrappers, WithinGroupSVS, SVS-P, AddOneBack, MarginalWithinGroups, Greedy Feature Selection, and shared mask utilities. * `arc lint -a` on changed files — no issues. * `arc lint -a --engine extra --take CITRINEAGENT` on implementation files — no issues. * `arc pyre check-owning-targets` on changed files — no type errors. Differential Revision: D117601317
…#1924) Summary: Summary `_tensorize_baseline` used `torch.full_like(input, baseline)`, which silently cast a floating-point scalar baseline to the input dtype. Integer inputs therefore lost fractional baseline values before Shapley attribution. Counterexample For integer input `[1, 2]` and scalar baseline `0.5`, additive Shapley attribution should be `[0.5, 1.5]`. The old tensorization converted the baseline to `[0, 0]` and returned `[1, 2]`. Fix Use PyTorch result-type promotion when materializing scalar baselines, while retaining `full_like` so device, layout, and memory format are preserved. Boolean inputs keep their historical boolean baseline dtype. Test Plan Before: the new sync regression failed with both elements off by `0.5`. After: * `buck test` across Shapley, WithinGroupSVS, DeepLift, LayerDeepLift, LayerIntegratedGradients, IntegratedGradients, and LayerConductance consumers — Pass 196, Fail 0. * Focused common/Shapley/WithinGroupSVS rerun — Pass 97, Fail 0. * Unit coverage verifies fractional integer promotion, boolean dtype preservation, and channels-last layout preservation. * `arc lint -a` — no issues. * `arc lint -a --engine extra --take CITRINEAGENT` — no issues. * `arc pyre check-owning-targets` — no type errors. Differential Revision: D117601321
Summary: Problem - Standard permutation may select a row as its own donor, violating explicitly requested leave-one-out semantics. - The initial implementation changed scripted defaults, duplicated retry logic, and leaked arbitrary config fields. - Rank Suggest then hard-coded --n-samples to SVS-P, regressing plain SVS. Fix - Preserve standard defaults and expose public exclude_self_donors options. - Generate one device-side random cycle and share donor ordering across grouped/nested tensors. - Route only supported kwargs with typed validation. - Determine --n-samples support from the selected args dataclass fields, preserving plain SVS and Kernel SHAP while rejecting shuffling. Differential Revision: D117601323
) Summary: - Weight rank/batch attribution means and coverage by their actual example counts. - Infer the local count from supervision tensors, including uneven final or filtered batches, with the configured size only as a fallback. - Gather counts alongside results and skip zero-sample ranks without skipping collectives. - Scope every barrier, gather, and reduction to the supplied process group and translate subgroup rank 0 to its global destination rank. - Extend Captum online mean, variance, standard deviation, and sum statistics with backward-compatible frequency weights. Counterexample: Rank 0 reports mean attribution 1 from one example. Rank 1 reports mean attribution 3 from three examples. Equal rank weighting returns 2.0; the correct example-weighted global mean is `(1 * 1 + 3 * 3) / 4 = 2.5`. Implementation: The evaluator derives each local batch count from label/weight tensors. The publisher gathers those counts in the same process group as attribution tensors, and the singleton summarizer applies them as frequency weights. Existing unweighted callers retain weight 1 and custom Stat implementations still receive their legacy one-argument `update` call. Differential Revision: D117601324
craymichael
added a commit
to craymichael/captum
that referenced
this pull request
Aug 28, 2026
) Summary: - Weight rank/batch attribution means and coverage by their actual example counts. - Infer the local count from supervision tensors, including uneven final or filtered batches, with the configured size only as a fallback. - Gather counts alongside results and skip zero-sample ranks without skipping collectives. - Scope every barrier, gather, and reduction to the supplied process group and translate subgroup rank 0 to its global destination rank. - Extend Captum online mean, variance, standard deviation, and sum statistics with backward-compatible frequency weights. Counterexample: Rank 0 reports mean attribution 1 from one example. Rank 1 reports mean attribution 3 from three examples. Equal rank weighting returns 2.0; the correct example-weighted global mean is `(1 * 1 + 3 * 3) / 4 = 2.5`. Implementation: The evaluator derives each local batch count from label/weight tensors. The publisher gathers those counts in the same process group as attribution tensors, and the singleton summarizer applies them as frequency weights. Existing unweighted callers retain weight 1 and custom Stat implementations still receive their legacy one-argument `update` call. Differential Revision: D117601324
meta-codesync
Bot
changed the title
craymichael
force-pushed
the
export-D117601324
branch
from
August 28, 2026 17:48
f25a50a to
4a350e9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary:
Counterexample:
Rank 0 reports mean attribution 1 from one example. Rank 1 reports mean attribution 3 from three examples. Equal rank weighting returns 2.0; the correct example-weighted global mean is
(1 * 1 + 3 * 3) / 4 = 2.5.Implementation:
The evaluator derives each local batch count from label/weight tensors. The publisher gathers those counts in the same process group as attribution tensors, and the singleton summarizer applies them as frequency weights. Existing unweighted callers retain weight 1 and custom Stat implementations still receive their legacy one-argument
updatecall.Differential Revision: D117601324