Question about "include_non_param=False" in FedPart algorithm

[evanchar]

Hello,

First, thank you for the great framework. I would like to ask a question regarding the implementation of FedPart.

By changing the include_non_param argument from its default False to True, I noticed a significant jump in accuracy, with FedAvg performing better than FedPart.

Is there a theoretical or practical reason why this argument defaults to False, thereby excluding the running_mean and running_avg from being broadcast back to the server?

Attaching an image with a simple test on cifar10

Thank you

[kxzxvbk]

Hello,

Thanks for the great question and the detailed testing!

To answer your points:

1. Why it defaults to False: Excluding running variables (like running_mean and running_var) was introduced in FedBN and has gradually become a standard default, specially in heterogeneous federated learning to handle non-IID data.

2. Why FedPart's performance drops: We actually reproduced this exact phenomenon. Our hypothesis is that these statistics are crucial for numerical stability. Generally, it's best to either share everything (like FedAvg) or nothing (like FedBN). Because FedPart fundamentally relies on partial parameter sharing, sharing running statistics on top of that may break the numerical balance, leading to the drop you observed.

Really appreciate you bringing this up. We'd love to hear if you have any further insights or thoughts on this!