Skip to content

Fixed error in KDE calculation - #9

Open
MaroGol wants to merge 1 commit into
FLASK-LLNL:mainfrom
MaroGol:kde_fix
Open

Fixed error in KDE calculation#9
MaroGol wants to merge 1 commit into
FLASK-LLNL:mainfrom
MaroGol:kde_fix

Conversation

@MaroGol

@MaroGol MaroGol commented Mar 21, 2026

Copy link
Copy Markdown

In sklearn's implementation of KDE, under site-packages/sklearn/neighbors/_kde.py, we see the following:

if isinstance(self.bandwidth, str):
    if self.bandwidth == "scott":
        self.bandwidth_ = X.shape[0] ** (-1 / (X.shape[1] + 4))
    elif self.bandwidth == "silverman":
        self.bandwidth_ = (X.shape[0] * (X.shape[1] + 2) / 4) ** (
            -1 / (X.shape[1] + 4)
        )

When given a bandwidth of "scott", it calculates:

X.shape[0] ** (-1 / (X.shape[1] + 4))

Which in our case is n ** (-1/5).
But the standard Scott's rule formula is $\sigma \cdot n^{-\frac{1}{5}}$.
So sklearn is assuming $\sigma=1$, in other words, that the data has already been standardized, which isn't the case.

As a result, the KDE estimate overfits or underfits the data, which then results in erroneous ID_OOD splits down the line.

A fix could be to standardize the data before calculating the KDE, or to just pass the Scott's formula bandwidth directly, which is what I opted to do in this fix.

Let me know if you think I missed anything :)

…earn's implementation assumed standardized data
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant