Skip to content

compare_cor $statistic not z-score #115

Description

@Lakens

compare_cor() reports unstandardized numerator as z-statistic

Package: TOSTER
Function: compare_cor() with method = "fisher" and alternative = "equivalence"


Description

When running an equivalence test via compare_cor(), the reported z statistic
in the output is not the standardized test statistic. Instead, it is the raw
numerator (diff - bound) before dividing by the standard error. The p-value
is computed correctly (it does use zlo/z_se internally), but the labelled
statistic is misleading and inconsistent with what a z-statistic conventionally means.


Expected behavior

The reported z should be the standardized test statistic:

z = (diff - bound) / SE

A reader using the reported z to verify or recompute the p-value via pnorm()
should get the same p-value shown in the output.


Actual behavior

The reported z is the unstandardized numerator zlo = diff - min(znull).
Looking at the source:

zlo = diff - min(znull)
plo = p_from_z(zlo/z_se, alternative = "greater")  # p-value uses zlo/z_se
...
z = zlo   # but reported z stores only zlo, not zlo/z_se

Reproducible example

library(TOSTER)

result <- compare_cor(
  r1 = 0.6, df1 = 18,
  r2 = 0.8, df2 = 23,
  null = 0.4,
  method = "fisher",
  alternative = "equivalence"
)

result$statistic
#> z = 0.018184   ← this is diff - bound, not a z-statistic

# Manually computing the correct z-statistic:
z1   <- atanh(0.6); z2 <- atanh(0.8)
diff <- z1 - z2                              # -0.4055
SE   <- sqrt(1/17 + 1/22)                   #  0.3229
bound_z <- atanh(0.4)                        #  0.4236 (TOSTER converts null via atanh)

(diff - (-bound_z)) / SE                     # 0.0562 ← actual z-statistic

# Verify p-value matches the reported one:
1 - pnorm((diff - (-bound_z)) / SE)          # 0.4775 ✓
1 - pnorm(result$statistic)                  # 0.4928 ✗  (wrong if you use reported z)

Additional context

  • The p-value itself is correct — this is purely a reporting/labelling issue.
  • It becomes practically misleading if a user tries to verify the p-value from
    the reported z, or reports the z-statistic in a paper, as it has no standard
    interpretation (it is in the units of Fisher-z differences, not a
    standardized normal deviate).
  • Separately: null is passed through rho_to_z() internally, so null = 0.4
    is treated as r = 0.4 → z = 0.4236. This may also be worth documenting more
    explicitly, as users setting bounds "in Fisher-z units" will get slightly
    different bounds than intended.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions