Disambiguate names

[briatte]

Problem

(All examples from EPSA 2029 names.)

Variations in family names:

Ahmed Mohamed
Ahmed Mohammed

Shortened first names:

Andrew Guess
Andy Guess

Ambiguities:

Aina Gallego
Anna Gallego
Alina Vrânceanu
Alina Vranceau

Middle initials:

Andreas C. Goldberg
Andreas Goldberg

Accents and special characters:

Aysenur Dal
Ayşenur Dal

More complex case:

"Ferran Martinez i Coma"
"Ferran M i Coma"

… all cause some authors to have multiple first names (and thus multiple participant UIDs).

Partial solution

Use stringdist to detect simple cases:

https://cran.r-project.org/web/packages/stringdist/vignettes/RJournal_6_111-122-2014.pdf

n <- unique(d$full_name)

# Levenshtein distances
lv <- tibble::tibble(
  x = n,
  y = map(x, ~ n[ !n %in% .x ]),
  lv = map(x, ~ stringdist(.x, n[ !n %in% .x ], method = "lv"))
) %>%
  tidyr::unnest(c(y, lv))

# finds many true positives
filter(lv, lv < 3) %>%
  arrange(x, lv)

# finds "Alex Smith" and "Alex C. Smith"
filter(lv, lv == 3) %>%
  arrange(x, lv)

# finds mostly false positives
filter(lv, lv == 4) %>%
  arrange(x, lv)

Question to self

Fix names here, once all names are assembled, or in the source 2019, 2020, 2021 repos?

Cleanup in this repo makes more sense because it allows to treat all names at once, which avoids treating recurring participants several times.

If cleanup happens here, UIDs need to be regenerated here, after applying the fixes (not a huge hassle).

[briatte]

• Since names can be fixed with the help of affiliations, it might make more sense to do it year/repo-wise.
• Since fixing names using affiliations will reveal some possible affiliation fixes, that might also be done in parallel, also year/repo-wise.

… or perhaps collate all those things together and do a mega-fix session in this repo?

[briatte]

• Export all possible matches x, y, with affiliations
• Import in Google Sheets
• Add a correct_name column
  • If empty, not a correct match
  • If "x", then x is the right name
  • If "y", then y is the right name
• Add a correct_affiliation column
  • Repeat same coding as correct_name above
• Check that Google Sheet never corrects a single name/affiliation more than once!

Tough job, resembles what has been done with great pain on AFSP…

[briatte]

All of this is time-dependent (affiliations can change; even names, in fact, but let's not go there…) — it definitely makes sense to keep time/year somewhere in the process.