False Cells

[DarioS]

What are False cells? 731 of them in this data set.

> table(result$azimuth_medium)
 
              Amacrine cell                      B cell                   Cone cell       Cone-OFF-bipolar cell 
                       4053                           1                        2708                       10125 
       Cone-ON-bipolar cell            Endothelial cell      Epithelial cell of eye                       False 
                       3819                         202                           7                         731 
                 Fibroblast               Ganglion cell                  Glial cell             Horizontal cell 
                          3                       20898                       20916                        2833 
                 Macrophage                    Monocyte                     NK cell        PCP4L1+ bipolar cell 
                        632                           3                           3                        8294 
                   Pericyte                    Rod cell            Rod-bipolar cell                      T cell 
                        165                        6292                        3388                           7

[pedriniedoardo]

Hello, I was wondering the same. Here is my interpretation from empirical evidence using the tutorial.

I was checking the tutorial, and all the azimuth_* False annotations also match the full_consistent_hierarchy covariate.

Always from the tutorial

We also postprocess our predictions to provide labels at three consistent levels of granularity for easy handling, marking any cell with an invalid full hierarchical label (based on `full_consistent_hierarchy`) as `False`.
These categories provide a consistent level of granularity for each cell, but may differ from the `final_level_label`, either by forcing the model to predict further along the cell type hierarchy than its intial prediction, or by rolling back its prediction to a lower level of granularity.

So final_level_labels is the final classification before the postprocessing that defines the congruency of the annotation. Therefore it means that those cells, along the stepwise annotation/classification process, had an incongruent annotation.
I cannot find any recommendation on how to handle those though. In the tutorial for example they suggeste removing the ones that have lower than 0.5 in final_level_confidence, and it is also true that the distribution of this consistency score is strongly associated with the full_consistent_hierarchy (might just be this dataset though)

md %>%
  ggplot(aes(x=full_consistent_hierarchy,y=final_level_confidence)) + geom_boxplot()

Maybe something to ask the maintainer for clarification on how to handle the situation.

[pedriniedoardo]

I have further explored using the tutorial dataset (bmcite).
While I understand that all the cells that have an incongruent hierarchical annotation have the FALSE flag in full_consistent_hierarchy (for example cells where level_5_labels = "CD8 T cell" but level_6_labels = "Naive CD4 T cell", which is not congruent, are all FALSE).

bmcite_ann <- CloudAzimuth(bmcite)

md <- bmcite_ann@meta.data %>%
  rownames_to_column("barcodes") %>%
  tibble()

md %>%
  filter(full_hierarchical_labels == "Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell") %>%
  select(barcodes,full_hierarchical_labels,full_consistent_hierarchy,contains("azimuth_")) %>%
  split(f = .$full_consistent_hierarchy)

$`FALSE`
# A tibble: 22 × 7
   barcodes             full_hierarchical_labels                                               full_consistent_hierarchy azimuth_broad azimuth_medium azimuth_fine azimuth_label   
   <chr>                <chr>                                                                  <lgl>                     <chr>         <chr>          <chr>        <chr>           
 1 a_ACACTGATCTGACCTC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 2 a_ACCGTAACATACAGCT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 3 a_ACTATCTTCTTGAGAC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 4 a_AGAGCTTGTCTTCGTC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 5 a_AGTCTTTGTCTCTCTG-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 6 a_AGTGTCAAGCGTTCCG-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 7 a_CCACCTACACCTCGGA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 8 a_CCTAGCTTCATTCACT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 9 a_CGTTAGAGTCTTCGTC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
10 a_GACACGCGTGATGTCT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD8 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
# ℹ 12 more rows
# ℹ Use `print(n = ...)` to see more rows

I do not understand why some cells are still flagged as FALSE in full_consistent_hierarchy even if the hierarchy seems ok? (for example the path "Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell" seems like a congreunt hierarchical annotation, but some cells are TRUE and some are FASLE).

md %>%
  filter(full_hierarchical_labels == "Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell") %>%
  select(barcodes,full_hierarchical_labels,full_consistent_hierarchy,contains("azimuth_")) %>%
  split(f = .$full_consistent_hierarchy)

$`FALSE`
# A tibble: 61 × 7
   barcodes             full_hierarchical_labels                                               full_consistent_hierarchy azimuth_broad azimuth_medium azimuth_fine azimuth_label   
   <chr>                <chr>                                                                  <lgl>                     <chr>         <chr>          <chr>        <chr>           
 1 a_AAAGATGCATAGAAAC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 2 a_AAATGCCCACCAGGCT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 3 a_AACCGCGCACCGAAAG-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 4 a_ACCGTAAAGAGATGAG-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 5 a_ACTTGTTAGGCTCTTA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 6 a_AGATTGCAGCTAACAA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 7 a_AGCGGTCGTCGGGTCT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 8 a_AGCTCTCTCGCGTAGC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
 9 a_ATCATGGGTAAGAGGA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
10 a_CACACTCCACGAAAGC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell FALSE                     False         False          False        Naive CD4 T cell
# ℹ 51 more rows
# ℹ Use `print(n = ...)` to see more rows

$`TRUE`
# A tibble: 7,461 × 7
   barcodes             full_hierarchical_labels                                               full_consistent_hierarchy azimuth_broad azimuth_medium azimuth_fine     azimuth_label   
   <chr>                <chr>                                                                  <lgl>                     <chr>         <chr>          <chr>            <chr>           
 1 a_AAACCTGAGGTGGGTT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 2 a_AAACCTGAGTACATGA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 3 a_AAACCTGCAGGCAGTA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 4 a_AAACCTGCAGTGACAG-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 5 a_AAACCTGGTAGTACCT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 6 a_AAACCTGGTCTAGGTT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 7 a_AAACCTGTCGAATGCT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 8 a_AAACCTGTCGCATGAT-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
 9 a_AAACCTGTCGCTTAGA-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell
10 a_AAACCTGTCTCCAACC-1 Immune cell|Lymphoid cell|T/NK cell|T cell|CD4 T cell|Naive CD4 T cell TRUE                      Immune cell   T cell         Naive CD4 T cell Naive CD4 T cell

Can anyone explain what I am missing?

[DarioS]

Oh, dear. Azimuth thinks that a red blood cell is a type of immune cell. Red Blood Cells vs. White Blood Cells.

                 Label
Broad             Lymphoid cell Plasma cell Red blood cell
  Epithelial cell             0           0              0
  False                     196         156              4
  Immune cell                 0        1567              1
  Unassigned                  0           0              0

[pedriniedoardo]

@DarioS, where does the Label variable come from? I assume Broad is the azimuth_broad level annotation!?

[DarioS]

azimuth_label the final column of the data frame. Broad is azimuth_broad.

[pedriniedoardo]

I see, yes, that seems dodgy indeed