-
Notifications
You must be signed in to change notification settings - Fork 16
Home
Some how-to's for code review and assertions (useful when assessing whether code and analyses are likely to be correct) and statistical power analysis (i.e., project 3, smoothly segueing from project 2).
Per Monday's discussion, let's assume we take an "all-or-nothing" approach to review.
-
(no conflicts, merge should succeed without resolution, vignettes build)
-
(conflicts must be resolved before pull request can be merged, vignettes build)
-
another negative control
(no conflicts, but the merge will break the build)
Let's consider just cherry-picking a commit (i.e., one discrete change in the code) which is separated from the rest of a request. Furthermore, we will consider a very strict (non-statistically-minded, i.e. "true or false") notion of hypothesis testing here.
-
Specifically, let's assert that samples are either AML or NBM, and NBM is defined as the marrows that aren't AMLs.
-
Specifically, an assertion of strict mutual exclusivity for IDH and TET2 mutations, where strict means 0 double mutants.
Since we've been testing for the presence or absence of things a lot lately, let's continue in that vein for a bit here.
-
Using various tests, let's ask how many violations we can tolerate at a given sample size and still detect significant mutual exclusivity between certain variants (say, IDH1, IDH2, TET2, and WT1). In this case we will extrapolate from a limited sample.
-
As from a pilot experiment (e.g. scRNA library preparation methods) where we have the ability to downsample or simulate (upsample).
-
...and N will need to be very large (neurogenesis). We can use the data from the study to estimate limits of detection.
This is just the part where we draw pictures. Also known as the most important part of papers, grants, and presentations.
-
fit a mixture model as outlined in my project 2 chunk
We can use this to "gate" cells, plot results in terms of discriminating populations by species, and visualize our prior (above).
-
same thing but discriminate by cell type (relatively common, see also scGate )
This is the sort of task that a great many groups take upon themselves via scRNAseq alone, but inevitably orthogonal validation is requested during the review process. It so happens that we have a convenient example of just such a study.
-
not entirely dissimilar but take advantage of additional dimensions
Consider a balance between "not enough data reduction" and "too much data reduction" in terms of statistical power. Consider how one might demonstrate the ability of untransformed, linear, and nonlinear approaches to separate groups or continua of interest. The 3D version is easy. The higher-dimensional versions are not as easy but will eventually take over in a lot of situations where we use crappy flat 2D spin-art in papers at the moment. Consider the difference between two degrees of freedom (two dimensions), where separating groups can be very difficult, versus perhaps 10 degrees of freedom (10 dimensions, for the sake of argument) where the groups of interest are perfectly separated, versus tens of thousands of degrees of freedom (say, 60000 transcripts, or 3000000 regions of chromatin) that are in fact mostly degenerate (i.e. many are perfectly exchangeable with each other in terms of information). You pay a huge price in sample size and statistical power if you conduct tens of thousands of tests, but if you over-reduce or conduct too few, it's also not going to end well. If you can get comfortable with the notion of dimensions beyond the usual 2-3, it will help in thinking about productive experimental and analytical designs.