Hello,
I want to check whether I can run qpgraph in parallel, and I find that SnowParam with MPI backend is used in the source code.
I try the sample data generated in vignette and run the qpgraph with and without parallel computing, but the results are different when I set the cluster to multiple workers. I then try to set it to 1 worker, and the results are the same.
library(GenomeInfoDb)
library(qtl)
library(qpgraph)
###################################################
### code chunk number 4: eQTLnetworks.Rnw:79-84
###################################################
map <- sim.map(len=rep(100, times=9),
n.mar=rep(10, times=9),
anchor.tel=FALSE,
eq.spacing=TRUE,
include.x=FALSE)
###################################################
### code chunk number 5: eQTLnetworks.Rnw:96-99
###################################################
set.seed(12345)
sim.eqtl <- reQTLcross(eQTLcrossParam(map=map, genes=50, cis=0.5, trans=rep(5, 5)),
a=2, rho=0.5)
###################################################
### code chunk number 6: simeqtlnet
###################################################
#plot(sim.eqtl, main="Simulated eQTL network")
###################################################
### code chunk number 7: eQTLnetworks.Rnw:106-109
###################################################
set.seed(12345)
cross <- sim.cross(map, sim.eqtl, n.ind=100)
#cross
###################################################
### code chunk number 8: eQTLnetworks.Rnw:128-134
###################################################
annot <- data.frame(chr=as.character(sim.eqtl$genes[, "chr"]),
start=sim.eqtl$genes[, "location"],
end=sim.eqtl$genes[, "location"],
strand=rep("+", nrow(sim.eqtl$genes)),
row.names=rownames(sim.eqtl$genes),
stringsAsFactors=FALSE)
###################################################
### code chunk number 9: eQTLnetworks.Rnw:141-147
###################################################
pMap <- lapply(map, function(x) x * 5)
class(pMap) <- "map"
annot$start <- floor(annot$start * 5)
annot$end <- floor(annot$end * 5)
genome <- Seqinfo(seqnames=names(map), seqlengths=rep(100 * 5, nchr(pMap)),
NA, "simulatedGenome")
###################################################
### code chunk number 10: eQTLnetworks.Rnw:153-155
###################################################
param <- eQTLnetworkEstimationParam(cross, physicalMap=pMap,
geneAnnotation=annot, genome=genome)
###################################################
### code chunk number 11: eQTLnetworks.Rnw:159-161
###################################################
eqtlnet.q0 <- eQTLnetworkEstimate(param, ~ marker + gene, verbose=FALSE)Calculating pairwise (conditional) independence tests
library(Rmpi)
library(BiocParallel)
snowparam <- bpstart(SnowParam(3, type = "MPI"))
3 slaves are spawned successfully. 0 failed.
#snowparam <- bpstart(SnowParam(1, type = "MPI"))
eqtlnet.q0.1 <- eQTLnetworkEstimate(param, ~ marker + gene,
verbose=FALSE, BPPARAM = snowparam)Calculating pairwise (conditional) independence tests
Testing conditional independences using a MPI cluster of 3 nodes
bpstop(snowparam)
identical(eqtlnet.q0,eqtlnet.q0.1)
table(is.na(eqtlnet.q0.1@pvaluesG0@x))
table(is.na(eqtlnet.q0@pvaluesG0@x))
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 22621)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8
time zone: America/New_York
tzcode source: internal
attached base packages:
[1] stats4 stats graphics grDevices utils datasets methods base
other attached packages:
[1] BiocParallel_1.38.0 Rmpi_0.7-2.1 qpgraph_2.38.0 qtl_1.70
[5] GenomeInfoDb_1.40.1 IRanges_2.38.1 S4Vectors_0.42.1 BiocGenerics_0.50.0
loaded via a namespace (and not attached):
[1] SparseArray_1.4.8 bitops_1.0-9 RSQLite_2.3.7
[4] lattice_0.22-6 grid_4.4.1 mvtnorm_1.3-1
[7] fastmap_1.2.0 blob_1.2.4 jsonlite_1.8.9
[10] Matrix_1.7-0 AnnotationDbi_1.66.0 restfulr_0.0.15
[13] graph_1.82.0 DBI_1.2.3 httr_1.4.5
[16] UCSC.utils_1.0.0 XML_3.99-0.17 Rgraphviz_2.48.0
[19] Biostrings_2.72.1 codetools_0.2-20 abind_1.4-8
[22] cli_3.6.3 rlang_1.1.4 crayon_1.5.3
[25] XVector_0.44.0 Biobase_2.64.0 bit64_4.5.2
[28] DelayedArray_0.30.1 cachem_1.1.0 yaml_2.3.10
[31] GenomicFeatures_1.56.0 S4Arrays_1.4.1 tools_4.4.1
[34] parallel_4.4.1 memoise_2.0.1 annotate_1.82.0
[37] GenomeInfoDbData_1.2.12 Rsamtools_2.20.0 SummarizedExperiment_1.34.0
[40] curl_5.2.3 vctrs_0.6.5 R6_2.5.1
[43] png_0.1-8 matrixStats_1.4.1 BiocIO_1.14.0
[46] rtracklayer_1.64.0 zlibbioc_1.50.0 KEGGREST_1.44.1
[49] bit_4.5.0 GenomicRanges_1.56.2 GenomicAlignments_1.40.0
[52] MatrixGenerics_1.16.0 rstudioapi_0.17.1 xtable_1.8-4
[55] rjson_0.2.23 snow_0.4-4 compiler_4.4.1
[58] RCurl_1.98-1.16
I'm using a windows PC, and I follow the instruction from https://fisher.stats.uwo.ca/faculty/yu/Rmpi/ to use Microsoft MPI, the output of mpiexec from shell is:
PS C:\Users\12696> mpiexec
Microsoft MPI Startup Program [Version 10.1.12498.52]
Launches an application on multiple hosts.
Usage:
mpiexec [options] executable [args] [ : [options] exe [args] : ... ]
mpiexec -configfile <file name>
Common options:
...
...
Hello,
I want to check whether I can run qpgraph in parallel, and I find that SnowParam with MPI backend is used in the source code.
I try the sample data generated in vignette and run the qpgraph with and without parallel computing, but the results are different when I set the cluster to multiple workers. I then try to set it to 1 worker, and the results are the same.
I'm using a windows PC, and I follow the instruction from https://fisher.stats.uwo.ca/faculty/yu/Rmpi/ to use Microsoft MPI, the output of
mpiexecfrom shell is: