File formats

Ymap generates many files during its operation. This page documents the format of each.

Reference genome

figure_description.txt

Contains information about the chromosomes of the genome. In particular, contains a mapping between the chromosome names and their indices in Ymap scripts.

Dataset genome

Common_CNV.mat

CNVplot2 - a one-dimensional cell of matrices. The cell indices are chromosome numbers. The cell values are one-dimensional matrices. Each matrix is a list of estimated CNVs per bin.

SNP_v4.mat

Created by LOH_hapmap_v4.m.

chr_SNPdata - a two dimensional cell. The row indices are chromosomes. There are four columns:

1. Phased data - allelic ratios correlated with a hapmap (if given).
2. Unphased data - allelic ratios not correlated with a hapmap.
3. Phased coordinates - the SNP coordinates correlated with a hapmap (if given).
4. Unphased coordinates - the SNP coordinates not correlated with a hapmap.

The 'data' columns contain a cell of bins. Each bin contains a matrix of allelic ratios. Each ratio correlates to the corresponding base pair position in the 'coordinate' columns, which contain a cell of bins, where each bin contains a matrix of base pair positions.

So, for example, chr_SNPdata{3,2}{10}(5) is the 5-th unphased allelic ratio in the 10-th bin of the 3-rd chromosome. The base pair position of this SNP is stored in chr_SNPdata{3,4}{10}(5).

dataType.txt

Created by: project.create_server.php
Source: user input
Description: part of the dataset description - dataset type (whole genome, ddRAD, etc.) and the read type (single or paired).

Format: "dataset_type:read_type"

Dataset types (defined in project.create_window.php):

• 0 - SNP CGH Array
• 1 - Whole-genome sequencing
• 2 - ddRADseq
• 3 - RNAseq
• 4 - IonExpress seq

Read types (defined in project.create_window.php):

• 0 - single end
• 1 - paired end
• 2 - BAM/SAM
• 3 - TXT file

dataBiases.txt

The file consists of N lines, a boolean ('True' or 'False') on each line. Biases may be different for different sequencing technologies.

ddRADseq

1. Length bias.
2. GC bias.
3. Repetitiveness bias.
4. Chromosome end bias.

putative_SNPs_v4.txt

Created by: [scripts_seqModules/]counts_SNPs_v5.py.
Source: samtools pileup as input.
Description: the list of all positions where more than two bases were seen (even if only once).

Column format:

1. Chromosome name
2. Position in chromosome
3. Reference base
4. A reads
5. T reads
6. G reads
7. C reads

SNP_CNV_v1.txt

Created by: counts_CNVs-SNPs_v1.py.
Source: samtools pileup.
Description: read data per each position.

Column format:

1. Chromosome name
2. Position
3. Total reads in position (=read depth)
4. Reference base
5. A reads
6. T reads
7. G reads
8. C reads

SNPdata_parent.txt

Created by: hapmap.preprocess_parent.py in LOH mode.
Source: putative_SNPs_v4.txt from the parent project (self if no other parent specified).
Description: a list of "called" SNPs - those which are within the 25%-75% range, and those which are most frequent (so if there is a 31%/30%/29%/10% distribution, only the 31% and 30% will be used).

Column format:

1. Chromosome name
2. Position
3. Allele 1
4. Allele 2

(Alleles 1 and 2 are sorted alphabetically.)

preprocessed_SNPs.txt

Created by: dataset_process_for_SNP_analysis[.WGseq].py in LOH mode.
Source: SNP_CNV_v1.txt (called 'child' in the script), SNPdata_parent.txt (called 'parent' in the script)
Description: SNP information - ratio, position, and BPs - per chromosome bins. Separated into phased and unphased lists. The lists are of the format "(x, y, z, ...)".

Column format:

1. Chromosome index (not the name!)
2. First position of bin within the chromosome
3. Last position of bin within the chromosome
4. List of phased SNP ratios.
5. List of unphased ratios.
6. List of phased SNP positions.
7. List of unphased SNP positions.
8. List of phased alleles.
9. List of unphased alleles.

Allele format: "B:D/E".

Unphased: 'B' is always Z. 'D' and 'E' are the 'parent' alleles.

Unphased: 'B' is the most frequent allele.

Hapmap

haplotypeMap.txt

Created by: hapmap.install_3.php, with data from hapmap.install_2.php
Source: User input.
Description: Used for diploid-based hapmaps. For each child dataset the user can specify which regions in the child belong to which chromosome homolog (this is presumably known in advance from LOH areas, which the child genome should contain).

Row format:

1. Reference genome (the diploid parent).
2. Child genome (presumably, an LOH strain).
3. C1(a[r1:r2] b[r3:r4]); C2(b[r5:r6]); - a list of chromosomes (can be a subset), each a list of homolog labels per the chromosome fragments.
4. 'C#' is the numeric chromosome ID.
5. 'a' and 'b' are the homolog names.
6. r# - the positions in the chromosome.

haplotypeFragments.N.txt

Where 'N' is an index of the child used to create this file.

Created by: hapmap.expand_definitions.py
Source: haplotypeMap.txt
Description: a reformatted haplotypeMap.txt, where every homolog fragment gets its own line.

SNPdata_parent.txt - from "one diploid" method hapmap creation

Created by: goes through a few incarnations, the last of them being at the hands of hapmap.process_child.py (called from hapmap.install_4.sh).
Source: putative_SNPs_v4.txt of the parent (called 'inputFile1') and SNP_CNV_v1.txt of the child (called 'inputFile2')
Description: the final hapmap.

The column format is the same as in putative_SNPs_v4, with the added last column of phasing data:

• 0 - the left allele belongs to the A homolog (and the right to the B homolog).
• 1 - the left allele belongs to the B homolog (and the right to the A homolog).
• 10 - the child didn't contain a read at this position (perhaps due to a deletion).
• 11 - this position didn't appear in any fragments defined in haplotypeFragments (perhaps the child was diploid at that position, just as the parent was).
• 12 - the child's base call (the most frequent read) at this position didn't match any of the two parent's alleles (perhaps a novel mutation or error in sequencing).