Instructions on how to train a function for a new kmer size is not clear

[mahmudhera]

The repository README contains Usage instructions for n=100 and n=20. However, there are no clear instructions on how to train a function for say n=31. Should we first generate training samples? How do we generate training samples? Do we then use siaincpu.py to do the training? If so, how do we set the parameters? A guideline on these would be super helpful for someone who is not familiar with the codebase.

Thank you!

[XY5180]

Hi @mahmudhera, I appreciate your interest in our work! Our model supports training with different sequence lengths, and we offer two versions of the model for this purpose. For shorter sequences, such as those of length 20 or, in your example, n=31, it is recommended to use the one-layer Siamese CNN model: seq_n20/functions/siaincp_runner.py. For longer sequences, it is better to choose the deeper Siamese CNN model, such as one with a length of 100: seq_n100/functions/siaincp_runner.py. Of course, you can directly train with n=31 sequences if you already have them, or a function in simulation/seqSim.py to generate sequence pairs of any length, as many as you need. Regarding parameter settings, it depends on your desired model performance, and more details are available in our paper: https://academic.oup.com/bioinformatics/article/40/Supplement_1/i318/7700858. Please let me know if you have more specific questions.

[mahmudhera]

Hi @XY5180 , I am trying to train from the scratch with n=31 sequences, since I believe if I use the function trained for n=20 for 31-mers, the performance may not be as well as expected. In brief, what I am trying to do are the following:

a. generate training examples for n=31
b. training using these training examples

Before doing this for n=31, I thought I should try to train the lsb-function for n=100 first using your code, and recreate the results you reported in the paper. As I am trying to do so, I am running into many issues. First, what I am struggling to understand is: how do I generate the training examples for n=100? There is a python script, and a shell script. Which should I run? It seems that running the bash script generate many files -- do I concatenate them and use all of them as training data? (A related discussion is #3).

Next, the other thing I am trying to understand is the following: once I have the training examples, how do I take the existing code for n=100, and train a function? When I tried to run seq_n100/siaincp_runner.py, I ran into undefined names (related issue is #2).

And finally, I am trying to understand the minimal-energy-consuming route to change the code for n=20, and make it work for n=31. I understand that the paper has the theoretical details, but I thought the github repo might have a mechanism to plug-and-run training samples for n=something else, for the ease of general users. Correct me if I am wrong.

Thanks and best,
MRH

[XY5180]

Hi @mahmudhera, thank you for sharing your background. Of course, you can start by simulating and training the LSB function for a sequence length of 100; however, it will take a longer time (depending on the number of training samples and the deeper network structure). I suggest trying length = 20 first if your goal is to understand the code better. I also provided the answer under the other two sub-functions. Please check.
GitHub repository may have a mechanism to plug-and-run training samples, but it might require further encapsulation. I’m not very familiar with this part, and I regret that. However, I can provide you with an easier guide to run it:

1. Modify the path in seqSim.py and customize the parameters in the bash script (see question: simulation function for more details on parameters). This step aims to generate enough sequence pair samples, with length=31, for training and testing the model.
2. Train the model using the generated sequence pair samples by siaincp_runner.py. The parameters that need to be customized include m, which represents the bucket dimension, and k, which represents the number of buckets.
3. The trained model is then used as the bucket function.

Feel free to let me know if you have more questions.