Error while executing pretrain_model_pssn_l_x_l.ipynb

[Lanart93]

I'm a newbie in this environment of deep learning and I'm just trying to replicate the results you got from this code using Google Colab. Some issues I tried to solve them myself but I got stuck in this point.

First of all, I'd like to know if the libraries extracted from requirements.txt are updated because some of the problems I got where because the libraries weren't updated. (Installing notebook==6.1.5 will require to update jupyter-client==5.3.4, jupyter-core==4.6.1 and terminado==0.8.3). Also, I had to delete pywinpty because the Google Colab environment is based in Linux (I believe this library is meant to work on a Windows environment through Jupyter or something like that, correct me if I'm wrong).

Now, the error that I get when I run the codes is the one in the line mentioned below:
In [13]: dists = np.array([embedding_matrix(matrix) for matrix in dists])

If I decide to run the entire code without installing the proper libraries mentioned in requirements.txt (That will mean working with tensorflow-gpu==2.4.1 instead of 1.12.0 and all the rest of the libraries will be updated compared to the ones written in the txt file), I can get a little further on the codes and I will get the error until the next couple of lines:

In [22]: his = model.fit(inputs, outputs, epochs=35, batch_size=2, verbose=1, shuffle=True, validation_split=0.1)
print(his.history)

Still I can continue to run the rest of the lines of this code but the model won't be trained and I'll get silly results:

Is there a way to fix the issue and run all the notebooks smoothly in Google Colab? or will be better to just do it on Jupyter Notebooks? Hope someone can help me with this issue.

[hypnopump]

1. Libraries are not updated.
2. I think this project was in TF 1.13 or 1.14 (maybe 2.2 or 2.1, i don't remember exactly), in case you want to try with it. I think you might have to run several !pip uninstall and !pip install to add the right versions.
3. How will you be getting the data if this is not run in a local setup? from the issue where i provided a google drive link? It's likely that the file will be removed in the future (i'm approaching the drive limit sorry ) So i'd advise to do it in a local setup.
4. for the int/float thing of your latest error, idk what's failing, so you could try to inspect a bit more and post here your findings so i can get a sense of what's going on.

For what i know, @roberCO managed to run it on local, so you should be able to do it as well.

[Lanart93]

1. and 2. So is possible to run the codes having Tensorflow>=2.0? I'll have a look on that running it several times and check which one works perfect.

2. I looked for the link you provided but it has been removed. I didn't run it properly on a local setup, what I did is downloading all the preliminary data and uploading it on my Google Drive, then I mounted my Google Drive on Colab so I can reach all the data and execute the notebooks according to your instructions. I'll try to run it locally to see if that can be the issue.

3. I'll check closely that last error and see if I can get some hints about what's the problem in here.

[hypnopump]

so you saying that you're running the julia preprocessing notebook on colab as well?

[Lanart93]

Yes, on a different Colab notebook for the preprocessing because once I installed Julia on Colab, I found difficult to go back to Python to keep executing the rest of the notebooks. So what I did is to execute the Julia preprocessing notebook and save the processed data in my Google Drive so I can continue in a different notebook the rest of the coding.

[hypnopump]

cool i never tried this, that's why i was asking. okay so the data does not seem to be the problem, but i guess you can always double check the shapes of things and do some debugging. I haven't touched the code for a while and you might already know the saying "any code of your own that you haven't looked at for six or more months might as well have been written by someone else"...

So if you find the error, could you report it so future people can take a look and benefit from the knowledge?

[Lanart93]

Yes sure, I'll do my best to find the error and report it. If someone else come with the same error hope we can discuss it and find it more easily.

[hypnopump]

a weird think i've just realized is that the predicted label and true label from the confusion matrix above are flipped lol