This is a project of my course "Deep learning Fundamentals and Practice". Note, code and other materials will be uploaded
Hugging face intro-Chinese version in Zhihu
Hugging face courses and tutorial
gpt-2like Byte-Pair-Enconding(BPE) tokenizer: roneneldan
See the file: Server_Training_a_causal_language_model_from_scratch_(PyTorch).ipynb This is adapted from the NLP course in hugging face. See here.
Right now this model and relative information of the pretain process can be found here
See file: Server_Fineture_process.ipynb
Right now this model and relative information of the pretain process can be found here
The final report can be found here