git-market/rasbt-LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-14 15:21:39 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
46d09b30d9
rasbt-LLMs-from-scratch/ch05/01_main-chapter-code
History
rasbt 90d239b4f7 fix merge conflict
2024-04-22 07:05:40 -05:00
..
.gitignore Make datesets and loaders compatible with multiprocessing (#118) 2024-04-13 13:57:56 -05:00
ch05.ipynb Return nan if val loader is empty (#124) 2024-04-20 08:02:30 -05:00
exercise-solutions.ipynb Make datesets and loaders compatible with multiprocessing (#118) 2024-04-13 13:57:56 -05:00
gpt_download.py fix merge conflict 2024-04-22 07:05:40 -05:00
gpt_generate.py remove requests dependency (#125) 2024-04-21 14:15:05 -05:00
gpt_train.py Return nan if val loader is empty (#124) 2024-04-20 08:02:30 -05:00
previous_chapters.py update numbering 2024-04-22 07:00:20 -05:00
README.md Organized setup instructions (#115) 2024-04-10 22:09:46 -04:00
tests.py rename hparams to settings 2024-04-05 07:24:46 -05:00

README.md

Chapter 5: Pretraining on Unlabeled Data

  • ch05.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module from the previous chapter, which we import in ch05.ipynb to pretrain the GPT model
  • gpt_train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model
  • gpt_generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI