git-market/rasbt-LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-14 15:21:39 +08:00
Code Issues Actions Packages Projects Releases Wiki Activity
90b25ece3d
rasbt-LLMs-from-scratch/ch05/01_main-chapter-code
History
rasbt 31806828d0
add links to summary sections
2024-06-29 07:33:26 -05:00
..
ch05.ipynb add links to summary sections 2024-06-29 07:33:26 -05:00
exercise-solutions.ipynb minor fixes (#235) 2024-06-21 08:40:54 -05:00
gpt_download.py fix merge conflict 2024-04-22 07:05:40 -05:00
gpt_generate.py update generate to match output in main chapter 2024-06-22 12:01:51 -05:00
gpt_train.py fix typo in comment 2024-06-09 06:14:02 -05:00
previous_chapters.py fixed num_workers (#229) 2024-06-19 17:36:46 -05:00
README.md add main and optional sections 2024-06-19 17:48:25 -05:00
tests.py check gpt files (#208) 2024-06-12 07:19:10 -05:00

README.md

Chapter 5: Pretraining on Unlabeled Data

Main Chapter Code

  • ch05.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module and GPTModel class from the previous chapters, which we import in ch05.ipynb to pretrain the GPT model
  • gpt_download.py contains the utility functions for downloading the pretrained GPT model weights
  • exercise-solutions.ipynb contains the exercise solutions for this chapter

Optional Code

  • gpt_train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model (you can think of it as a code file summarizing this chapter)
  • gpt_generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI