git-market/rasbt-LLMs-from-scratch
1
0
Fork 0
mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-11 14:21:41 +08:00
Code Issues Actions 11 Packages Projects Releases Wiki Activity
7643c6c0c4
rasbt-LLMs-from-scratch/ch05/01_main-chapter-code
History
SSebo 7643c6c0c4
Update ch05.ipynb (#297) 
typo
2024-08-05 07:12:27 -05:00
..
ch05.ipynb Update ch05.ipynb (#297) 2024-08-05 07:12:27 -05:00
exercise-solutions.ipynb Test code in pytorch 2.4 (#285) 2024-07-24 21:53:41 -05:00
gpt_download.py Add download help message (#274) 2024-07-19 08:29:29 -05:00
gpt_generate.py update generate to match output in main chapter 2024-06-22 12:01:51 -05:00
gpt_train.py Test code in pytorch 2.4 (#285) 2024-07-24 21:53:41 -05:00
previous_chapters.py fixed num_workers (#229) 2024-06-19 17:36:46 -05:00
README.md add main and optional sections 2024-06-19 17:48:25 -05:00
tests.py check gpt files (#208) 2024-06-12 07:19:10 -05:00

README.md

Chapter 5: Pretraining on Unlabeled Data

Main Chapter Code

  • ch05.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module and GPTModel class from the previous chapters, which we import in ch05.ipynb to pretrain the GPT model
  • gpt_download.py contains the utility functions for downloading the pretrained GPT model weights
  • exercise-solutions.ipynb contains the exercise solutions for this chapter

Optional Code

  • gpt_train.py is a standalone Python script file with the code that we implemented in ch05.ipynb to train the GPT model (you can think of it as a code file summarizing this chapter)
  • gpt_generate.py is a standalone Python script file with the code that we implemented in ch05.ipynb to load and use the pretrained model weights from OpenAI