mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-11 14:21:41 +08:00

History

Sebastian Raschka 4f9c9fb703 Update tests.py		2024-10-23 07:48:33 -05:00
..
tests	Update tests.py	2024-10-23 07:48:33 -05:00
config.json	move access token to config.json	2024-09-23 08:56:16 -05:00
converting-gpt-to-llama2.ipynb	Implement Llama 3.2 (#383 )	2024-10-05 07:30:47 -05:00
converting-llama2-to-llama3.ipynb	RoPE increase (#407 )	2024-10-21 19:58:38 -05:00
previous_chapters.py	GPT to Llama (#368 )	2024-09-23 07:34:06 -05:00
README.md	Implement Llama 3.2 (#383 )	2024-10-05 07:30:47 -05:00
requirements-extra.txt	fixed Llama 2 to 3.2 NBs (#388 )	2024-10-06 09:56:55 -05:00
standalone-llama32.ipynb	RoPE increase (#407 )	2024-10-21 19:58:38 -05:00

README.md

Converting GPT to Llama

This folder contains code for converting the GPT implementation from chapter 4 and 5 to Meta AI's Llama architecture in the following recommended reading order:

converting-gpt-to-llama2.ipynb: contains code to convert GPT to Llama 2 7B step by step and loads pretrained weights from Meta AI
converting-llama2-to-llama3.ipynb: contains code to convert the Llama 2 model to Llama 3, Llama 3.1, and Llama 3.2
standalone-llama32.ipynb: a standalone notebook implementing Llama 3.2