mirror of https://github.com/rasbt/LLMs-from-scratch.git synced 2026-04-11 02:11:40 +08:00

History

Sebastian Raschka 0972ded530 Add a note about weight tying in Llama 3.2 (#386 )		2024-10-05 09:20:54 -05:00
..
tests	Improve rope settings for llama3 (#380 )	2024-10-03 08:29:54 -05:00
config.json	move access token to config.json	2024-09-23 08:56:16 -05:00
converting-gpt-to-llama2.ipynb	Implement Llama 3.2 (#383 )	2024-10-05 07:30:47 -05:00
converting-llama2-to-llama3.ipynb	Add a note about weight tying in Llama 3.2 (#386 )	2024-10-05 09:20:54 -05:00
previous_chapters.py	GPT to Llama (#368 )	2024-09-23 07:34:06 -05:00
README.md	Implement Llama 3.2 (#383 )	2024-10-05 07:30:47 -05:00
requirements-extra.txt	Llama 3.2 requirements file	2024-10-05 07:32:43 -05:00
standalone-llama32.ipynb	Add a note about weight tying in Llama 3.2 (#386 )	2024-10-05 09:20:54 -05:00

README.md

Converting GPT to Llama

This folder contains code for converting the GPT implementation from chapter 4 and 5 to Meta AI's Llama architecture in the following recommended reading order:

converting-gpt-to-llama2.ipynb: contains code to convert GPT to Llama 2 7B step by step and loads pretrained weights from Meta AI
converting-llama2-to-llama3.ipynb: contains code to convert the Llama 2 model to Llama 3, Llama 3.1, and Llama 3.2
standalone-llama32.ipynb: a standalone notebook implementing Llama 3.2