This project provides a Google Colab that explains the core ideas in GPT-2 with code-snippets for illustration. It includes an PyTorch implementation of GPT-2. This implementation while not identical to the OpenAI implementation, is final built version of the concepts explained in the Colab. The variations are highlighted.
- Python 3.9+
- PyTorch
- Tiny Shakespeare Dataset