Member-only story
GPT-Neo is the name of our codebase for transformer-based language models loosely styled around the GPT architecture. One of our goals is to use GPT-Neo to replicate a GPT-3 sized model and open source it to the public, for free.
GPT-3 is open sourced? An open source project called GPT-Neo launched by Eleuther AI, the publicly released GPT-3 equivalently reproduces the pre-trained model (1.3B & 2.7B) and updates it on the Colab notebook, but GPT-NeoGP is not OpenAI GPT-3.
Although GPT-Neo is compared with the real GPT-3, the amount of parameters is still very small (the larger version only has the parameter amount of the smallest model of GPT-3). However, it is open source and free. After the news was released, it was reposted by various big Vs immediately. Within 10 hours, it quickly received 1.5K likes, and the corresponding github link has also received close to 3k stars.
On the Eleuther AI official website, their description of GPT-Neo is a series of code names of…
