The Pre-trained Language Model What Can GTP-3, Switch Transformer, and Enlightenment 2.0 Do?

In just two years, the parameter scale of the pre-trained language model has increased tenfold, and competition in the 21st century is indeed everywhere. What is a pre-trained language model? What can they do?

In May 2020, OpenAI released the pre-trained model GPT-3 with 175 billion parameters. It can not…