Fact of the day20 November Archive

2017

the transformer paper

The “T” in GPT comes from a 2017 paper

Google researchers published “Attention Is All You Need” in 2017. It introduced the transformer, a network that weighs how much every word in a text matters to every other word. GPT stands for Generative Pre-trained Transformer, and almost every chatbot you use today is built on the same idea.

Source: Vaswani et al., arXiv (2017)