Pretraining
The massive next-token-prediction phase over trillions of tokens that gives a model its raw capability, before any tuning makes it useful as an assistant.
The massive next-token-prediction phase over trillions of tokens that gives a model its raw capability, before any tuning makes it useful as an assistant.