Pretraining: Learning from the Internet

See what pretraining really is — your tiny GPT scaled to trillions of tokens, what learning from the internet means, the base model it produces, and where it fits in the two-phase recipe.