train a mini-language-model:

https://github.com/jingyaogong/minimind

Learning Rate

RMSNorm

Pre-train

SFT

Generate