Small Language Model

Large models are trained on everyone. This was an experiment in the opposite direction: take DistilGPT-2, small enough to fine-tune on a laptop, and train it on a decade of my own journal entries.

The output is not useful, exactly. It is something stranger — a model that has learned the shape of how one person writes without learning anything much about the world. Built with standard PyTorch training and evaluation patterns, with the data preparation and analysis in Jupyter, pandas and scikit-learn.