StemmaLM

A lineage of small language models, each raised from scratch on what adults say to children.

Every individual hears real family conversations in order of the child's age, month by month, and can be asked to speak at every month of its life. Grown individuals can teach the next generation with their own speech, and the numbers join into a family tree.

lineage →log →about →

fig. 0 — the lineage so far

Registry

no.idgenparentheardwords per utterance, 0→72 mat 72 mloss

Words per utterance: real words only, 1,000 utterances per month. Loss: training loss at 72 months (nats per character).

Latest log

    How a generation is made

    1. raise A model that knows nothing hears a book of speech in order of the child's age, from 0 to 72 months.
    2. speak At every month it is given an adult's line and answers; what it says is kept.
    3. copy Grown, it copies the book word by word; at 5% of words it writes its own guess. Its copy raises the next generation.