July 25, 2023

Andrej Karpathy, former Tesla AI director, has released a simplified version of the Llama 2 language model that can run on a single computer.

The model, called “Baby Llama,” is based on the Llama 2 architecture but uses a much smaller number of parameters. This makes it possible to run the model on a laptop or other resource-constrained device.

The model can generate text at a rate of 100 tokens per second, which is significantly faster than other LLMs that can only run on GPUs and could lead to the development of new applications for LLMs, such as on-device language translation or chatbots.

Karpathy trained Baby Llama on the TinyStories dataset, which contains a collection of short stories. He reports that the model can generate text at a rate of 100 tokens per second. This is significantly faster than other LLMs that can only run on GPUs.

Karpathy’s work could also lead to the development of new applications for LLMs, such as on-device language translation or chatbots.

The sources for this piece include an article in AnalyticsIndiaMag.