March 9, 2023
Meta’s newest large language model family, LLaMA (Large Language Model Meta AI), has surfaced online illegally, along with its weights, and is now freely accessible via torrents.
Since LLaMA was not released as a public chatbot, but rather as an open-source package to which anyone in the AI community can request access, it is intended to be accessible only to approved researchers, government officials, or members of civil society. Unfortunately, anyone can now download it thanks to 4chan, where a member uploaded a torrent file for LLaMA.
On Thursday around 9:45 p.m. IST, user ‘llamanon’ posted on 4chan’s technology board, releasing the LLaMA 7B and 65B models via torrent. The model was made available on 4chan’s AI Chatbot General megathread, which serves as a central location for testing the roleplaying capabilities of the latest AI chatbots.
This torrent link was then added to the LLaMA GitHub page as a pull request with the title’save bandwidth by using a torrent to distribute more efficiently’. This pull request was posted alongside the Google Forms link Meta was using to provide access to the bot, implying a dig at the LLM application process. A second pull request was also submitted to the project, this time with a torrent link to a different set of weights.
According to the leaked version, LLaMa, like other AI models such as OpenAI’s GPT-3, is built on a massive collection of word fragments, or “tokens.” LLaMa can take an input of words and predict the next word to recursively generate more text, just exactly as Meta explains in a February blog post.
LLaMa comes in a variety of sizes, with the LLaMa 65B and LLaMa 33B trained on 1.4 trillion tokens. The model was trained using datasets scraped from Wikipedia, books, academic papers from ArXiv, GitHub, Stack Exchange, and other sites, according to the LLaMA model card.
The sources for this piece include an article in TheRegister.
