Nvidia’s GPT-4 language model is now being used by Linxi “Jim” Fan, an AI researcher at Nvidia, and colleagues in Minecraft to power a bot called Voyager.
Voyager solves challenges and improves its skills by using GPT-4’s ability to analyze the game’s state. It generates objectives and code to guide its actions, like suggesting fishing if it has a fishing rod and sees a river. Voyager also learns from its mistakes and refines its code using error messages and game feedback. It builds a library of code, allowing it to handle complex tasks, explore more of the game, gather more items, travel farther, and build tools faster than other AI agents.
The employment of language models by Voyager demonstrates their capacity to execute practical activities on computers, demonstrating substantial technical progress. This method may be used to create a software assistant that automates operations on PCs or mobile devices, akin to Voyager’s navigation in Minecraft.
In fact, OpenAI provides “plugins” that allow Voyager to communicate with web businesses like Instacart. Microsoft, the game’s owner, is also teaching AI programs how to play the game and has announced Windows 11 Copilot, an operating system feature that uses machine learning and APIs to automate particular activities.
The sources for this piece include an article in Wired.
