AI · AI model
NitroGen
Open vision-to-action foundation model for generalist autonomous game playing
NitroGen is an autonomous gaming foundation model developed by NVIDIA that translates raw visual gameplay frames directly into gamepad inputs. Trained on 40,000 hours of gameplay across more than 1,000 titles, it uses a SigLip2 vision transformer and diffusion transformer to play games without needing game-specific APIs.
What it's for
- Deploy autonomous agents to play PC games using standard gamepad inputs
- Benchmark vision-to-action model performance across different video game genres
- Research behavior cloning and generalist agents in interactive visual environments
Features
- Processes 256x256 RGB video frames directly from game screens
- Outputs gamepad actions including continuous joystick vectors and 17 binary buttons
- Trained on 40,000 hours of video gameplay across 1,000+ games
- Combines a SigLip2 vision transformer with a Diffusion Transformer (DiT)
- Includes inference server and automated game execution scripts for Windows
Pricing
Free for non-commercial research use under NVIDIA's license
Details
- Type
- AI model
- Category
- AI
- Topic
- Autonomous Agents
- Open source
- Yes
- Platforms
- Windows, Linux, Command line
- For
- AI researchers, game developers, and autonomous agent engineers
- Alternative to
- VPT (Video PreTraining), SIMA
- Tags
- gaming, behavior cloning, vision to action, nvidia, reinforcement learning, ai agents
Why it was saved
- @evolving.ai: Enables general-purpose gaming agents capable of playing diverse PC and console games directly from raw visual frames without custom game-specific APIs.
Related
Open-source, self-hosted AI agent with persistent memory and messaging integrations.
Open-source, fully local autonomous AI agent that browses and writes code
Open-source platform and framework for building autonomous AI agents
A Python multi-agent pattern pairing Opus orchestration with Haiku worker instances.
Open-source long-horizon agent harness with Docker sandboxes, memory, and sub-agents
All-in-one AI workspace with autonomous agent tools for content, research, and analysis.
Updated