NS-Gym offers a first standardized simulation toolkit for non-stationary decision problems
Researchers introduce NS-Gym, an open-source simulation framework for Markov decision processes whose dynamics change over time, built inside the widely used Gymnasium ecosystem.
In plain words
Reinforcement learning agents are usually tested in environments whose rules stay fixed, but real-world conditions often shift. NS-Gym is a new open-source toolkit that lets researchers build and compare algorithms for these changing settings. It plugs into the popular Gymnasium library, so existing workflows can be adapted rather than rebuilt.
What was announced
The authors, publishing on arXiv, argue that traditional decision-making models typically assume stationary dynamics, while many real applications require agents to act as environmental dynamics evolve over time. Non-stationary Markov decision processes (NS-MDPs) provide a way to model such problems, but the field has lacked standardized simulation frameworks comparable to those available for stationary problems. NS-Gym is presented as the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework.
A key design choice separates the evolution of the environmental parameters that characterize non-stationarity from the agent's decision-making module, which the authors say allows modular and flexible adaptation to dynamic environments. The toolkit encapsulates key problem characteristics and types in NS-MDPs and is described as the first effort to develop standardized interfaces and benchmark problems for consistent, reproducible evaluation of algorithms under non-stationary conditions. The team also benchmarked several algorithmic approaches from prior work using NS-Gym. Documentation, a project webpage, and code are available at nsgym.io and on GitHub under the scope-lab-vu organization.
What it does not show
- The source is the paper's abstract, so it reports the authors' own claims; no independent evaluation of the toolkit or its benchmarks is included.
- The abstract does not specify which algorithms were benchmarked, the benchmark results, supported environments, or software requirements.
For security and IT teams
- Access
- Project documentation, webpage, and code are published at nsgym.io and github.com/scope-lab-vu/ns_gym.
Sources
Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 19 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.