← News

ResearchSources: arXiv · 18 Nov 2026

NS-Gym offers a first standardized simulation toolkit for non-stationary decision problems

Researchers introduce NS-Gym, an open-source simulation framework for Markov decision processes whose dynamics change over time, built inside the widely used Gymnasium ecosystem.

In plain words

Reinforcement learning agents are usually tested in environments whose rules stay fixed, but real-world conditions often shift. NS-Gym is a new open-source toolkit that lets researchers build and compare algorithms for these changing settings. It plugs into the popular Gymnasium library, so existing workflows can be adapted rather than rebuilt.

What was announced

The authors, publishing on arXiv, argue that traditional decision-making models typically assume stationary dynamics, while many real applications require agents to act as environmental dynamics evolve over time. Non-stationary Markov decision processes (NS-MDPs) provide a way to model such problems, but the field has lacked standardized simulation frameworks comparable to those available for stationary problems. NS-Gym is presented as the first simulation toolkit designed explicitly for NS-MDPs, integrated within the popular Gymnasium framework.

A key design choice separates the evolution of the environmental parameters that characterize non-stationarity from the agent's decision-making module, which the authors say allows modular and flexible adaptation to dynamic environments. The toolkit encapsulates key problem characteristics and types in NS-MDPs and is described as the first effort to develop standardized interfaces and benchmark problems for consistent, reproducible evaluation of algorithms under non-stationary conditions. The team also benchmarked several algorithmic approaches from prior work using NS-Gym. Documentation, a project webpage, and code are available at nsgym.io and on GitHub under the scope-lab-vu organization.

What it does not show

  • The source is the paper's abstract, so it reports the authors' own claims; no independent evaluation of the toolkit or its benchmarks is included.
  • The abstract does not specify which algorithms were benchmarked, the benchmark results, supported environments, or software requirements.

For security and IT teams

Access
Project documentation, webpage, and code are published at nsgym.io and github.com/scope-lab-vu/ns_gym.

Sources

Source-support review completed 19 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 19 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.