Hypernoa Astrum
Hypernoa Astrum is an adaptive RL environment built on OpenEnv 0.2.1 that trains AI to reason, adapt, and align — not just solve tasks.
Core trick
A GRPO-trained Qwen model is rewarded on multi-objective scores across five stakeholder groups, with traps that penalize reward hacking.
Description
Hypernoa Astrum is an adaptive RL environment built on OpenEnv 0.2.1 that trains AI to reason, adapt, and align — not just solve tasks. It simulates 5 competing stakeholder groups (Workers, Management, Regulators, Customers, AI Systems), 3 dynamic phases (stable, value shift, crisis), and deliberately designed alignment traps that test whether AI agents cheat their reward function. The multi-objective reward measures effectiveness, fairness, alignment, and adaptability. Our trained agent reaches 24.7 reward starting from scratch, nearly matching the expert baseline at 25.1, while random scores only 14.6. All 3 alignment traps are resisted. Trained with HF TRL GRPO on Qwen2.5-0.5B using CoreWeave H100 via Northflank. Addresses Problem Statement 3.1 (World Modeling) and Statement 5 (Wild Card).
README excerpt
# Hypernoa Astrum **Where Next-Generation Intelligence Learns to Think, Adapt, and Align** Today's AI is trained to complete tasks. Tomorrow's AI must learn to **reason under uncertainty, adapt when the world changes, balance competing objectives, and align with values it wasn't explicitly programmed for**. This requires a fundamentally new kind of training infrastructure. **Hypernoa Astrum** is that infrastructure: a platform of **adaptive, evolving environments** where intelligent systems — running on GPUs, TPUs, and future compute substrates — develop the capabilities that matter beyond raw performance: **multi-objective reasoning, value alignment, long-horizon adaptation, and safe self-improvement**. Built on [OpenEnv 0.2.1](https://github.com/meta-pytorch/OpenEnv) | [OpenEnv Hackathon SF](https://cerebralvalley.ai/e/openenv-hackathon-sf) **Live Demo**: [HF Space](https://huggingface.co/spaces/ABNaidu/hypernoa-astrum) | **Repo**: [GitHub](https://github.com/naidu1212/hypernoa-astrum) --- ## The Gap in Current AI Training Current training pipelines optimize for **single objectives on static benchmarks**. This produces systems that are powerful but brittle: - **No adaptation**: Models fail when distributions shift. Real-world values, rules, and constraints change constantly. - **No multi-objective balance**: Real decisions require trading off efficiency vs fairness vs safety vs cost. Current systems optimize one thing. - **No alignment under pressure**: When a syst