Ground-Truth-Free World Model Fine-Tuning with GameWorldScore

World models for video generation face a critical challenge: how to improve generation quality without access to ground-truth future frames during fine-tuning. We propose GameWorldScore, a unified ground-truth-free reward system that enables reinforcement learning fine-tuning of diffusion-based world models. Our approach combines five complementary reward components: Inverse Kinematics Score (RIK) for action fidelity, Temporal Consistency Score (RTC) for motion smoothness, Aesthetic Quality Score (RAQ) for visual appeal, Reality Grounding Score (RRG) for domain anchoring, and Anti-Drift Reward (AD) for preventing degradation. We demonstrate that Group Relative Policy Optimization (GRPO) with GameWorldScore successfully fine-tunes the Oasis world model on Minecraft video generation, achieving improved visual quality, temporal coherence, and action-conditioned generation without requiring paired ground-truth data.

• ›GameWorldScore: a comprehensive, ground-truth-free reward system for world model evaluation combining five complementary signals.
• ›Detailed formulations for RIK (action fidelity), RTC (temporal consistency), RAQ (aesthetic quality), RRG (reality grounding), and AD (anti-drift).
• ›GRPO-based fine-tuning of the Oasis diffusion transformer (DiT-S/2) on Minecraft video generation, with a frozen VAE and autoregressive rollouts.
• ›Elimination of dependency on paired ground-truth data via a proxy reward mechanism for domain adaptation.

GameWorldScore combines five reward signals into a unified objective: R_total = w1·RIK + w2·RTC + w3·RAQ + w4·RRG + w5·RAD. The Oasis world model, consisting of a frozen VAE and a fine-tuned Diffusion Transformer (DiT), generates frames autoregressively. GRPO normalises advantages within groups of samples from the same prompt, making it well-suited to diffusion model training without requiring paired ground-truth observations.

The unified model, trained on all five rewards at once, matches each single-reward specialist on that specialist's own metric while holding the rest — action fidelity 0.469 against the RIK-only model's 0.466, anti-drift 0.842 against 0.859. Validation means over the held-out set:

• ›RIK only: enforces action compliance, but temporal smoothness and visual quality both degrade without RTC and RAQ.
• ›RTC only: maximises frame-to-frame smoothness and generates 'slippery' content that ignores the intended camera movement.
• ›RRG only: anchors texture to the Minecraft distribution and prevents domain drift, but motion stays uncontrolled.
• ›AD only: prevents sharpness degradation across long sequences, yet cannot steer the model toward any action or semantic content.

• ›Reward weight selection is unsolved — choosing the wrong weight degrades other metrics, and automatic weighting is left to future work.
• ›RIK depends on a pre-trained inverse dynamics model, which may not exist for other domains.
• ›Long-horizon generation still shows some degradation.