Skip to content

[Feature Request] Integrate Dreamerv4-MC as a new interactive world model backend #1

@songbingze

Description

@songbingze

Hi WMFactory team,

First of all, thank you for open-sourcing WMFactory. The direction of providing one environment, one backend entry point, and a unified session API for multiple interactive world models is very valuable, especially for comparing and standardizing different world model inference experiences.

I would like to suggest adding support for Dreamerv4-MC / Dreamer-MC:

https://github.com/IamCreateAI/Dreamerv4-MC

Dreamerv4-MC is a real-time autoregressive world model inference project for Minecraft. It supports generating future game frames based on historical frames and user actions, and includes components such as an MAE tokenizer, a dynamic model, long-context inference, real-time interaction, and infinite generation.

Since WMFactory already provides a unified backend, session API, and frontend, integrating Dreamerv4-MC could be useful in several ways:

  1. Allow Dreamerv4-MC to be loaded and run through WMFactory’s unified backend/session interface.
  2. Make it easier to compare Dreamerv4-MC with existing Minecraft or interactive world model backends.
  3. Reuse WMFactory’s frontend so users can interact with Dreamerv4-MC using a consistent WASD / mouse / action format.
  4. Provide a reference adapter for integrating more autoregressive or diffusion-based world models in the future.

Thanks again for building WMFactory!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions