Abstract

Recent advances in video generation enable a new paradigm for 3D scene creation: generating camera-controlled videos that simulate scene walkthroughs, then lifting them to 3D via feed-forward reconstruction techniques. This generative reconstruction approach combines the visual fidelity and creative capacity of video models with 3D outputs ready for real-time rendering and simulation. Scaling to large, complex environments requires 3D-consistent video generation over long camera trajectories with large viewpoint changes and location revisits, a setting where current video models degrade quickly. Existing methods for long-horizon generation are fundamentally limited by two forms of degradation: spatial forgetting and temporal drifting. As exploration proceeds, previously observed regions fall outside the model's temporal context, forcing the model to hallucinate structures when revisited. Meanwhile, autoregressive generation accumulates small synthesis errors over time, gradually distorting scene appearance and geometry. We present Lyra 2.0, a framework for generating persistent, explorable 3D worlds at scale. To address spatial forgetting, we maintain per-frame 3D geometry and use it solely for information routing -- retrieving relevant past frames and establishing dense correspondences with the target viewpoints -- while relying on the generative prior for appearance synthesis. To address temporal drifting, we train with self-augmented histories that expose the model to its own degraded outputs, teaching it to correct drift rather than propagate it. Together, these enable substantially longer and 3D-consistent video trajectories, which we leverage to fine-tune feed-forward reconstruction models that reliably recover high-quality 3D scenes.

Paper: https://arxiv.org/abs/2604.13036

Code: https://github.com/nv-tlabs/lyra

Weights: https://huggingface.co/nvidia/Lyra-2.0

Project Page: https://research.nvidia.com/labs/sil/projects/lyra2/

Example Workflows: https://github.com/Lightricks/ComfyUI-LTXVideo/tree/master/example_workflows/2.3

https://registry.comfy.org/publishers/masternc80/nodes/bboxnodes

cross-posted from: https://lemmy.world/post/45437770

What do people here think about local AI based solutions for generating Alt-Text? Could a tool be developed to use the AI horde to help generate Alt Text? I think having ways to do it in an automated fashion would be useful to the community, and could help increase adoption and usage of Alt Text.

Sorry if this isn't the right community for this kind of discussion.

Abstract

While proprietary systems such as Seedance-2.0 have achieved remarkable success in omni-capable video generation, open-source alternatives significantly lag behind. Most academic models remain heavily fragmented, and the few existing efforts toward unified video generation still struggle to seamlessly integrate diverse tasks within a single framework. To bridge this gap, we propose OmniWeaving, an omni-level video generation model featuring powerful multimodal composition and reasoning-informed capabilities. By leveraging a massive-scale pretraining dataset that encompasses diverse compositional and reasoning-augmented scenarios, OmniWeaving learns to temporally bind interleaved text, multi-image, and video inputs while acting as an intelligent agent to infer complex user intentions for sophisticated video creation. Furthermore, we introduce IntelligentVBench, the first comprehensive benchmark designed to rigorously assess next-level intelligent unified video generation. Extensive experiments demonstrate that OmniWeaving achieves SoTA performance among open-source unified models. The codes and model have already been publicly available. Project Page: this https URL.

Paper: https://arxiv.org/abs/2603.24458

Code: https://github.com/Tencent-Hunyuan/OmniWeaving

Models: https://huggingface.co/tencent/HY-OmniWeaving

Project Page: https://omniweaving.github.io/

Release: https://github.com/Lightricks/LTX-Desktop/releases