PAPER

Mind‑to‑Image is an end‑to‑end pipeline developed at Obvious which translates the brain’s fMRI signals into pictures a person is merely imagining. While earlier projects could only rebuild what someone was looking at, here the participants were asked to picture scenes in their minds, and we collected six hours of brain‑scan data, and worked on the basis of a top‑tier brain‑to‑image system to build one which could handle the richer signals of pure imagination. The result is a first demonstration that both remembered scenes and purely invented scenes can be turned into recognizable portraits or landscapes straight from neural data.

Mind-to-Image

We collected six hours of brain‑scans, and worked on the basis of a top‑tier brain‑to‑image system to build one which could handle the richer signals of pure imagination. The result is a demonstration that remembered and purely invented scenes can be turned into visuals straight from neural data.

Mind‑to‑Image is an end‑to‑end pipeline developed at Obvious which translates the brain’s fMRI signals into pictures a person is merely imagining. While earlier projects could only rebuild what someone was looking at, here the participants were asked to picture scenes in their minds.

An introduction to Mind-to-Image

Once this mapping was established, it could be applied to the imagination phase, allowing the reconstruction of scenes that existed only in thought.

The model was trained using several hours of fMRI data collected while an artist first viewed, then remembered, and finally imagined images inside the scanner. From the first two phases, it learned to associate distinct patterns of brain activation with visual features such as shape and composition.

A 3-step process for reconstructing imagination

The picture is meant to correspond to the imagined content: whether a portrait, a landscape, or something more abstract. In this way, the system turns purely mental images into visible ones, offering a direct glimpse of imagination itself.

When a participant lies in the MRI and imagines a scene, Mind-to-Image interprets the resulting pattern of brain activity and converts it into a “visual fingerprint.” That fingerprint guides an image generator to produce a picture.

Image generation based on brain scan analysis

The mind-to-image protocol inscribes itself as a scientific follow up of techniques such as hypnosis, automatic writing and exquisite corpse, allowing creation to emerge from the mind and be given a physical form.

This technique inscribes itself as a following of the surrealist movement. Surrealism is a cultural movement which explores the expression of the subconscious, and the techniques which allow for the suppression of the different filters between the realm of the mind, and physical artistic expression.

The extension of the surrealist movement

Projecting Visual Mental Imagination of the Brain from fMRI

OnlyFlow

Traditional approaches struggle to combine an arbitrary motion with an unrelated prompt, while OnlyFlow tackles this challenge by using optical flow as a conditioning signal.

OnlyFlow is a method developed by Obvious to improve control over video diffusion models. Its goal is to generate a video that follows both a text prompt and a desired movement, such as a camera pan or object motion.

Motion Conditioning for Video Diffusion Models

Schema OnlyFlow1

The pipeline works by first extracting optical flow from an auxiliary video, which describes how pixels move between frames. This motion is encoded through a dedicated motion encoder, and the resulting features are injected into the temporal attention layers of the video diffusion model.

Optical-Flow-Guided Motion

The continuity in the movement used to generate the videos create a global coherence, and the artwork acts both as a proof of work, and a visual demonstration of what can be done using OnlyFlow.

“Be Water”, an artwork created using OnlyFlow, was showcased in an immersive space at Artechouse NYC and at Kanvas Dubaï, highlighting the creative potential of this tool. The video artwork is composed of four independently generated videos, each one of them displayed on a wall.

Application as a tool for artistic video creation

Optical Flow based Motion Conditioning for Video Diffusion Models

JoLT

Unlike standard low-to-high pipelines, both resolutions evolve at the same time and continuously exchange information, reducing the usual trade-off between global coherence and local richness.

JoLT generates large, densely detailed images by denoising low- and high-resolution latent streams together. The low-resolution stream preserves the overall composition, while the high-resolution stream builds local detail.

Two Resolutions, One Generation

This bidirectional exchange synchronizes the two trajectories, helping fine details remain consistent with the global scene instead of being added only after the composition is already fixed.

At every denoising step, JoLT turns the low-resolution estimate into a joint prior that guides the high-resolution tiled generation. The updated high-resolution canvas is then fed back into the low-resolution stream.

A Bidirectional Exchange

On 4096² images, JoLT produces substantially more measured visual complexity than strong high-resolution baselines while remaining competitive in text alignment, aesthetics, and overall quality.

JoLT gives artists direct control over image complexity. A master prompt defines the global scene, a detail prompt controls local appearance, and a single parameter α adjusts how strongly high-resolution detail influences the composition.

Controlling Visual Complexity

Joint Latent Trajectories for Context-Guided High-Resolution Tiled Generation

AnimatedFresco

FrescoDiffusion

Instead of generating one low-resolution clip and simply upscaling it, the method performs tiled diffusion directly on a large latent canvas.

FrescoDiffusion turns a single ultra-high-resolution image into video at the same scale, preserving fine details that standard image-to-video models lose when complex scenes are resized.

Animating Frescoes

This prior regularization keeps separate tiles coherent while still allowing the model to create new details and motion at full resolution.

A low-resolution animation first captures the global structure and motion of the image. Its latent trajectory is then upscaled and used as a prior to guide every step of the 4K tiled denoising process.

A Global Prior for Local Detail

Tests on FrescoArchive and VBench-I2V show improved coherence, fidelity and controllability over tiled baselines, with only a small runtime overhead.

FrescoDiffusion can treat different regions independently, relaxing the prior earlier on active areas while keeping backgrounds constrained for longer. This creates motion without breaking the overall composition.

Research

PAPER