Generative Refocusing: Flexible Defocus Control from a Single Image

Quick Overview

Generative Refocusing (GenFocus) allows users to infinitely adjust focus, depth of field, and the shape of bokeh from a single input image by employing a clever two-stage pipeline consisting of Diblernet for image restoration and Bokenet for controllable image synthesis, successfully overcoming limitations of prior methods by utilizing a hybrid training strategy combining synthetic data with unpaired real data guided by EXIF metadata.

Key Points: GenFocus turns any single photo into a virtual camera, enabling infinite adjustment of focus plane, depth of field, and even the actual shape of the bokeh post-capture. The method solves three prior AI weaknesses: it works on blurry inputs (unlike systems requiring an All-In-Focus input), avoids renderer bias by using real data, and allows artistic aperture shape control beyond simple round bokeh. The two-stage pipeline separates tasks: Diblernet restores the image to a perfectly sharp, all-in-focus version using positional disentanglement and pre-dibler dropout for robust detail recovery. Bokenet handles image synthesis, applying customized blur based on a defocus map derived from scene depth and an intensity level K, which is calculated using physics-based guidance. The breakthrough training strategy is a hybrid semi-supervised approach, combining synthetic data for geometry with unpaired real data guided by EXIF metadata (focal length, f-number) to learn authentic optical qualities. The system demonstrated top performance in deblurring, cleanly recovering fine details like text, and achieved more natural synthesis because its blur is more forgiving due to training on real-world data. A surprising feature is text-guided deblurring, where users can input text prompts during inference to visually correct reconstruction hallucinations made by the diffusion-based Diblernet.

Context: The research discussed is Generative Refocusing (GenFocus), a novel computer vision technique designed to provide flexible defocus control from just one photograph, effectively decoupling artistic choices like selective focus and bokeh effects from the moment the shutter clicks. This technology aims to supersede previous AI editing methods that were brittle, often requiring perfectly sharp inputs or producing results marred by 'renderer bias' due to over-reliance on synthetic simulation data.