EDGAR: Efficient Diffusion-Driven Gaussian Avatar Refinement via Synthetic Failure

Why Gaussian avatars still break

Gaussian splatting has made photorealistic head avatars substantially faster to build and render. Instead of relying on a conventional mesh or expensive neural rendering, these systems represent a face with many deformable 3D Gaussians that can be rasterized efficiently. That combination is attractive for virtual reality, telepresence, social games, and mixed-reality applications.

The trade-off is that monocular video and few-image capture leave much of a person’s geometry and appearance unobserved. A reconstructed avatar may look convincing from familiar views but fail in side profiles, wide-open mouths, raised eyebrows, hair regions, or other weakly constrained areas. The authors of EDGAR: Efficient Diffusion-Driven Gaussian Avatar Refinement via Synthetic Failure describe common symptoms including blurry textures, stretched details, floating Gaussian blobs, unstable geometry, and temporal inconsistency.

EDGAR is designed as a refinement layer for these imperfect avatars. Its goal is not to replace the underlying Gaussian representation, but to make the resulting avatar sharper and more faithful while preserving identity and facial motion.

Learning from synthetic failure

The central data challenge is straightforward: there is no standard collection of pairs showing a failed Gaussian avatar beside the ideal reconstruction of the same subject. The authors address this with EDGAR-SET, a triplet dataset containing a degraded avatar rendering, a ground-truth image, and a separate reference image of the same identity.

EDGAR-SET combines real-captured and synthetic data. The real component is intended to anchor training in natural human appearance, while the synthetic component makes it possible to vary viewpoints, expressions, lighting, and the number of input views. This produces failures such as blur, floating Gaussians, geometric misalignment, and view-dependent inconsistencies. The authors report a final collection of 400,000 high-resolution images covering thousands of identities.

Importantly, the dataset emphasizes recoverable failures. Examples in which expression or geometry diverges too severely from the target are filtered out using LPIPS-based similarity checks. This choice makes the learning problem more coherent, but it also defines the method’s practical boundary: EDGAR is not trained to repair arbitrary structural collapse.

Reference-guided, one-step diffusion

At inference time, EDGAR receives a degraded rendering and a reference image. The reference supplies identity and appearance information, reducing the risk that a diffusion model will simply invent plausible but incorrect facial details.

The refinement network is built on SD-Turbo, a one-step diffusion backbone. The authors adapt the backbone by learning self-attention and added decoder components while retaining much of the pretrained structure. Reference features are extracted with the self-supervised Sapiens model and injected through a fusion block into the diffusion network. In effect, the degraded image says what needs fixing, while the reference helps determine what the subject should look like.

EDGAR also uses a Mixture-of-Experts design. A lightweight router sends intermediate features to one of several experts, allowing the model to handle both nearly clean renderings and more strongly degraded images. Training combines pixel, perceptual, and style-related losses with an expression-consistency loss based on facial features. A load-balancing objective discourages the experts from collapsing into a single commonly used pathway.

The authors then take the method beyond frame-by-frame enhancement. They render the source Gaussian avatar from multiple views, refine those images with EDGAR, and use the refined results as pseudo-targets for optimizing one shared 3D avatar. This distillation step means the final avatar can render improved frames directly, without applying diffusion to every output.

What the experiments show

The authors evaluate EDGAR on three avatar systems: CAP4D, RGBAvatar, and GaussianAvatars. These cover different reconstruction regimes, including image-based, video-based, and multi-view settings. Across the reported tests, EDGAR generally improves perceptual and reconstruction measures over the unrefined outputs and an SDEdit-style diffusion baseline.

The gains are especially consistent for RGBAvatar and GaussianAvatars, where EDGAR leads the reported CLIP, L1, LPIPS, and PSNR comparisons at the tested input settings. CAP4D shows a more mixed metric-level picture, but EDGAR still achieves the strongest CLIP and L1 results in the authors’ table. Qualitatively, the method reduces blur and floating artifacts while recovering details around hair, eyes, skin, ears, and wrinkles.

The comparison with SDEdit is important. The authors report that SDEdit often introduces stochastic changes, oversmoothing, and identity drift. EDGAR’s reference conditioning produces smaller identity losses in cross-identity reenactment, although it does not completely preserve or improve the input avatar’s identity similarity. The ablation results also identify reference conditioning and synthetic training data as particularly important components; removing either generally hurts performance.

There is evidence for improved geometric consistency as well. Calibrated cross-view tests show better depth discrepancy and valid reprojection coverage for CAP4D and GaussianAvatars. RGBAvatar improves at four and eight inputs but worsens in the single-input case. Large-yaw and open-mouth tests are favorable for the two video-based pipelines, while CAP4D produces mixed results.

Efficiency is one of EDGAR’s practical strengths. The reported one-time refinement takes an average of 5.04 minutes for CAP4D, 7.96 minutes for RGBAvatar, and 16.60 minutes for GaussianAvatars in the authors’ setup. The refined avatars retain the original Gaussian count and rendering path, with output throughput remaining within 2% of the base models in the reported benchmarks.

Limitations and practical implications

EDGAR should be viewed as a quality upgrade for recoverable appearance artifacts, not as a substitute for better capture. Severe errors in unobserved regions may remain because the training data excludes cases where the target is ambiguous or too far from the degraded image. Additional observations or explicit geometric supervision may be necessary for those failures.

Refined views can also disagree. When pseudo-targets conflict, optimizing a shared avatar may average details or retain view-dependent mistakes. The temporal results reinforce this caution: temporal warping error fell for CAP4D and RGBAvatar but increased for GaussianAvatars, even though the average across pipelines decreased.

For practitioners, the most useful takeaway is architectural rather than cosmetic. A reference image, avatar-specific failure data, expression-aware supervision, and multi-view optimization together offer a controlled way to apply diffusion priors without adding diffusion inference to the final rendering loop. EDGAR therefore looks most promising for systems that already have a usable Gaussian avatar but need sharper, more stable output under limited capture. It is less suited to avatars whose geometry is fundamentally missing or structurally wrong.


Sources

AI-assisted research summary. Check the original paper for complete methods, results, and limitations.

Avatar photo

Tech News

Jeff is a passionate blog writer who shares clear, practical insights on technology, digital trends and AI industries. With a focus on simplicity and real-world experience, his writing helps readers understand complex topics in an accessible way. Through his blog, Jeff aims to inform, educate, and inspire curiosity, always valuing clarity, reliability, and continuous learning.

↑ ↓ navigate Enter open Esc close