DySENSE: Dynamic Spatiotemporal ENergy Synthesis and Emulation with Satellite Imagery
Urban building energy modeling is usually treated as a prediction problem: given an existing city, estimate how much energy its buildings will consume. The authors of DySENSE argue that this framing is too narrow for planners who need to compare future urban forms, climate conditions, and seasonal demand. Their proposed framework generates satellite imagery and spatially aligned building-energy maps together, including monthly changes in consumption.
The motivation is practical. Cities are major consumers of energy and sources of emissions, while detailed monthly building data are difficult to obtain because of cost, privacy, and administrative constraints. A model trained only on a small set of observed cities may therefore struggle to support peak-load analysis, demand response, or early-stage urban design. DySENSE is intended to make scarce labeled data more useful by synthesizing additional training examples.
Separating urban form from energy dynamics
The central design decision is to separate what changes slowly from what changes quickly. Static urban morphology includes location and land-use composition, such as residential, commercial, industrial, service, park, and natural areas. Dynamic conditions include the month and five climate variables: maximum and minimum temperature, precipitation, water vapor pressure, and surface shortwave radiation.
The authors model these inputs as a joint conditional distribution over satellite imagery and energy maps. Two autoencoders first compress the modalities into latent representations. A conventional image VAE handles RGB satellite imagery, while a domain-specific Energy VAE is trained for sparse, long-tailed energy maps. This separate energy representation matters because building consumption does not have the same statistical structure as pixels in a natural image.
DySENSE then uses a U-ViT-style diffusion network to denoise the image and energy latents jointly. The shared denoising process is designed to preserve cross-modal relationships, so generated energy patterns remain spatially associated with the generated urban form. Static and dynamic text prompts are embedded separately: CLIP represents land-use and location information, while RemoteCLIP represents the monthly climate context.
At inference time, the framework can hold an urban image and static condition fixed while changing the month or climate prompt. In principle, this produces a sequence of energy maps for the same morphology rather than unrelated samples for each month. That capability is the paper’s main conceptual contribution: treating seasonal energy demand as a dynamic layer over a comparatively stable urban structure.
Dataset and evaluation
The authors assemble MUSE2, a multi-city dataset covering New York City, Washington, D.C., Norfolk, and Busan. Each sample is based on a 2 km by 2 km tile containing satellite imagery, land-use text, and a building-level energy map aggregated to 100-meter cells. Monthly electricity and natural-gas records are converted to a common energy unit and represented as ten energy classes, including background and nine positive consumption levels.
After expert review of missing or implausible annotations, the reported dataset contains 2,565 tiles and 23,880 monthly paired samples across the four cities. The authors evaluate both generation quality and usefulness for downstream energy prediction. For generated imagery they report FID, PSNR, and LPIPS. For energy prediction they use mean intersection over union, accuracy, and ASHRAE-oriented NMBE and CVRMSE metrics, which capture bias and prediction error on recovered energy values.
What the experiments show
The reported results support several claims, although they do not establish that synthetic data can replace real measurements. For static generation, DySENSE achieves an FID of 36.7362, PSNR of 11.5177, and LPIPS of 0.5604. The authors interpret these values, together with visual examples, as evidence that the model produces plausible urban imagery and energy maps with useful spatial correspondence.
The stronger evidence comes from downstream augmentation experiments. When limited real training data are combined with synthetic static imagery-energy pairs, the authors report an improvement of about 6.8 percentage points in mIoU and a 16.7% reduction in CVRMSE relative to their real-data baseline. The benefit is most visible when the real training set is small and narrows as more real data become available.
Dynamic augmentation performs better in the reported comparison. With synthetic monthly samples, mIoU improves by about 8.68 percentage points and CVRMSE falls by 9% to 18%, depending on the baseline. For one listed predictor trained with 25% of the real data, accuracy rises from 0.5343 to 0.6217, while CVRMSE decreases from 33.58% to 15.31%. The authors also report low distribution distances for generated monthly maps and an average Pearson correlation of 0.96 for monthly patterns across cities. These results suggest that the model captures broad seasonal structure, not necessarily every building’s exact load.
Limitations and practical meaning
The geographic scope is the clearest limitation. Four cities in North America and East Asia cannot represent the full range of climates, construction practices, energy systems, building regulations, and urban morphologies. The authors explicitly call for testing in more cities and climate zones, especially regions with limited data. They also identify external validation with utility audits or smart-meter data as an important next step.
There are modeling limitations as well. Energy labels are discretized into quantile-based classes, and continuous values are recovered using representative class values. That simplifies a highly variable physical quantity and may hide errors within each class. The paper’s seasonal correlations and downstream improvements are encouraging, but they should not be read as proof that generated maps are suitable for operational grid control or investment decisions without local calibration.
For researchers, DySENSE offers an open dataset and a concrete recipe for multimodal urban-energy augmentation. For planners and building scientists, its most credible near-term use is scenario exploration: testing how a fixed urban layout might behave across months, or improving predictive models when labeled data are scarce. Any deployment should compare synthetic outputs with local metering, preserve uncertainty estimates, and treat generated energy maps as decision-support evidence rather than ground truth.
Sources
AI-assisted research summary. Check the original paper for complete methods, results, and limitations.