[CCoE Notice] Thesis Announcement: Yuzhen Hu, "A Unified Diffusion based Representation Learning Framework for Hyperspectral Image Analysis"

[Greenwell, Stephen J]

[Thesis Defense Announcement at the Cullen College of Engineering]
A Unified Diffusion based Representation Learning Framework for Hyperspectral Image Analysis

Yuzhen Hu
April 29, 2025, 1:00 p.m. to 3:00 p.m. (CST)
Teams link<https://urldefense.com/v3/__https://teams.microsoft.com/l/meetup-join/19*3ameeting_MGVhZjFmODMtNTY0ZC00OGQ1LWE4MTYtNDYxZGM0NDFkYzdk*40thread.v2/0?context=*7b*22Tid*22*3a*22170bbabd-a2f0-4c90-ad4b-0e8f0f0c4259*22*2c*22Oid*22*3a*22c0a4c8cf-9aed-4850-a3b2-880cc2ea1c47*22*7d__;JSUlJSUlJSUlJSUlJSUl!!LkSTlj0I!Cv6A-l8L4h-IT3Kc2XwSwVwrszNXbEZsxz13on1EX3VLIWtXnCtZjo2Dd45V-5sYnnxWDRHmECwvpsu94UM7lFPWrD4$ >
Meeting ID: 287 765 081 841 0
Passcode: Vw9YG7LH

Committee Chair:
Saurabh Prasad, Ph.D.
Committee Members:
Yashashree Kulkarni, Ph.D. | David Mayerich, Ph.D.
Abstract
Hyperspectral imaging is a promising remote sensing modality for robust land-cover mapping - however, analysis of such imagery is often challenging, owing to the high spectral dimensionality and limited pixel-level annotations for training. Additionally, hyperspectral imagery acquired from satellites are often lower resolution compared to their multi-spectral or color-image counterparts.
Diffusion models exhibit strong generative capabilities and effectively preserve spatial structure, making them well-suited for feature extraction from low spatial resolution hyperspectral imagery with degraded textures. We validate their efficacy by proposing an approach -- GeoDiffNet-F, which leverages pseudo-RGB representations and a diffusion model pre-trained on natural images (ImageNet) without domain adaptation to extract transferable low-level spatial features. Combined with per-pixel spectral reflectance, these features significantly improve classification performance and outperform existing baselines, highlighting the strength of diffusion-based spatial feature extraction in hyperspectral land-cover mapping. While GeoDiffNet-F demonstrates the utility of low-level features, the full potential of diffusion models lies in their ability to generate hierarchical, tree-like representations through multi-step denoising-progressions ranging from global structures to fine details. Fully leveraging this capacity requires adaptation to the target domain. A central challenge is catastrophic forgetting, which can degrade generalization from large-scale pretraining, especially under limited data. To address this, we propose a parameter-efficient domain-adaptive pre-training strategy for unsupervised representation learning, which updates only adaptive normalization layers (e.g., FiLM-like affine modulation). This enables the model to extract modality-aware features and adapt rapidly while preserving general spatial priors.
Building on these insights, we introduce UniDiff-MM, a unified diffusion-based framework that addresses the dual challenges of domain adaptation and multimodal fusion in hyperspectral imagery. UniDiff-MM combines two key innovations: (1) the above parameter-efficient adaptation strategy, and (2) modality-aware conditioning, where the diffusion process is conditioned on distinct spectral views-pseudo-RGB for spatial and PCA-reduced bands for spectral content. This enables a single shared diffusion model to adapt to multiple domain-specific representations. It preserves modality-specific features while aligning them through shared weights, projecting each view into a shared representation space. In practice, UniDiff-MM can generate modality-specific outputs when conditioned accordingly, demonstrating effective domain adaptation and consistent structure across modalities.
We validate UniDiff-MM on hyperspectral pixel-wise classification tasks, where it achieves state-of-the-art performance and demonstrates its effectiveness for robust, multi-modal domain-adaptive representation learning.
[Engineered For What's Next]

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://Bug.EGR.UH.EDU/pipermail/engi-dist/attachments/20250416/5ce3b347/attachment-0001.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 28468 bytes
Desc: image001.png
Url : http://Bug.EGR.UH.EDU/pipermail/engi-dist/attachments/20250416/5ce3b347/attachment-0002.png 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 5699 bytes
Desc: image002.png
Url : http://Bug.EGR.UH.EDU/pipermail/engi-dist/attachments/20250416/5ce3b347/attachment-0003.png