Publications

2026

1. 

   Real Noise Decoupling for Hyperspectral Image Denoising

   Yingkai Zhang, Tao Zhang, Jing Nie, and Ying Fu

   In Proc. of Association for the Advancement of Artificial Intelligence (AAAI), 2026

   Abs arXiv Bib PDF Code

   Hyperspectral image (HSI) denoising is a crucial step in enhancing the quality of HSIs. Noise modeling methods can fit noise distributions to generate synthetic HSIs to train denoising networks. However, the noise in captured HSIs is usually complex and difficult to model accurately, which significantly limits the effectiveness of these approaches. In this paper, we propose a multi-stage noise-decoupling framework that decomposes complex noise into explicitly modeled and implicitly modeled components. This decoupling reduces the complexity of noise and enhances the learnability of HSI denoising methods when applied to real paired data. Specifically, for explicitly modeled noise, we utilize an existing noise model to generate paired data for pre-training a denoising network, equipping it with prior knowledge to handle the explicitly modeled noise effectively. For implicitly modeled noise, we introduce a high-frequency wavelet guided network. Leveraging the prior knowledge from the pre-trained module, this network adaptively extracts high-frequency features to target and remove the implicitly modeled noise from real paired HSIs. Furthermore, to effectively eliminate all noise components and mitigate error accumulation across stages, a multi-stage learning strategy, comprising separate pre-training and joint fine-tuning, is employed to optimize the entire framework. Extensive experiments on public and our captured datasets demonstrate that our proposed framework outperforms state-of-the-art methods, effectively handling complex real-world noise and significantly enhancing HSI quality.

   @inproceedings{zhang2026real,
     title = {Real Noise Decoupling for Hyperspectral Image Denoising},
     author = {Zhang, Yingkai and Zhang, Tao and Nie, Jing and Fu, Ying},
     booktitle = {Proc. of Association for the Advancement of Artificial Intelligence (AAAI)},
     volume = {40},
     number = {15},
     pages = {12925-12933},
     year = {2026},
   }

2. 

   Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration

   Yingkai Zhang^*, Shuang Chen^*, Ye Tian, Yunyi Gao, and 2 more authors

   Tsinghua Science and Technology, 2026

   Abs arXiv Bib PDF Code

   Positron emission tomography (PET) offers powerful functional imaging but involves radiation exposure. Efforts to reduce this exposure by lowering the radiotracer dose or scan time can degrade image quality. While using magnetic resonance (MR) images with clearer anatomical information to restore standard-dose PET (SPET) from low-dose PET (LPET) is a promising approach, it faces challenges with the inconsistencies in the structure and texture of multi-modality fusion, as well as the mismatch in out-of-distribution (OOD) data. In this paper, we propose a supervise-assisted multi-modality fusion diffusion model (MFdiff) for addressing these challenges for high-quality PET restoration. Firstly, to fully utilize auxiliary MR images without introducing extraneous details in the restored image, a multi-modality feature fusion module is designed to learn an optimized fusion feature. Secondly, using the fusion feature as an additional condition, high-quality SPET images are iteratively generated based on the diffusion model. Furthermore, we introduce a two-stage supervise-assisted learning strategy that harnesses both generalized priors from simulated in-distribution datasets and specific priors tailored to in-vivo OOD data. Experiments demonstrate that the proposed MFdiff effectively restores high-quality SPET images from multi-modality inputs and outperforms state-of-the-art methods both qualitatively and quantitatively.

   @article{zhang2026supervise,
     title = {Supervise-assisted Multi-modality Fusion Diffusion Model for PET Restoration},
     author = {Zhang, Yingkai and Chen, Shuang and Tian, Ye and Gao, Yunyi and Jiang, Jianyong and Fu, Ying},
     journal = {Tsinghua Science and Technology},
     year = {2026},
   }

3. 

   Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning

   Yingkai Zhang, Tao Zhang, Jing Nie, and Ying Fu

   In Proc. of Conference on Computer Vision and Pattern Recognition (CVPR), 2026

   Abs arXiv Bib Code

   Unregistered hyperspectral image (HSI) super-resolution (SR) typically aims to enhance a low-resolution HSI using an unregistered high-resolution reference image. In this paper, we propose an unmixing-based fusion framework that decouples spatial-spectral information to simultaneously mitigate the impact of unregistered fusion and enhance the learnability of SR models. Specifically, we first utilize singular value decomposition for initial spectral unmixing, preserving the original endmembers while dedicating the subsequent network to enhancing the initial abundance map. To leverage the spatial texture of the unregistered reference, we introduce a coarse-to-fine deformable aggregation module, which first estimates a pixel-level flow and a similarity map using a coarse pyramid predictor. It further performs fine sub-pixel refinement to achieve deformable aggregation of the reference features. The aggregative features are then refined via a series of spatial-channel abundance cross-attention blocks. Furthermore, a spatial-channel modulated fusion module is presented to merge encoder-decoder features using dynamic gating weights, yielding a high-quality, high-resolution HSI. Experimental results on simulated and real datasets confirm that our proposed method achieves state-of-the-art super-resolution performance. The code will be available at \urlhttps://github.com/yingkai-zhang/UAFL.

   @inproceedings{zhang2026enhancing,
     title = {Enhancing Unregistered Hyperspectral Image Super-Resolution via Unmixing-based Abundance Fusion Learning },
     author = {Zhang, Yingkai and Zhang, Tao and Nie, Jing and Fu, Ying},
     booktitle = {Proc. of Conference on Computer Vision and Pattern Recognition (CVPR)},
     year = {2026},
   }

2025

1. 

   Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance

   Yingkai Zhang, Zeqiang Lai, Tao Zhang, Ying Fu, and 1 more author

   International Journal of Computer Vision, 2025

   Abs arXiv Bib PDF Code

   Hyperspectral images (HSIs) super-resolution (SR) aims to improve the spatial resolution, yet its performance is often limited at high-resolution ratios. The recent adoption of high-resolution reference images for super-resolution is driven by the poor spatial detail found in low-resolution HSIs, presenting it as a favorable method. However, these approaches cannot effectively utilize information from the reference image, due to the inaccuracy of alignment and its inadequate interaction between alignment and fusion modules. In this paper, we introduce a Spatial-Spectral Concordance Hyperspectral Super-Resolution (SSC-HSR) framework for unaligned reference RGB guided HSI SR to address the issues of inaccurate alignment and poor interactivity of the previous approaches. Specifically, to ensure spatial concordance, i.e., align images more accurately across resolutions and refine textures, we construct a Two-Stage Image Alignment (TSIA) with a synthetic generation pipeline in the image alignment module, where the fine-tuned optical flow model can produce a more accurate optical flow in the first stage and warp model can refine damaged textures in the second stage. To enhance the interaction between alignment and fusion modules and ensure spectral concordance during reconstruction, we propose a Feature Aggregation (FA) module and an Attention Fusion (AF) module. In the feature aggregation module, we introduce an Iterative Deformable Feature Aggregation (IDFA) block to achieve significant feature matching and texture aggregation with the fusion multi-scale results guidance, iteratively generating learnable offset. Besides, we introduce two basic spectral-wise attention blocks in the attention fusion module to model the inter-spectra interactions. Extensive experiments on three natural or remote-sensing datasets show that our method outperforms state-of-the-art approaches on both quantitative and qualitative evaluations. Our code is publicly available to the community (https://github.com/BITYKZhang/SSC-HSR).

   @article{zhang2025unaligned,
     title = {Unaligned RGB Guided Hyperspectral Image Super-Resolution with Spatial-Spectral Concordance},
     author = {Zhang, Yingkai and Lai, Zeqiang and Zhang, Tao and Fu, Ying and Zhou, Chenghu},
     journal = {International Journal of Computer Vision},
     volume = {133},
     number = {9},
     pages = {6590--6610},
     year = {2025},
     publisher = {Springer}
   }

2024

1. 

   Technique Report of CVPR 2024 PBDL Challenges

   Ying Fu, Yu Li, Shaodi You, Boxin Shi, and 7 more authors

   arXiv preprint arXiv:2406.10744, 2024

   Abs arXiv Bib

   The intersection of physics-based vision and deep learning presents an exciting frontier for advancing computer vision technologies. By leveraging the principles of physics to inform and enhance deep learning models, we can develop more robust and accurate vision systems. Physics-based vision aims to invert the processes to recover scene properties such as shape, reflectance, light distribution, and medium properties from images. In recent years, deep learning has shown promising improvements for various vision tasks, and when combined with physics-based vision, these approaches can enhance the robustness and accuracy of vision systems. This technical report summarizes the outcomes of the Physics-Based Vision Meets Deep Learning (PBDL) 2024 challenge, held in CVPR 2024 workshop. The challenge consisted of eight tracks, focusing on Low-Light Enhancement and Detection as well as High Dynamic Range (HDR) Imaging. This report details the objectives, methodologies, and results of each track, highlighting the top-performing solutions and their innovative approaches.

   @article{fu2024technique,
     title = {Technique Report of CVPR 2024 PBDL Challenges},
     author = {Fu, Ying and Li, Yu and You, Shaodi and Shi, Boxin and Chen, Linwei and Zou, Yunhao and Wang, Zichun and Li, Yichen and Han, Yuze and Zhang, Yingkai and others},
     journal = {arXiv preprint arXiv:2406.10744},
     year = {2024}
   }

2022

1. 

   Reinforcement learning based plug-and-play method for hyperspectral image reconstruction

   Ying Fu and Yingkai Zhang

   In CAAI International Conference on Artificial Intelligence, 2022

   Abs Bib PDF

   Hyperspectral images have multi-dimensional information and play an important role in many fields. Recently, based on the compressed sensing (CS), spectral snapshot compressive imaging (SCI) can balance spatial and spectral resolution compared with traditional methods, so it has attached more and more attention. The Plug-and-Play (PnP) framework based on spectral SCI can effectively reconstruct high-quality hyperspectral images, but there exists a serious problem of parameter dependence. In this paper, we propose a PnP hyperspectral reconstruction method based on reinforcement learning (RL), where a suitable policy network through deep reinforcement learning can adaptively tune the parameters in the PnP method to adjust the denoising strength, penalty factor of the deep denoising network, and the terminal time of iterative optimization. Compared with other model-based and learning-based methods and methods with different parameters tuning policies, the reconstruction results obtained by the proposed method have advantages in quantitative indicators and visual effects.

   @inproceedings{fu2022reinforcement,
     title = {Reinforcement learning based plug-and-play method for hyperspectral image reconstruction},
     author = {Fu, Ying and Zhang, Yingkai},
     booktitle = {CAAI International Conference on Artificial Intelligence},
     pages = {466--477},
     year = {2022},
     organization = {Springer}
   }