Advanced Trends in Optical Remotely Sensed Data Fusion: Pansharpening Case Study

Remote Sensing is the process of collecting information about objects from a distance by using airborne or spaceborne sensors. They are designed to detect specific sources of energy. Moreover, they interpret these observations to provide data about objects on the Earth’s surface and atmosphere. Remote sensing images are acquired by multiple operating satellites, ranged from multispectral [1], super-spectral [2] to hyperspectral [3] sensors. Examples of optical data are collected from Pleiades [4], WorldView-2 [5], Sentinel-5P [6] and Hyperion [7]. They have been employed in many large-scale applications, such as land cover classification, environmental monitoring and change detection, in which both high spatial and spectral qualities are desired. In the following, a short review about the recent techniques applied to pansharpening as well as the quality assessment protocols have been reported.

Due to physical constraints, commercial satellites can only provide panchromatic (PAN) data with high spatial and low spectral resolution and multi spectral (MS) data with high spectral and low spatial resolution. By fusing the MS and the PAN images, it is possible to synthesize spatially enhanced MS data, which provides a better understanding of the observed scene. This process is known as pan-sharpening [1]. According to this topic, several data fusion contests have been proposed in the scientific communities which dated from 2006 to 2024 [8- 12]. In the same spectra, researchers have been advanced in the hyper- sharpening [13]. It is viewed as the combination of the panchromatic and the hyperspectral imageries focusing on the high spectral qualities of the data, which are required to many real applications.

An extensive review of pan-sharpening approaches can be found in [14-19]. Credited methods existing in the literature are ranged from classical techniques [1], which are based on a simple modelling of the fusion task, to recent developed methods using complex models for resolving inverse problems [15] (i.e. Bayesian technique, sparse representation theory, total variation regularization), and nonlinear methods exploiting deep learning models [16]. These methods can achieve good fusion products in terms of spatial and spectral qualities. However, they may suffer from the high computational complexity caused by the optimization techniques which limits their application in practice [19].

The classical pan-sharpening methods can be recast into a general protocol [14] and summarized by the two steps: i) extracting the geometrical details from PAN image; and ii) inferring them into the expanded MS bands by using different injection models. Generally, all classical pan sharpening algorithms differ in the manner of spatial details extraction. In this context, they can be classified into two main groups: Component Substitution (CS) and Multi-Resolution Analysis (MRA) algorithms. These methods have been extensively explored and benchmarked in the literature review [15]. High-frequency details are defined as an image residual between the original PAN image and either a linear combination of the MS bands (for CS methods) or its low-resolution version performed by means of a pyramidal or multi-scale decomposition [19] (for MRA methods). The novel methods based on deep learning [16] and statistical models [17] are based on diverse techniques of computing and optimization for enhancing the spatial quality of MS/HS imageries [20].

The assessment of the quality of a high-resolution fused image is a challenging task because of the lack of the reference images. Typical methods are related to the reduced-resolution evaluation by exploiting Wald’s protocol [21] or to the full-resolution indexes without reference, called QNR protocol [22]. Recently, new methods have been proposed in the literature based on the combination of the two above-mentioned protocols [23]. More recently, the researchers include the two protocols into diverse deep learning models inspired from the technical physics of optical sensors [24- 26].

In this paper, a mini review of remotely sensed data fusion is reported. Specifically, it concerns the pansharpening field, in which several approaches have been stated, ranged from classical (as CS and MRA) to advanced methods (statistical and deep learning). Indeed, the pansharpening algorithms are classified according to the principle of details extraction and injection, and the spectral enhancement. Furthermore, the quality assessment protocols have been introduced for evaluating the fused products.

The author declares no conflict of interest.

1. Vivone G, et al. (2015) A critical comparison among pan-sharpening algorithms. IEEE Transactions on Geoscience and Remote Sensing 53(5): 2565-2586.
2. Salvini R, Garzelli A, Rindinella A, Beltramone L, Vanneschi C, et al. (2024) Sentinel-2 and PRISMA satellite data for urban land classification in Tuscany region (TUS:CAN PROJECT). In Proceedings of Earth Resources and Environmental Remote Sensing/GIS Applications XV; SPIE pp. 1319711.
3. Amigo JM, Grassi S (2019) Configuration of hyperspectral and multispectral imaging systems. In Data Handling in Science and Technology, Elsevier 32: 17-34.
4. Perko R, Raggam H, Roth PM (2019) Mapping with Pléiades-End-to-End Workflow. Remote Sensing 11(17): 2052.
5. Padwick C, Deskevich M, Pacifici F, Smallwood S (2010) WorldView-2 pan-sharpening. In Proceedings of the American Society for Photogrammetry and Remote Sensing Annual Conference: Opportunities for Emerging Geospatial Technologies, San Diego pp. 1-14.
6. Carbone A, Restaino R, Vivone G, Chanussot J (2024) "Model-based Super-Resolution for Sentinel-5P data". IEEE Transactions on Geoscience and Remote Sensing 99: 1-1.
7. Dian R, Li S, Sun B, Guo A (2021) Recent advances and new guidelines on hyperspectral and multispectral image fusion. Information Fusion 69: 40-51.
8. Alparone L, Wald L, Chanussot J, Thomas C, Gamba P, et al. (2007) Comparison of pansharpening algorithms: Outcome of the 2006 GRSS data-fusion contest. IEEE Transactions on Geoscience and Remote Sensing 45(10): 3012-3021.
9. Hänsch et al. (2023) "Report on the 2023 IEEE GRSS Data Fusion Contest - Large-Scale Fine- Grained Building Classification for Semantic Urban Reconstruction". IEEE Geoscience and Remote Sensing Magazine 11(4): 146-149.
10. Vivone et al. (2023) "Computer Vision for Earth Observation: The First IEEE GRSS Image Analysis and Data Fusion School". IEEE Geoscience and Remote Sensing Magazine 11(2): 95-100.
11. Ullo SL, Vivone G, Taşkın G, Hänsch R, Verma U (2024) "Computer Vision for Earth Observation: The Second GRSS Image Analysis and Data Fusion School ". IEEE Geoscience and Remote Sensing Magazine 12(1): 195-204.
12. Persello C, Prasad S, Vivone G, Lonjou V, Bretar F, et al (2024) "2024 IEEE GRSS Data Fusion Contest: Rapid Flood Mapping ". IEEE Geoscience and Remote Sensing Magazine 12(2): 109-112.
13. Ciotola et al. (2025) "Hyperspectral Pansharpening: Critical Review, Tools and Future Perspectives". IEEE Geoscience and Remote Sensing Magazine 13(1): 311-338.
14. Vivone G, Dalla Mura M, Garzelli A, Pacifici F (2021) "A Benchmarking Protocol for Pansharpening: Dataset, Pre-processing, and Quality Assessment". IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 99: 1-1.
15. Vivone et al. (2021) A New Benchmark Based on Recent Advances in Multispectral Pansharpening: Revisiting Pansharpening with Classical and Emerging Pansharpening Methods. IEEE Geoscience and Remote Sensing Magazine 9(1): 53-81.
16. Vivone et al. (2025) Deep Learning in Remote Sensing Image Fusion: Methods, Protocols, Data and Future Perspectives". IEEE Geoscience and Remote Sensing Magazine 13(1): 269-310.
17. Deng et al. (2022) Machine Learning in Pansharpening: A Benchmark, from Shallow to Deep Networks". IEEE Geoscience and Remote Sensing Magazine 10(3): 279-315.
18. Meng X, et al. (2021) A Large-Scale Benchmark Data Set for Evaluating Pansharpening Performance: Overview and Implementation. IEEE Geoscience and Remote Sensing Magazine 9(1): 18-52.
19. Jin C, Deng LJ, Huang TZ, Vivone G (2022) Laplacian pyramid networks: A new approach for multispectral pansharpening. Information Fusion 78: 158-170.
20. Vivone G (2023) Multispectral and Hyperspectral Image Fusion in Remote Sensing: A Survey. Information Fusion 89: 405-417.
21. Wald L, Ranchin T, Mangolini M (1997) Fusion of satellite images of different spatial resolutions: Assessing the quality of resulting images. Photogrammetric Engineering and Remote Sensing 63(6): 691-699.
22. Alparone L, Aiazzi B, Baronti S, Garzelli A, Nencini F, et al. (2008) Multispectral and panchromatic data fusion assessment without reference. Photogrammetric Engineering and Remote Sensing 74(2): 193-200.
23. Vivone G, Addesso P, Chanussot J (2019) A Combiner-based Full Resolution Quality Assessment Index for Pansharpening. IEEE Geoscience and Remote Sensing Letters 16(3): 437-441.
24. Agudelo-Medina OA, Benitez-Restrepo HD, Vivone G, Bovik A (2019) Perceptual Quality Assessment of Pan-Sharpened Images. MDPI Remote Sensing 11(7): 877.
25. Arienzo A, Vivone G, Garzelli A, Alparone L, Chanussot J (2022) Full Resolution Quality Assessment of Pansharpening: Theoretical and Hands-on Approaches. IEEE Geoscience and Remote Sensing Magazine 10(3): 168-201.
26. Xing Y, Wang M, Yang S, Jiao L (2018) Pan-sharpening via deep metric learning. ISPRS Journal of Photogrammetry and Remote Sensing 145: 165-183.