Full search page →

U2Fusion: one unsupervised model across fusion tasks

Optional · paper guide 8 of 11 for DLCV 10 · Image Fusion: Complementary Images, Useful Outputs. Citation numbers refer to the reference list at the bottom of this page.

Navigate: this page and this track: Navigate: this page and this track

On this page

Supplementary notes · 6

  1. The defocus spread effect
  2. Building and collapsing a Laplacian pyramid
  3. Sparse-representation fusion
  4. Mutual information as a fusion metric
  5. Training signals of representative fusion networks
  6. VIF and source-referenced fusion scores

Paper guides · 11

  1. Exposure fusion (Mertens, Kautz and Van Reeth)
  2. VIFB: a visible and infrared image fusion benchmark
  3. The infrared–visible fusion survey of Liu et al. (TPAMI 2025)
  4. DenseFuse: learn reconstruction, choose fusion in feature space
  5. DDFM: a diffusion prior with likelihood rectification
  6. FusionGAN: thermal intensity with an adversarial visible-image prior
  7. IFCNN: supervised fusion with a variable number of inputs
  8. U2Fusion: one unsupervised model across fusion tasks
  9. SwinFusion: attention within and across the sources
  10. SeAFusion: let semantic segmentation influence the fused image
  11. TarDAL: fuse for both appearance and object detection

Deep Learning for Computer Vision

  1. 02 · Deep Image Classification
View the whole course tree →

Xu et al.’s TPAMI 2022 paper treats infrared–visible, medical, multi-exposure and multi-focus fusion as related information-preservation problems [32]. Its distinctive goal is one set of learned parameters, trained across tasks without ground-truth fused images. The following details refer to the journal version, which extends the earlier FusionDN design.

Challenge

Different fusion tasks appear to need incompatible rules. A warm infrared target should remain salient; an anatomical image should retain structure; an underexposed shot may contain useful content despite weak raw gradients; an out-of-focus region should contribute less than its sharp counterpart. A single hand-designed activity measure is unlikely to express all these preferences.

There are two separate obstacles. First, without fused targets, what should supervise the output? Second, when the same network learns another fusion task, how can it retain earlier capabilities? U2Fusion answers the first with an adaptively weighted similarity loss and the second with elastic weight consolidation, or EWC. “Unified” therefore means more than reusing an architecture with separately trained checkpoints.

Method

Source features estimate information-preservation weights for training; the extra feature/measurement branches are not all part of the deployed image-to-image network.
Original paper figure 3 — Source features estimate information-preservation weights for training; the extra feature/measurement branches are not all part of the deployed image-to-image network.

Figure source: [32], Public manuscript PDF, PDF page 4. Original PDF.

1. Generate an image, rather than a decision map. Concatenate the two single-channel inputs along the channel axis. For a patch of height HH and width WW, the input is H×W×2H\times W\times2 and the output is H×W×1H\times W\times1. The fully convolutional generator uses dense feature connections, reflection padding and stride-one 3×33\times3 convolutions, without pooling. Dense connections make earlier features available to later layers and help information and gradients propagate.

Figure 8 and the official implementation specify an initial 44-channel convolution, five dense convolutions adding 44 channels each, and reconstruction convolutions reducing the resulting 264 channels to 128, 64, 32 and 1 [32], [56]. This gives ten convolutions. The prose’s reference to “first seven” dense layers is inconsistent with its diagram and the code; the explicit tensor path above follows the latter. Intermediate activations use a leaky rectifier with slope 0.2, and the final activation is tanh. Input/output scaling must match the implementation when reproducing it.

2. Measure source information using a fixed pretrained network. Each grayscale source is repeated into three channels and passed through VGG-16 [57]. Features are taken before its five max-pooling stages. For a 64×6464\times64 patch, their spatial sizes are 64,32,16,8,464,32,16,8,4 on each side, with channel counts 64,128,256,512,51264,128,256,512,512. Shallow features respond to texture; deeper features provide larger-context structure. The feature extractor is used to define training supervision, not as a second generator.

The paper measures activity through a Laplacian on these feature maps:

g(I)=15∑j=151HjWjDj∑k=1Dj∥Δϕjk(I)∥F2.g(I)=\frac15\sum_{j=1}^{5} \frac{1}{H_jW_jD_j}\sum_{k=1}^{D_j} \left\|\Delta\phi_j^k(I)\right\|_F^2.

II is a source patch; ϕjk(I)\phi_j^k(I) is channel kk of the feature tensor at stage jj; Hj,Wj,DjH_j,W_j,D_j are that tensor’s height, width and channel count. Δ\Delta is the spatial Laplacian and ∥⋅∥F2\|\cdot\|_F^2 sums squared entries. Normalizing by the tensor size and averaging the stages gives a scalar activity score for each source patch. It is a designed feature statistic, not Shannon information or a learned estimate of physical sensor reliability.

3. Turn the measurements into preservation weights. For source patches I1,I2I_1,I_2, compute

(ω1,ω2)=softmax⁡(g(I1)c,g(I2)c).(\omega_1,\omega_2)=\operatorname{softmax} \left(\frac{g(I_1)}{c},\frac{g(I_2)}{c}\right).

c>0c>0 is a task-specific scaling constant. The weights are nonnegative and sum to one. A smaller cc makes their difference more decisive. These are source weights for a training patch, not a dense per-pixel segmentation of which sensor is correct. Across many patches, they encourage the generator to preserve the source whose measured feature activity is larger.

4. Use complementary similarity terms. With generator output Fθ(I1,I2)F_\theta(I_1,I_2), define

LSSIM=E[∑s=12ωs(1−SSIM⁡(Fθ,Is))],LMSE=E[∑s=12ωsMSE⁡(Fθ,Is)],Lsim=LSSIM+αLMSE.\begin{aligned} \mathcal L_{\mathrm{SSIM}} &=\mathbb E\left[\sum_{s=1}^{2}\omega_s \bigl(1-\operatorname{SSIM}(F_\theta,I_s)\bigr)\right],\\ \mathcal L_{\mathrm{MSE}} &=\mathbb E\left[\sum_{s=1}^{2}\omega_s \operatorname{MSE}(F_\theta,I_s)\right],\\ \mathcal L_{\mathrm{sim}} &=\mathcal L_{\mathrm{SSIM}}+\alpha\mathcal L_{\mathrm{MSE}}. \end{aligned}

θ\theta denotes generator parameters, the expectation averages training pairs, ss indexes the two sources, SSIM compares structure and luminance statistics [16], MSE is mean squared pixel error, and α\alpha controls their trade-off. The journal model uses α=20\alpha=20 [32]. SSIM encourages structural similarity while MSE constrains intensity differences. Neither term supplies a uniquely correct fused target, and a high weight does not guarantee every feature from that source survives.

Importantly, the journal objective does not retain FusionDN’s gradient-matching loss. The authors remove it to reduce false edges and add MSE to better constrain brightness. Do not implement U2Fusion by combining every loss mentioned in the historical discussion.

5. Learn tasks sequentially with EWC. Ordinary fine-tuning can overwrite useful parameters. EWC [58] penalizes movement of parameters estimated to matter to previous tasks:

L=Lsim+lambdaLEWC,LEWC=12∑iμi(θi−θi∗)2.\mathcal L=\mathcal L_{\mathrm{sim}}+lambda\mathcal L_{\mathrm{EWC}},\qquad \mathcal L_{\mathrm{EWC}}=\frac12\sum_i\mu_i(\theta_i-\theta_i^*)^2.

θi∗\theta_i^* is a parameter value after learning earlier tasks, θi\theta_i its current value, μi≥0\mu_i\ge0 an importance weight, and λ\lambda the consolidation strength. Parameters with larger μi\mu_i are more strongly protected. The paper motivates μi\mu_i using diagonal Fisher information and approximates it with an average squared gradient of the previous task’s loss at θ∗\theta^*. For this general fusion loss, treat it as an importance approximation rather than an independently calibrated probabilistic uncertainty.

Importance is estimated before discarding the previous training data. The method does not need to replay all old training images during the next task, although data are used to estimate the protection weights. With several completed tasks, importance information must be accumulated according to the stated training procedure; retaining only the immediately preceding task’s unweighted parameters would not reproduce the intended consolidation.

The adopted order is multimodal → multi-exposure → multi-focus. Multimodal combines visible–infrared and PET–MRI examples as one training task. The journal setup crops 64×6464\times64 patches, uses batch size 18, learning rate 10−410^{-4}, 3/2/2 epochs for the three tasks, c=3×103,3.5×103,102c=3\times10^3,3.5\times10^3,10^2 respectively, and λ=8×104\lambda=8\times10^4 [32]. These scales depend on the exact feature and intensity conventions; they are not portable defaults for arbitrary backbones.

6. Color and more than two inputs. Color images are converted to YCbCr, and the generator fuses luminance. Chroma is combined by a separate rule that favors stronger departures from neutral chroma. The paper writes, for either chroma channel,

Cf=C1∣C1−t∣+C2∣C2−t∣∣C1−t∣+∣C2−t∣,t=128.C_f=\frac{C_1|C_1-t|+C_2|C_2-t|} {|C_1-t|+|C_2-t|},\qquad t=128.

C1,C2C_1,C_2 are source chroma values in the paper’s 8-bit convention. A practical implementation must handle the zero denominator when both equal the neutral value, for example by returning tt. If only one source has color, its chroma supplies that part of the output. This is not end-to-end learned color fusion.

For a sequence, fuse two inputs, then fuse their intermediate result with the next input. This permits more than two sources but does not make the binary generator associative or order-independent. Errors or preferences can accumulate through the sequence.

What runs at inference. The learned fusion generator and the required color conversions run. VGG feature measurement, adaptive loss weights and EWC optimization are training mechanisms; they are not extra inference networks. This separation is central to understanding the model’s practical cost.

Results

The journal paper evaluates six collections across its tasks: TNO and RoadScene for visible–infrared, Harvard PET–MRI, the multi-exposure collection used in its training study and EMPA HDR, and Lytro multi-focus [32]. It introduces RoadScene with 221 aligned pairs. The quantitative visible–infrared comparisons use 20 TNO pairs and 45 RoadScene pairs; other images are shown qualitatively. Training, test selection and preprocessing therefore matter when comparing these tables with later papers.

Selected journal values illustrate both strengths and limits:

SettingU2Fusion result, as printedInterpretation
TNO, Table 1CC 0.537±0.110.537\pm0.11; SSIM 1.9909±0.0051.9909\pm0.005; PSNR 62.914±2.0762.914\pm2.07Source-comparison metrics under this paper’s definitions
RoadScene, Table 1CC 0.635±0.200.635\pm0.20; SCD 1.635±0.241.635\pm0.24It does not win SCD; FusionDN gives 1.778±0.171.778\pm0.17
Harvard, Table 2CC 0.834±0.080.834\pm0.08; SSIM 1.9921±0.0021.9921\pm0.002Correlation/structural preservation, not diagnostic validation
Lytro, Table 4CC 0.972±0.010.972\pm0.01; VIF 1.466±0.201.466\pm0.20Strong results on a small multi-focus benchmark

The SSIM values near 2 use a fusion-specific aggregation over sources. They are not ordinary single-reference SSIM values bounded above by 1. Likewise, the paper’s PSNR columns compare under its source-based evaluation; there is no hidden true fused image, and these large values should not be compared directly with restoration PSNR against clean ground truth. A reproduction must use the same toolkit and scaling.

The method also does not dominate every task metric: Table 3 gives 58.454 for its EMPA HDR PSNR column, while DeepFuse gives 58.518; Table 4 gives FusionDN a larger VIF than U2Fusion [32]. Report the full metric context instead of selecting only winning columns.

Ablations

EWC. Figures 19–20 compare sequential training with and without consolidation. Without EWC, learning later tasks changes earlier-task brightness and appearance; EWC alleviates this forgetting. The accompanying parameter-importance histograms describe the learned protection weights. These are qualitative and distributional analyses, not a table establishing a particular numerical AP-like gain for EWC.

Shared versus individual models. Figure 21 compares separately trained models with the unified model. The authors show examples where learning exposure handling helps overexposed regions in other tasks, and where multi-focus training helps sharp boundaries in exposure fusion. This supports cross-task transfer in those examples, not a guarantee that every added task improves all previous ones.

Adaptive weights. Figure 22 fixes both preservation weights to 0.5 and compares with the adaptive version. The equal-weight outputs lose details or retain overexposed regions, with blurred boundaries particularly visible in multi-focus examples. The paper provides images for this ablation; it does not provide a complete numerical table from which to invent a universal percentage improvement.

Training order. Table 5 supplies quantitative evidence. The adopted order gives Lytro CC 0.9723 / mean gradient 0.0677. Swapping the first two tasks gives 0.9724 / 0.0700. Starting with multi-focus and finishing with exposure gives 0.9803 / 0.0563 [32]. The last order has higher correlation but weaker mean gradient and visibly softer boundaries. Order still matters despite EWC, and the metrics disagree about what improved.

Journal changes from FusionDN. Figure 24 compares the earlier model with U2Fusion. Feature-based activity measurement replaces the earlier source-level information rule; the gradient loss is removed; MSE is added; and medical examples extend the first task. These changes are not all isolated numerically in a complete factorial study. The comparison supports the revised recipe, while limiting attribution to any single change.

Conclusion

U2Fusion unifies the training objective and the learned parameters across several fusion tasks. Fixed pretrained features define adaptive source preferences; structure and intensity losses train the generator; EWC reduces forgetting during sequential training. Its enduring contribution is to make “one model for several fusion tasks” a concrete optimization problem.

Problems

The feature activity statistic can reward noise or strong artifacts, and its source weights are scalar per training patch. A pretrained natural-image feature extractor is not guaranteed to measure medical or thermal importance correctly. The objective still expresses a designed preference, not a physical fused truth.

Registration is assumed, color is partly handled by a fixed rule, and sequential multi-input fusion is order-dependent. EWC reduces forgetting but does not eliminate it, as the training-order table shows. Small datasets and source-based metrics also limit claims of broad generalization. Later SwinFusion changes feature interaction, while task-aware approaches ask whether the fused image improves a specific downstream task; neither removes the need to check alignment, fidelity and evaluation assumptions.

References

  1. Y. Liu, X. Chen, Z. Wang, Z. J. Wang, R. K. Ward and X. Wang, “Deep learning for pixel-level image fusion: Recent advances and future prospects,” Information Fusion, vol. 42, pp. 158–173, 2018. doi
  2. J. Liu, G. Wu, Z. Liu, D. Wang, Z. Jiang, L. Ma, W. Zhong, X. Fan and R. Liu, “Infrared and visible image fusion: From data compatibility to task adaption,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 47, no. 4, pp. 2349–2369, 2025. doi arXiv
  3. J. Ma, Y. Ma and C. Li, “Infrared and visible image fusion methods and applications: A survey,” Information Fusion, vol. 45, pp. 153–178, 2019. doi
  4. P. E. Debevec and J. Malik, “Recovering high dynamic range radiance maps from photographs,” in Proc. SIGGRAPH, 1997, pp. 369–378. doi
  5. T. Mertens, J. Kautz and F. Van Reeth, “Exposure fusion: A simple and practical alternative to high dynamic range photography,” Computer Graphics Forum, vol. 28, no. 1, pp. 161–171, 2009. doi
  6. S. Xu, X. Wei, C. Zhang, J. Liu and J. Zhang, “MFFW: A new dataset for multi-focus image fusion,” arXiv:2002.04780, 2020. arXiv
  7. X. Zhang, “Multi-focus image fusion: A benchmark,” arXiv:2005.01116, 2020. arXiv
  8. P. J. Burt and E. H. Adelson, “The Laplacian pyramid as a compact image code,” IEEE Transactions on Communications, vol. 31, no. 4, pp. 532–540, 1983. doi
  9. P. J. Burt and R. J. Kolczynski, “Enhanced image capture through fusion,” in Proc. 4th International Conference on Computer Vision (ICCV), 1993, pp. 173–182. doi
  10. P. J. Burt and E. H. Adelson, “A multiresolution spline with application to image mosaics,” ACM Transactions on Graphics, vol. 2, no. 4, pp. 217–236, 1983. doi
  11. H. Li, B. S. Manjunath and S. K. Mitra, “Multisensor image fusion using the wavelet transform,” Graphical Models and Image Processing, vol. 57, no. 3, pp. 235–245, 1995. doi
  12. S. Li, X. Kang and J. Hu, “Image fusion with guided filtering,” IEEE Transactions on Image Processing, vol. 22, no. 7, pp. 2864–2875, 2013. doi
  13. K. He, J. Sun and X. Tang, “Guided image filtering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 6, pp. 1397–1409, 2013. doi
  14. B. Yang and S. Li, “Multifocus image fusion and restoration with sparse representation,” IEEE Transactions on Instrumentation and Measurement, vol. 59, no. 4, pp. 884–892, 2010. doi
  15. M. Nejati, S. Samavi and S. Shirani, “Multi-focus image fusion using dictionary-based sparse representation,” Information Fusion, vol. 25, pp. 72–84, 2015. doi
  16. Z. Wang, A. C. Bovik, H. R. Sheikh and E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. doi
  17. X. Zhang, P. Ye and G. Xiao, “VIFB: A visible and infrared image fusion benchmark,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2020. CVF arXiv
  18. X. Zhang, “Benchmarking and comparing multi-exposure image fusion algorithms,” Information Fusion, vol. 74, pp. 111–131, 2021. doi arXiv
  19. A. M. Eskicioglu and P. S. Fisher, “Image quality measures and their performance,” IEEE Transactions on Communications, vol. 43, no. 12, pp. 2959–2965, 1995. doi
  20. G. Qu, D. Zhang and P. Yan, “Information measure for performance of image fusion,” Electronics Letters, vol. 38, no. 7, pp. 313–315, 2002. doi
  21. M. Hossny, S. Nahavandi and D. Creighton, “Comments on ‘Information measure for performance of image fusion’,” Electronics Letters, vol. 44, no. 18, pp. 1066–1067, 2008. doi
  22. C. S. Xydeas and V. Petrović, “Objective image fusion performance measure,” Electronics Letters, vol. 36, no. 4, pp. 308–309, 2000. doi
  23. G. Piella and H. Heijmans, “A new quality metric for image fusion,” in Proc. IEEE International Conference on Image Processing (ICIP), 2003, vol. 3, pp. III-173–176. doi
  24. Y. Han, Y. Cai, Y. Cao and X. Xu, “A new image fusion performance metric based on visual information fidelity,” Information Fusion, vol. 14, no. 2, pp. 127–135, 2013. doi
  25. K. Ma, K. Zeng and Z. Wang, “Perceptual quality assessment for multi-exposure image fusion,” IEEE Transactions on Image Processing, vol. 24, no. 11, pp. 3345–3356, 2015. doi
  26. Z. Liu, E. Blasch, Z. Xue, J. Zhao, R. Laganiere and W. Wu, “Objective assessment of multiresolution image fusion algorithms for context enhancement in night vision: A comparative study,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 1, pp. 94–109, 2012. doi
  27. J. Liu, X. Fan, Z. Huang, G. Wu, R. Liu, W. Zhong and Z. Luo, “Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 5802–5811. CVF arXiv M3FD
  28. J. Liu, Z. Liu, G. Wu, L. Ma, R. Liu, W. Zhong, Z. Luo and X. Fan, “Multi-interactive feature learning and a full-time multi-modality benchmark for image fusion and segmentation,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 8115–8124. CVF arXiv
  29. Y. Zhang, Y. Liu, P. Sun, H. Yan, X. Zhao and L. Zhang, “IFCNN: A general image fusion framework based on convolutional neural network,” Information Fusion, vol. 54, pp. 99–118, 2020. doi
  30. H. Li and X.-J. Wu, “DenseFuse: A fusion approach to infrared and visible images,” IEEE Transactions on Image Processing, vol. 28, no. 5, pp. 2614–2623, 2019. doi arXiv
  31. K. R. Prabhakar, V. S. Srikar and R. V. Babu, “DeepFuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs,” in Proc. IEEE International Conference on Computer Vision (ICCV), 2017, pp. 4724–4732. doi CVF
  32. H. Xu, J. Ma, J. Jiang, X. Guo and H. Ling, “U2Fusion: A unified unsupervised image fusion network,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 1, pp. 502–518, 2022. doi RoadScene
  33. L. Tang, J. Yuan and J. Ma, “Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network,” Information Fusion, vol. 82, pp. 28–42, 2022. doi
  34. J. Ma, L. Tang, F. Fan, J. Huang, X. Mei and Y. Ma, “SwinFusion: Cross-domain long-range learning for general image fusion via Swin Transformer,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 7, pp. 1200–1217, 2022. doi
  35. X. Zhang and Y. Demiris, “Visible and infrared image fusion using deep learning,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 45, no. 8, pp. 10535–10554, 2023. doi
  36. Y. Liu, X. Chen, H. Peng and Z. Wang, “Multi-focus image fusion with a deep convolutional neural network,” Information Fusion, vol. 36, pp. 191–207, 2017. doi
  37. K. Ma, Z. Duanmu, H. Zhu, Y. Fang and Z. Wang, “Deep guided learning for fast multi-exposure image fusion,” IEEE Transactions on Image Processing, vol. 29, pp. 2808–2819, 2020. doi
  38. Z. Zhao, H. Bai, J. Zhang, Y. Zhang, S. Xu, Z. Lin, R. Timofte and L. Van Gool, “CDDFuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pp. 5906–5916. CVF arXiv
  39. J. Ma, W. Yu, P. Liang, C. Li and J. Jiang, “FusionGAN: A generative adversarial network for infrared and visible image fusion,” Information Fusion, vol. 48, pp. 11–26, 2019. doi · public article transcription
  40. H. Bai, J. Zhang, Z. Zhao, Y. Wu, L. Deng, Y. Cui, T. Feng and S. Xu, “Task-driven image fusion with learnable fusion loss,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025, pp. 7457–7468. CVF arXiv
  41. Z. Zhao, H. Bai, Y. Zhu, J. Zhang, S. Xu, Y. Zhang, K. Zhang, D. Meng, R. Timofte and L. Van Gool, “DDFM: Denoising diffusion model for multi-modality image fusion,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV), 2023, pp. 8082–8093. CVF arXiv
  42. X. Yi, H. Xu, H. Zhang, L. Tang and J. Ma, “Text-IF: Leveraging semantic text guidance for degradation-aware and interactive image fusion,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 27026–27035. CVF arXiv
  43. D. Wang, J. Liu, X. Fan and R. Liu, “Unsupervised misaligned infrared and visible image fusion via cross-modality image generation and registration,” in Proc. International Joint Conference on Artificial Intelligence (IJCAI), 2022, pp. 3508–3515. doi arXiv
  44. L. Tang, Y. Deng, Y. Ma, J. Huang and J. Ma, “SuperFusion: A versatile image registration and fusion network with semantic awareness,” IEEE/CAA Journal of Automatica Sinica, vol. 9, no. 12, pp. 2121–2137, 2022. doi
  45. A. Toet, “The TNO multiband image data collection,” Data in Brief, vol. 15, pp. 249–251, 2017. doi
  46. L. Tang, J. Yuan, H. Zhang, X. Jiang and J. Ma, “PIAFusion: A progressive infrared and visible image fusion network based on illumination aware,” Information Fusion, vol. 83–84, pp. 79–92, 2022. doi MSRS
  47. Q. Ha, K. Watanabe, T. Karasawa, Y. Ushiku and T. Harada, “MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes,” in Proc. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2017, pp. 5108–5115. doi
  48. X. Jia, C. Zhu, M. Li, W. Tang and W. Zhou, “LLVIP: A visible-infrared paired dataset for low-light vision,” in Proc. IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, 2021, pp. 3496–3504. CVF arXiv
  49. J. Cai, S. Gu and L. Zhang, “Learning a deep single image contrast enhancer from multi-exposure images,” IEEE Transactions on Image Processing, vol. 27, no. 4, pp. 2049–2062, 2018. doi
  50. X. Zhang, “Deep learning-based multi-focus image fusion: A survey and a comparative study,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 9, 2022. doi
  51. F. Xu, J. Liu, Y. Song, H. Sun and X. Wang, “Multi-exposure image fusion techniques: A comprehensive review,” Remote Sensing, vol. 14, no. 3, art. 771, 2022. doi
  52. Z. Zhao, H. Bai, J. Zhang, Y. Zhang, K. Zhang, S. Xu, D. Chen, R. Timofte and L. Van Gool, “Equivariant multi-modality image fusion,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, pp. 25912–25921. CVF arXiv
  53. L. Tang, Y. Wang, M. Gong, Z. Li, Y. Deng, X. Yi, C. Li, H. Xu, H. Zhang and J. Ma, “VideoFusion: A spatio-temporal collaborative network for multi-modal video fusion,” in Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026, pp. 19559–19569. CVF
  54. N. Yokoya, C. Grohnfeldt and J. Chanussot, “Hyperspectral and multispectral data fusion: A comparative review of the recent literature,” IEEE Geoscience and Remote Sensing Magazine, 2017. doi
  55. H. R. Sheikh and A. C. Bovik, “Image information and visual quality,” IEEE Transactions on Image Processing, 2006. doi
  56. H. Xu et al., “U2Fusion,” official implementation accompanying the TPAMI paper, 2022. code
  57. K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” ICLR, 2015. arXiv
  58. J. Kirkpatrick et al., “Overcoming catastrophic forgetting in neural networks,” Proceedings of the National Academy of Sciences, 2017. doi
  59. Y. Zhang et al., “IFCNN,” official implementation and errata accompanying the Information Fusion paper, 2020. code
  60. K. He, X. Zhang, S. Ren and J. Sun, “Deep residual learning for image recognition,” CVPR, 2016. CVF
  61. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li and L. Fei-Fei, “ImageNet: A large-scale hierarchical image database,” CVPR, 2009. doi
  62. N. Silberman, D. Hoiem, P. Kohli and R. Fergus, “Indoor segmentation and support inference from RGBD images,” ECCV, 2012. project
  63. P. Dhariwal and A. Nichol, “Diffusion models beat GANs on image synthesis,” NeurIPS, 2021. arXiv
  64. J. Liu et al., “TarDAL,” official implementation accompanying the CVPR paper, 2022. code
  65. G. Jocher et al., “YOLOv5,” Ultralytics object-detection software, 2020 onward. code
  66. J. Ma et al., “FusionGAN,” official implementation accompanying the Information Fusion paper, 2019. code
  67. L. Tang, J. Yuan and J. Ma, “SeAFusion,” official implementation accompanying the Information Fusion paper, 2022. code
  68. T. Mertens, J. Kautz and F. Van Reeth, “Exposure Fusion,” in Proc. Pacific Graphics, 2007. paper · author project