The Modern view in more detail
Optional · paper guide 1 of 1 for IP 06 · Color Image Processing. Citation numbers refer to the reference list at the bottom of this page.
Navigate: this page and this track: Navigate: this page and this track
Choose a reading by the ambiguity it addresses: color constancy, vector filtering, color mapping, or automatic colorization.

Color constancy: from assumptions to learning
Gijsenij, Gevers and van de Weijer [14] survey computational color constancy and sort methods into static methods with fixed assumptions (gray world, white patch, gray edge and the Minkowski family above), gamut-based methods that compare the observed colors with those seen under a canonical light, and learning-based methods trained on images with known illuminants. They also propose comparison criteria and benchmark public algorithms on public datasets; a recurring finding is that no single fixed assumption suits every scene. Learning took over afterwards. Fast Fourier Color Constancy (FFCC) [15] turns illuminant estimation into finding a peak in a log-chroma histogram, solved in the frequency domain; it returns a full distribution over illuminants and runs in real time on phones, which allows temporally smooth auto white balance in video. FC4 [16] is a fully convolutional network in which every patch votes for the illuminant with a learned confidence, so a plain wall counts less than a face or a white object. Afifi and Brown [17] correct white balance after the camera has already rendered a nonlinear sRGB image, which the diagonal model cannot do: their network re-renders the image under other white-balance settings. The diagonal model is still the backbone; the estimate and the corrections in rendered space are now learned.
Vector filtering for color
Lukac et al. [12] review filters that treat color pixels as vectors: vector median and vector directional filters, their variants, vector edge detectors, and demosaicking. Their takeaway matches this chapter: componentwise processing ignores inter-channel correlation and can create color artifacts, while vector order statistics keep the input’s colors. Learned denoisers are now stronger when training data and compute are available, but these filters remain standard baselines where speed and predictability matter.
Colormap design
Kovesi [7] gives a design method and test images for linear, diverging, rainbow and cyclic colormaps, built on one principle: perceived lightness must change uniformly along the map. The viridis family [8] became Matplotlib’s default in version 2.0. Crameri et al. [9] review how color is used across scientific publishing and argue that non-uniform and red–green maps should disappear from software defaults. Deep learning changed nothing here: the problem is human perception, and the answer is careful design measured in a perceptual color space.
Colorization with deep learning
Pseudocolor maps gray to color by a fixed rule; colorization tries to predict the true colors. Zhang, Isola and Efros [18] treat it as classification over quantized colors and rebalance rare classes so that vivid colors are not averaged into brown; they evaluate with a “colorization Turing test” in which people judge whether an image is real. Anwar et al. [19] survey deep colorization, group methods into seven categories, discuss losses and metrics, and introduce a new benchmark dataset. DDColor [20] (ICCV 2023) uses two decoders, one restoring spatial detail and one refining color with cross-attention to multi-scale semantic features, plus a colorfulness loss; this reduces color bleeding across object boundaries. Two classical ideas persist: work in L*a*b* (predict chroma, keep the given lightness) and treat color as a distribution, because a gray shirt could have been any color.
Reading these papers critically
Distinguish preserving measured color from choosing a display encoding and from inventing missing chromatic information. Color constancy estimates how illumination affects an observation. Vector filtering treats channels jointly when removing corruption. A colormap communicates a scalar field to a viewer. Colorization predicts plausible colors from incomplete information. Good performance in one task is not evidence that the same method solves the others.
Read color comparisons with the working color space, transfer function, white point and viewing assumptions specified. A perceptually smooth colormap can improve interpretability without improving a sensor measurement, while a plausible colorization can disagree with the unknown original color. For constancy, the illuminant-error metric and camera/data split matter; for filtering, preserve edge behavior as well as aggregate error. The next useful question is which information is observed, which is standardized for display and which is supplied by a learned prior.
References
- R. C. Gonzalez and R. E. Woods, Digital Image Processing, 4th ed., Pearson, 2018, Ch. 6. publisher page
- C. Wyman, P.-P. Sloan, and P. Shirley, “Simple Analytic Approximations to the CIE XYZ Color Matching Functions,” Journal of Computer Graphics Techniques, vol. 2, no. 2, 2013. jcgt.org
- IEC 61966-2-1:1999, “Multimedia systems and equipment – Colour measurement and management – Part 2-1: Colour management – Default RGB colour space – sRGB,” IEC, 1999. IEC webstore
- Recommendation ITU-R BT.709-6, “Parameter values for the HDTV standards for production and international programme exchange,” ITU, 2015. ITU
- ISO/CIE 11664-4:2019, “Colorimetry – Part 4: CIE 1976 L*a*b* colour space,” ISO/CIE, 2019. standard page
- Recommendation ITU-R BT.601-7, “Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios,” ITU, 2011. ITU
- P. Kovesi, “Good Colour Maps: How to Design Them,” arXiv:1509.03700, 2015. arXiv
- S. van der Walt and N. Smith, viridis colormap design notes (BIDS project page; presented at SciPy 2015), 2015. project page
- F. Crameri, G. E. Shephard, and P. J. Heron, “The misuse of colour in science communication,” Nature Communications, vol. 11, art. 5444, 2020. PMC
- G. Buchsbaum, “A spatial processor model for object colour perception,” Journal of the Franklin Institute, vol. 310, no. 1, pp. 1–26, 1980. DOI
- J. van de Weijer, T. Gevers, and A. Gijsenij, “Edge-Based Color Constancy,” IEEE Transactions on Image Processing, vol. 16, no. 9, pp. 2207–2214, 2007. DOI
- R. Lukac, B. Smolka, K. Martin, K. N. Plataniotis, and A. N. Venetsanopoulos, “Vector Filtering for Color Imaging,” IEEE Signal Processing Magazine, vol. 22, no. 1, pp. 74–86, Jan. 2005. PDF
- S. Di Zenzo, “A note on the gradient of a multi-image,” Computer Vision, Graphics, and Image Processing, vol. 33, pp. 116–125, 1986. DOI
- A. Gijsenij, T. Gevers, and J. van de Weijer, “Computational Color Constancy: Survey and Experiments,” IEEE Transactions on Image Processing, vol. 20, no. 9, pp. 2475–2489, 2011. DOI
- J. T. Barron and Y.-T. Tsai, “Fast Fourier Color Constancy,” CVPR, 2017. arXiv
- Y. Hu, B. Wang, and S. Lin, “FC4: Fully Convolutional Color Constancy with Confidence-Weighted Pooling,” CVPR, 2017. paper
- M. Afifi and M. S. Brown, “Deep White-Balance Editing,” CVPR, 2020. arXiv
- R. Zhang, P. Isola, and A. A. Efros, “Colorful Image Colorization,” arXiv:1603.08511, 2016. arXiv
- S. Anwar, M. Tahir, C. Li, A. Mian, F. S. Khan, and A. W. Muzaffar, “Image Colorization: A Survey and Dataset,” arXiv:2008.10774, 2020. arXiv
- X. Kang, T. Yang, W. Ouyang, P. Ren, L. Li, and X. Xie, “DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders,” ICCV, 2023. arXiv