Color similarity/distance in RGBA color space

Question

How to compute similarity between two colors in RGBA color space? (where the background color is unknown of course)

I need to remap an RGBA image to

Answer 1

Finally, I've found it! After thorough testing and experimentation my conclusions are:

• The correct way is to calculate maximum possible difference between the two colors.
  Formulas with any kind of estimated average/typical difference had room for discontinuities.

• I was unable to find a working formula that calculates the distance without blending RGBA colors with some backgrounds.

• There is no need to take every possible background color into account. It can be simplified down to blending maximum and minimum separately for each of R/G/B channels:

  1. blend the channel in both colors with channel=0 as the background, measure squared difference
  2. blend the channel in both colors with channel=max as the background, measure squared difference
  3. take higher of the two.

Fortunately blending with "white" and "black" is trivial when you use premultiplied alpha (r = r×a).

The complete formula is:

max((r₁-r₂)², (r₁-r₂ - a₁+a₂)²) +
max((g₁-g₂)², (g₁-g₂ - a₁+a₂)²) +
max((b₁-b₂)², (b₁-b₂ - a₁+a₂)²)

C Source including SSE2 implementation.