Under Win32, it is a common technique to generate a monochrome bitmask from a bitmap for transparency use by doing the following:
SetBkColor(hdcSource, clrTr
You can't do it if there is an alpha channel. COLORREF's use the top 8 bits for a number of purposes, including specifying wether or not the lower 3 bytes are a color table index into the current palette, or a RGB triplet. As such you can't specify anything except 0x00 in the upper byte of clrTransparency.
If you have an alpha bitmap then, to GDI that remains "unaware" of the alpha channel, theres no sane way to actually compare a 24bit BkColor with 32bit pixels in the bitmap.
I would expect GDI to treat the alpha channel in 32bpp bitmaps as "Reserved", and only successfully compare pixels where the reserved channel is zero. i.e. your mask color must be fully transparent anyway to have a chance of succeeding. (and, if youve made a legitimate premultiplied bitmap, that implies the RGV values would be zero too, rather constraining your choice of mask colors :P)