gcc, strict-aliasing, and casting through a union

Question

Do you have any horror stories to tell? The GCC Manual recently added a warning regarding -fstrict-aliasing and casting a pointer through a union:

[.

Answer 1

Well it's a bit of necro-posting, but here is a horror story. I'm porting a program that was written with the assumption that the native byte order is big endian. Now I need it to work on little endian too. Unfortunately, I can't just use native byte order everywhere, as data could be accessed in many ways. For example, a 64-bit integer could be treated as two 32-bit integers or as 4 16-bit integers, or even as 16 4-bit integers. To make things worse, there is no way to figure out what exactly is stored in memory, because the software is an interpreter for some sort of byte code, and the data is formed by that byte code. For example, the byte code may contain instructions to write an array of 16-bit integers, and then access a pair of them as a 32-bit float. And there is no way to predict it or alter the byte code.

Therefore, I had to create a set of wrapper classes to work with values stored in the big endian order regardless of the native endianness. Worked perfectly in Visual Studio and in GCC on Linux with no optimizations. But with gcc -O2, hell broke loose. After a lot of debugging I figured out that the reason was here:

double D;
float F; 
Ul *pF=(Ul*)&F; // Ul is unsigned long
*pF=pop0->lu.r(); // r() returns Ul
D=(double)F; 

This code was used to convert a 32-bit representation of a float stored in a 32-bit integer to double. It seems that the compiler decided to do the assignment to *pF after the assignment to D - the result was that the first time the code was executed, the value of D was garbage, and the consequent values were "late" by 1 iteration.

Miraculously, there were no other problems at that point. So I decided to move on and test my new code on the original platform, HP-UX on a RISC processor with native big endian order. Now it broke again, this time in my new class:

typedef unsigned long long Ur; // 64-bit uint
typedef unsigned char Uc;
class BEDoubleRef {
        double *p;
public:
        inline BEDoubleRef(double *p): p(p) {}
        inline operator double() {
                Uc *pu = reinterpret_cast(p);
                Ur n = (pu[7] & 0xFFULL) | ((pu[6] & 0xFFULL) << 8)
                        | ((pu[5] & 0xFFULL) << 16) | ((pu[4] & 0xFFULL) << 24)
                        | ((pu[3] & 0xFFULL) << 32) | ((pu[2] & 0xFFULL) << 40)
                        | ((pu[1] & 0xFFULL) << 48) | ((pu[0] & 0xFFULL) << 56);
                return *reinterpret_cast(&n);
        }
        inline BEDoubleRef &operator=(const double &d) {
                Uc *pc = reinterpret_cast(p);
                const Ur *pu = reinterpret_cast(&d);
                pc[0] = (*pu >> 56) & 0xFFu;
                pc[1] = (*pu >> 48) & 0xFFu;
                pc[2] = (*pu >> 40) & 0xFFu;
                pc[3] = (*pu >> 32) & 0xFFu;
                pc[4] = (*pu >> 24) & 0xFFu;
                pc[5] = (*pu >> 16) & 0xFFu;
                pc[6] = (*pu >> 8) & 0xFFu;
                pc[7] = *pu & 0xFFu;
                return *this;
        }
        inline BEDoubleRef &operator=(const BEDoubleRef &d) {
                *p = *d.p;
                return *this;
        }
};

For some really weird reason, the first assignment operator only correctly assigned bytes 1 through 7. Byte 0 always had some nonsense in it, which broke everything as there is a sign bit and a part of order.

I have tried to use unions as a workaround:

union {
    double d;
    Uc c[8];
} un;
Uc *pc = un.c;
const Ur *pu = reinterpret_cast(&d);
pc[0] = (*pu >> 56) & 0xFFu;
pc[1] = (*pu >> 48) & 0xFFu;
pc[2] = (*pu >> 40) & 0xFFu;
pc[3] = (*pu >> 32) & 0xFFu;
pc[4] = (*pu >> 24) & 0xFFu;
pc[5] = (*pu >> 16) & 0xFFu;
pc[6] = (*pu >> 8) & 0xFFu;
pc[7] = *pu & 0xFFu;
*p = un.d;

but it didn't work either. In fact, it was a bit better - it only failed for negative numbers.

At this point I'm thinking about adding a simple test for native endianness, then doing everything via char* pointers with if (LITTLE_ENDIAN) checks around. To make things worse, the program makes heavy use of unions all around, which seems to work ok for now, but after all this mess I won't be surprised if it suddenly breaks for no apparent reason.