With integral types, | is a bitwise or, ^ a bitwise xor and for completeness & is a bitwise and.
With boolean types, | is a boolean or, ^ a boolean xor and & a boolean &.
In comparison, || is a short-circuit boolean or - if the first operand evaluates as true the second operand isn't evaluated. && is a short-circuit boolean and - if the first operand is false, the second isn't evaluated. There is no short-circuit ^ because there is no case where the second need not be evaluated.
|| and && are more often used than | and & in boolean cases as there is normally at least a tiny efficiency gain and never a loss. However if the right-hand operand had a side-effect that it was important to trigger in all cases, then | or & would be the one to use. In practice this is rare, and a bad smell (if the side-effect is important, it should be evaluated in a separate expression to make the purpose clearer).
Edit: A source of potential confusion, is that in some other languages integral types can be used as booleans (e.g. you can do if(53) and it's the same as if(true)) this makes the distinctions between the above operators quite different: They're the same if a "purely" boolean type is used (that has only true and false as its possible values) but not otherwise. C# deliberately doesn't allow boolean operations on integral types precisely to prevent the possibilities for mistakes that exist in such languages.
