What is the correct syntax for finding a substring (a string which is preceded and followed by specific strings) which does not match a specific pattern?
There is no general negation operator in sed, IIRC because compilation of regexes with negation to DFAs takes exponential time. You can work around this with
'/BEGIN_FOO_END/b; s/BEGIN_\(.*\)_END/(\1)/g'
where /BEGIN_FOO_END/b means: if we find BEGIN_FOO_END, then branch (jump) to the end of the sed script.