There are essentially three ways to use the with statement:
Use an existing context manager:
with manager:
pass
Create a contex
The difference between this case and similar-appearing cases like super is that here there is no enclosing frame to look at. A with statement is not a new scope. sys._getframe(0) (or, if you're putting the code into a function, sys._getframe(1)) will work just fine, but it'll return you the exact same frame you have before and after the with statement.
The only way you could do it would be by inspecting the bytecode. But even that won't help. For example, try this:
from contextlib import contextmanager
@contextmanager
def silly():
yield
with silly():
fr = sys._getframe(0)
dis.dis(fr.f_code)
Obviously, as SETUP_WITH explains, the method does get looked up and pushed onto the stack for WITH_CLEANUP to use later. So, even after POP_TOP removes the return value of silly(), its __exit__ is still on the stack.
But there's no way to get at that from Python. Unless you want to start munging the bytecode, or digging apart the stack with ctypes or something, it might as well not exist.