One can simply use
INSERT INTO MY_TABLE(MY_TIMESTAMP_FIELD)
VALUES (TIMESTAMP '2019-02-15 13:22:11.871+02:00');
This way you won't have to worry about date format string, just use default timestamp format.
Works with Oracle 11, have no idea if it does for earlier Oracle versions.