I allow users to enter a regular expression to match IP addresses, for doing an IP filtration in a related system. I would like to validate if the entered regular expression
A malformed regex isn't the worst of reasons for an exception.
Unless you resign to a very limited subset of regex syntax - and then write a regex (or a parser) for that - I think you have no other way of testing if it is valid but to try to build a state machine from it and make it match something.