In two strings:
\"Mary Had a Little Lamb\"
\"Mary Had a Big Lamb\"
should return
\"Mary Had a \"
Apache Commons to the rescue!
org.apache.commons.lang3.StringUtils.getCommonPrefix
... and compare the source code with the ingenious/valiant effort of dyross here (currently with highest vote). But her/his code, good though it is, only handles two Strings. This can handle any number.
Apart from not re-inventing the wheel, I can think of two reasons why using Apache Commons is always best for this sort of thing.
If the whole of a given Apache Commons module is really too much for your context (they're usually just a few kB, but OK) you can extract the bits you need from the source code (assuming this complies with the licence). In this case indexOfDifference is an essential function...