Split string based on regex but keep delimiters

Question

I'm trying to split a string using a variety of characters as delimiters and also keep those delimiters in their own array index. For example say I want to split the string:

if (x>1) return x * fact(x-1);

using '(', '>', ')', '*', '-', ';' and '\s' as delimiters. I want the output to be the following string array: {"if", "(", "x", ">", "1", ")", "return", "x", "*", "fact", "(", "x", "-", "1", ")", ";"}

The regex I'm using so far is split("(?=(\\w+(?=[\\s\\+\\-\\*/<(<=)>(>=)(==)(!=)=;,\\.\"\\(\\)\\[\\]\\{\\}])))")

which splits at each word character regardless of whether it is followed by one of the delimiters. For example

test + 1

outputs {"t","e","s","t+","1"} instead of {"test+", "1"}

Why does it split at each character even if that character is not followed by one of my delimiters? Also is a regex which does this even possible in Java? Thank you

Answer 1

Well, you can use lookaround to split at points between characters without consuming the delimiters:

(?<=[()>*-;\s])|(?=[()>*-;\s])

This will create a split point before and after each delimiter character. You might need to remove superfluous whitespace elements from the resulting array, though.

Quick PowerShell test (| marks the split points):

PS Home:\> 'if (x>1) return x * fact(x-1);' -split '(?<=[()>*-;\s])|(?=[()>*-;\s])' -join '|'
if| |(|x|>|1|)| |return| |x| |*| |fact|(|x|-|1|)|;|

Answer 2

How about this pattern?

(\w+)|([\p{P}\p{S}])

Answer 3

To answer your question, "Why?", it's because your entire expression is a lookahead assertion. As long as that assertion is true at each character (or maybe I should say "between"), it is able to split.

Also, you cannot group within character classes, e.g. (<=) is not doing what you think it is doing.