In PHP I have the following string :
$str = \"AAA, BBB, (CCC,DDD), \'EEE\', \'FFF,GGG\', (\'HHH\',\'III\'), ((\'JJJ\',\'KKK\'), LLL, (MMM,NNN)) , OOO\";
A spartan regex that tokenizes and also validates all the tokens that it extracts:
\G\s*+((\((?:\s*+(?2)\s*+(?(?!\)),)|\s*+[^()',\s]++\s*+(?(?!\)),)|\s*+'[^'\r\n]*+'\s*+(?(?!\)),))++\))|[^()',\s]++|'[^'\r\n]*+')\s*+(?:,|$)
Regex101
Put it in string literal, with delimiter:
'/\G\s*+((\((?:\s*+(?2)\s*+(?(?!\)),)|\s*+[^()\',\s]++\s*+(?(?!\)),)|\s*+\'[^\'\r\n]*+\'\s*+(?(?!\)),))++\))|[^()\',\s]++|\'[^\'\r\n]*+\')\s*+(?:,|$)/'
ideone
The result is in capturing group 1. In the example on ideone, I specify PREG_OFFSET_CAPTURE flag, so that you can check against the last match in group 0 (entire match) whether the entire source string has been consumed or not.
\s. Consequently, it may not span multiple lines.(, ), ' or ,.'.,( and ends with ).() is not allowed.,. Single trailing comma , is considered valid.\s, which includes new line character) are arbitrarily allowed between token(s), comma(s) , separating tokens, and the bracket(s) (, ) of the bracket tokens.
\G\s*+
(
(
\(
(?:
\s*+
(?2)
\s*+
(?(?!\)),)
|
\s*+
[^()',\s]++
\s*+
(?(?!\)),)
|
\s*+
'[^'\r\n]*+'
\s*+
(?(?!\)),)
)++
\)
)
|
[^()',\s]++
|
'[^'\r\n]*+'
)
\s*+(?:,|$)