strtok() and empty fields

Question

I am serializing some C structure to string and than deserializing it with strtok(). But, unfortunately, strtok() don't detect empty fields (eg 1:2::4).

Is there any alternative function?

Answer 1

On linux there's strsep.

The strsep() function was introduced as a replacement for strtok(), since the latter cannot handle empty fields. However, strtok() conforms to C89/C99 and hence is more portable.

Answer 2

You can use strchr (for just one delimiter character) or strcspn (for a group of possible delimiters) to find the next delimiter, process the token, and then just step one character forward. Do this in a loop and you have what you need.

Answer 3

Drakosha gave the correct answer. I want to add an example for both variants.

With strtok:

char *token;
char *tmp_string;
char delimiter[10] = " |,.:";
strcpy (tmp_string, "1:2::4");
token = strtok(tmp_string, delimiter); // first token
while(token != NULL) {
    i++;
    printf ("i=%d\tToken: len(%d)\t%s", i, strlen(token), token);
    // do something
    token = strtok(NULL, delimiter); /* next token */
}

With strsep (will recognize ""):

char *token;
char *tmp_string;
char delimiter[10] = " |,.";
strcpy (tmp_string, "1:2::4");
token = strsep(&tmp_string, delimiter); // first token
while(token != NULL) {
    i++;
    printf ("i=%d\tToken: len(%d)\t%s", i, strlen(token), token);
    // do something
    token = strsep(&tmp_string, delimiter); /* next token */
}