C parsing input text file into words

Question

I am trying to parse input file (containing a text document with multiple lines and delimiters, i.e. "!,.?") into words. My function 'splitting function' is:

int splitInput(fp) {

    int i= 0;
    char  line[255];
    char *array[5000];
    int x;
    while (fgets(line, sizeof(line), fp) != NULL) {     
        array[i] = strtok(line, ",.!? \n");
        printf("Check print - word %i:%s:\n",i, array[i]);
        i++;
    }
    return 0;
}

Answer 1

Here's the corrected function [sorry for extra the style cleanup]:

int
splitInput(fp)
{
    int i = 0;
    char *cp;
    char *bp;
    char line[255];
    char *array[5000];
    int x;

    while (fgets(line, sizeof(line), fp) != NULL) {
        bp = line;
        while (1) {
            cp = strtok(bp, ",.!? \n");
            bp = NULL;

            if (cp == NULL)
                break;
            array[i++] = cp;

            printf("Check print - word %i:%s:\n",i-1, cp);
        }
    }

    return 0;
}

Now, take a look at the man page for strtok to understand the bp trick

Answer 2

If I understand your question correctly you want to read every line and split each line into words and add that into an array.

    array[i] = strtok(line, ",.!? \n");

That will not work for obvious reasons because it will only return the first word for each line and you never allocate memory.

This is probably what you want.

    char *pch;
    pch = strtok(line, ",.!? \n");
    while(pch != NULL) {
      array[i++] = strdup(pch); // put the content of pch into array at position i and increment i afterwards.
      pch = strtok(NULL, ",.!? \n"); // look for remaining words at the same line
    }

Don't forget to free your array elements afterwards though using free.