Q:

Reconstructing an Index in C

I have a program that reads in a giant text file of lines in this format, and I need to construct a data structure from this text file. 
microfinance 5 41 5 1650 2 1667 1 1811 1 1988 5 
subminiature 1 432 1 

The number 1st number after the word is the number of documents in which the word is found. The following numbers alternate between the document ID number and the amount of occurrences of the word found in the document. So for microfinance, there are 5 documents, the first one is document 41 with 5 occurences, next is doc 1650 with 2, etc.
I use strtok to get each element and organize them. I know strtok is working fine. The problem is attaching the elements correctly to my data structures.
DocumentNode *myDoc;
while (fgets(theLine, sizeof(theLine), newPointer) != NULL)
    {
        counter = 0;
        pch = strtok (theLine," ");
        while (pch != NULL)
        {
         if (0 == counter)
         {
            WordNode *toInsertPtr = (malloc(sizeof(struct WordNode)));
            word = (malloc(100));
            strncpy (word, pch, strlen(pch));
            toInsertPtr->word = word;
            toInsertPtr->next = NULL;

            currIndex = JenkinsHash(word, MAX_HASH_SLOT);
            if ((TheIndex->index[currIndex]) == NULL)
            {
                TheIndex->index[currIndex] = toInsertPtr;
            }
            else 
            {
                TheIndex->index[currIndex]->next = toInsertPtr;
            }   
         }

         if (1 == counter)
         {
            numOfDocs = atoi(pch);
         }

         if (counter % 2 == 0 && counter != 0 && pch != NULL)
         {
            myDoc= (malloc(sizeof(struct DocumentNode)));
            myDoc->next = NULL;
            int doc_id = atoi(pch);
            myDoc->documentID = doc_id;         
         }

         if (counter % 2 != 0 && counter != 1 && pch != NULL)
         {
            myDoc->occurences = atoi(pch);

            if (TheIndex->index[currIndex]->page == NULL)
            {
                TheIndex->index[currIndex]->page = myDoc;
            }
            else
            {
                TheIndex->index[currIndex]->page->next = myDoc;
            }
         }
          pch = strtok (NULL, " ");
          counter++;
        }
    } 

I've GDBed to find out that the problem is here. The first if statement checking to see if there is a doc node at the index is always catching as null (even when there is clearly something at that point in the index) and it's overwriting one slot again and again. Why would it always believe it's NULL when it's not?
        if (TheIndex->index[currIndex]->page == NULL)
        {
            TheIndex->index[currIndex]->page = myDoc;
        }
        else
        {
            TheIndex->index[currIndex]->page->next = myDoc;
        }

The data structures are as follows:
typedef struct DocumentNode {
    struct DocumentNode *next;      // pointer to next member of the list.
    int documentID;                 //doc identifier (filename, ie. 1, 2, etc.)
    int occurences;                 //num. occurances. 
} DocumentNode;

typedef struct WordNode {                
    struct WordNode *next;           //pointer to the next word (for collisions)
    char *word;                      //the word itself.
    DocumentNode *page;              // pointer to the first element of the page list.
} WordNode;

typedef struct InvertedIndex {
    WordNode *index[MAX_HASH_SLOT];   
} InvertedIndex;

A:

Your approach is too complex: the loop is trying to maintain the state, and has a long chain of conditions to decide what to do with the next token.
Rather than doing your strtoks one at a time, do the first one to get the word, the second one to get the count, and then do the rest of them in pairs. It should go as follows:
while (fgets(theLine, sizeof(theLine), newPointer) != NULL) {
    pch = strtok (theLine," ");
    char *word = malloc(strlen(pch)+1);
    strcpy(word, pch);
    ... // Add the word
    pch = strtok(NULL, " ");
    int pairCount = atoi(pch);
    for (int i = 0 ; i != pairCount ; i++) {
         pch = strtok(NULL, " ");
         int id = atoi(pch);
         pch = strtok(NULL, " ");
         int count = atoi(pch);
         ... // Add the document
    }
}

P.S. If you understood this approach well, you would probably enjoy this tale by Edsger Dijkstra.

