Implement strtok function in C

In the blog post, you will learn how to create or implement strtok function in C. I have already written a detailed article on strtok and its uses if you can read this blog post if you have not used strtok previously. The purpose of this post is to only describe that how you can create your own version of the strtok function in C. But before creating your own version of strtok let’s see the introduction of the strtok function in C.

What is strtok in C?

A sequence of calls to the strtok function breaks the string pointed to by s1 into a sequence of tokens, each of which is delimited by a character from the string pointed to by s2. The first call in the sequence has a non-null first argument; subsequent calls in the sequence have a null first argument. If any of the subsequent calls in the sequence is made by a different thread than the first, the behavior is undefined. The separator string pointed to by s2 may be different from call to call.

Syntax of strtok function in C:

//General syntax of strtok function in C

char *strtok(char * restrict s1,
const char * restrict s2);

Parameters:

s1— The s1 string is modified and broken into smaller strings (tokens).

s2— The s2 string contains the delimiter characters. These may vary from one call to another.

Return:

The strtok function returns a pointer to the first character of a token or a null pointer if there is no token.

Implementing of own version strtok function in C:

Sometimes people ask the questions to implement the strtok function in C. Also, some times require to create your own strtok function. So let’s create our own strtok function to break the input string into a sequence of tokens

Note:Below function only to understand the working of strtok. There is a lot of scenarios that are not handled in this function. You can handle the scenario as your requirement and if possible then use the library function.

#include <stdio.h>

unsigned int is_delim(char c, char *delim)
{
    while(*delim != '\0')
    {
        if(c == *delim)
            return 1;
        delim++;
    }
    return 0;
}

char *my_strtok(char *srcString, char *delim)
{
    static char *backup_string; // start of the next search

    if(!srcString)
    {
        srcString = backup_string;
    }
    if(!srcString)
    {
        // user is bad user
        return NULL;
    }

    // handle beginning of the string containing delims
    while(1)
    {
        if(is_delim(*srcString, delim))
        {
            srcString++;
            continue;
        }
        if(*srcString == '\0')
        {
            // we've reached the end of the string
            return NULL; 
        }
        break;
    }

    char *ret = srcString;
    while(1)
    {
        if(*srcString == '\0')
        {
            /*end of the input string and
            next exec will return NULL*/
            backup_string = srcString;
            return ret;
        }
        if(is_delim(*srcString, delim))
        {
            *srcString = '\0';
            backup_string = srcString + 1;
            return ret;
        }
        srcString++;
    }
}

int main()
{
    //input string
    char srcString[] = "HI Aticleworld;Reader";
    //delimiter
    char *delim = "; ";
    //calling own version of strtok function
    char *token = my_strtok(srcString, delim);
    while(token)
    {
        printf("%s\n", token);
        token = my_strtok(NULL, delim);
    }

    return 0 ;
}

Output:

You should not assume that strtok() leaves the parse string unchanged. To keep your original srcString unchanged you should first copy srcString into some tmpString variable and then use that tmpString in strtok(). See the below code.

char str[] = "Hi Aticleworld How are you";


//strlen not safe to use. You can use it as per your requirement 
char* tmp = calloc(strlen(str)+1, sizeof(char));


/*strcpy is not safe to use here I am using it because 
  I am assuming input string has only one null terminating char*/

strcpy(tmp, str);