strtok vs strtok_r - extract tokens from strings

Name

strtok, strtok_r - extract tokens from strings

Synopsis

       #include <string.h>

       char *strtok(char *str, const char *delim);
       char *strtok_r(char *str, const char *delim, char **saveptr);

   Feature Test Macro Requirements for glibc (see feature_test_macros(7)):
       strtok_r(): _POSIX_C_SOURCE
           || /* Glibc versions <= 2.19: */ _BSD_SOURCE || _SVID_SOURCE

Note:

1. Don't use original string pointer because, it will replace the
   delimiters with '\0'.
2. Suppose Original string is constant then crash will be hit.
3. strtok will use the global pointer, so could not use in thread & 
   2 strtok simultaneously.
4. strtok_r is a thread save, because it use the local pointer for
   next iteration.

Description:

• strtok() and strtok_r() for splitting a string by some delimiter.
• The strtok() function breaks a string into a sequence of zero or more nonempty tokens.
• On the first call to strtok(), the string to be parsed should be specified in str.
• In  each subsequent call that should parse the same string, str must be NULL.
•  
• The  delim argument specifies a set of bytes that delimit the tokens in the parsed string.
• The strtok_r() function is a reentrant  version  strtok().
• The  saveptr  argument  is a pointer  to  a  char * variable that is used internally by strtok_r() in order to maintain context between successive calls that parse the same string.

 Return:

• Return a pointer to the next token, or NULL if there are no more tokens.

Program:

/* strtok vs strtok_r by Velraj.K
   Check : http://velrajcoding.blogspot.in
 */

#include <stdio.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char *argv[])
{
    char *str1, *str2, *token, *subtoken;
    char *saveptr1, *saveptr2;
    char str_second[256] = {'\0'};
   // *str_second = "VelrajKutralam Indian";
    int j;

    if (argc != 4) {
        fprintf(stderr, "Usage: %s  <1st delim="">  <2nd subdelim="">\n",
                argv[0]);
        exit(EXIT_FAILURE);
    }

    /* The original string will be replaced with null in delimiter places */
    strncpy(str_second, argv[1], sizeof(str_second));

    printf("\n\n\t\t Do the token using strtok_r \n");

    /* use strtok_r */
    for (j = 1, str1 = argv[1]; ; j++, str1 = NULL) {
        token = strtok_r(str1, argv[2], &saveptr1);
        if (token == NULL)
            break;
        printf("%d: %s\n", j, token);

        for (str2 = token; ; str2 = NULL) {
            subtoken = strtok_r(str2, argv[3], &saveptr2);
            if (subtoken == NULL)
                break;
            printf(" --> %s\n", subtoken);
        }
    }
   printf("Original value = %s \n", argv[1]);

    printf("\n\n\t\tDo the token using strtok str_second = %s \n", str_second);
    /* Initialize the variable to NULL to start token again with strtok */
    str1 = NULL;
    str2 = NULL;
    token = NULL;
    subtoken = NULL;
    saveptr1 = NULL;
    saveptr2 = NULL;

    /* use strtok */
    for (j = 1, str1 = str_second; ; j++, str1 = NULL) {
        token = strtok(str1, argv[2]);
        if (token == NULL)
            break;
        printf("%d: %s\n", j, token);

       /*
        *  Could not use the 2nd strtok with different delim because it uses the global pointer
        *  to process the next token.
        */
#if 0
        for (str2 = token; ; str2 = NULL) {
            subtoken = strtok(str2, argv[3]);
            if (subtoken == NULL)
                break;
            printf(" --> %s\n", subtoken);
        }
#endif
    }

    exit(EXIT_SUCCESS);
}

 

Output:

:~/velrajk/sample/learn$ ./a.out  velrajkutralamindia l v


                 Do the token using strtok_r
1: ve
 --> e
2: rajkutra
 --> rajkutra
3: amindia
 --> amindia
Original value = ve


                Do the token using strtok str_second = velrajkutralamindia
1: ve
2: rajkutra
3: amindia