What's the difference between strtok_r and strtok_s in C?

Question

I'm trying to use this function in a C program that needs to be able to compile in Linux and Windows. At first I tried using strtok_r, but then when I compiled on windows, it complained about the function not existing and said it would assume it's an extern function, but then failed. I then used strtok_s and it compiled! Then I tried on Linux but now it's complaining that there is an "undefined reference to 'strtok_s'".

Is one a windows only function and the other a linux function??? What can I do to make it compile on both?

Answer 1

Both of these functions are really ugly, unintuitive idioms for parsing strings, and usually fail to meet your particular application's requirements in subtle ways. Even moreso for the plain strtok in standard C. Just throw them out and write your own code to iterate over the char array and break it up as needed. strchr, strspn, and strcspn can be helpful in doing this or you can just work from scratch on the array.

Answer 2

strtok_s is simply the Windows version of strtok_r which is standard everywhere else.

One (common I would think) way to make a program portable when it comes to functions like strtok_s/strtok_r is to use the preprocessor:

#if defined(_WIN32) || defined(_WIN64)
/* We are on Windows */
# define strtok_r strtok_s
#endif

As the prototypes and functionality is the same, you can now use only strtok_r.

Answer 3

I don't have enough reputation to comment on other answers, so I'll have to provide my own.

1) To address this statement:

"strtok_s is a buffer overrun safe version of strtok on Windows. The standard strtok on windows is thread safe..."

This is not true. strtok_s is the thread safe version for the MSVC compiler. strtok is not thread safe!

2) To address this statement:

"This would probably break if compiling on Cygwin which reports itself as windows but has POSIX interfaces like strtok_r already defined."

Again, not true. The difference is which compiler you use. When using Microsoft's Visual C++ compiler, MSVC, the function is strtok_s. Another compiler, such as the GNU Compiler Collection, GCC, may use a different standard library implementation such as strtok_r. Think compiler, not target platform, when identifying which function to use.

In my opinion, Joachim Pileborg's answer is the best one on this page. However, it needs a small edit:

#if defined(_WIN32) /* || defined(_WIN64) */
#define strtok_r strtok_s
#endif

Both _WIN32 and _WIN64 are predefined macros provided by the MSVC compiler. _WIN64 is defined when compiling a 64 bit target. _WIN32 is defined for both 32 and 64 bit targets. This is a compromise that Microsoft made for backwards compatibility. _WIN32 was created to specify the Win32 API. Now you should consider _WIN32 to specify Windows API -- it is not specific to a 32 bit target.

Answer 4

strtok_r is a thread safe version of strtok on POSIX systems

strtok_s is a buffer overrun safe version of strtok on Windows. The standard strtok on windows is thread safe, so strtok_s should be.

Answer 5

Just to clarify. strtok is thread safe in Windows. strtok uses a TLS variable to maintain the last pointer for each thread. However, you can not use strtok to interleave access to more than one token string per thread. strtok_r and strtok_s both address this interleaving problem by allowing the user to maintain the context via the third parameter. Hope this helps.

Answer 6

MinGW also predefines _WIN32 but it supports strtok_r, so I don't think it is a good idea to check the _WIN32 macro. It is better to check the _MSC_VER macro, which is the macro for Microsoft Visual Studio.

#ifdef _MSC_VER
#define strtok_r strtok_s
#endif

WARNING: Microsoft strtok_s and C11 strtok_s are completely different! Microsoft strtok_s has only 3 parameters, while C11 strtok_s has 4 parameters, so it can be a compatible issue in the future.

The prototype of Microsoft strtok_s is

char* strtok_s(char* str, const char* delimiters, char** context);

The prototype of C11 strtok_s is

char *strtok_s(char *restrict str, rsize_t *restrict strmax, const char *restrict delim, char **restrict ptr);