How can I use non-default delimiters when reading a text file with std::fstream?

Question

In my C++ code, I want to read from a text file (*.txt) and tokenize every entry. More specifically, I want to be able to read individual words from a file, such as \"format

Answer 1

An istream treats "white space" as delimiters. It uses a locale to tell it what characters are white space. A locale, in turn, includes a ctype facet that classifies character types. Such a facet could look something like this:

#include 
#include 
#include 
#include 
#include 
#include 

class my_ctype : public
std::ctype
{
    mask my_table[table_size];
public:
    my_ctype(size_t refs = 0)  
        : std::ctype(&my_table[0], false, refs)
    {
        std::copy_n(classic_table(), table_size, my_table);
        my_table['-'] = (mask)space;
        my_table['\''] = (mask)space;
    }
};

And a little test program to show it works:

int main() {
    std::istringstream input("This is some input from McDonald's and Burger-King.");
    std::locale x(std::locale::classic(), new my_ctype);
    input.imbue(x);

    std::copy(std::istream_iterator(input),
        std::istream_iterator(),
        std::ostream_iterator(std::cout, "\n"));

    return 0;
}

Result:

This
is
some
input
from
McDonald
s
and
Burger
King.

istream_iterator uses >> to read the individual strings from the stream, so if you use them directly, you should get the same results. The parts you need to include are creating the locale and using imbue to make the stream use that locale.