Using boost::tokenizer with string delimiters

Question

I\'ve been looking boost::tokenizer, and I\'ve found that the documentation is very thin. Is it possible to make it tokenize a string such as \"dolphin--monkey--baboon\" and

Answer 1

I know the theme is quite old, but it is shown in the top links in google when I search "boost tokenizer by string"

so I will add my variant of TokenizerFunction, just in case:

class FindStrTFunc
{
public:
    FindStrTFunc() : m_str(g_dataSeparator)
    {
    }

    bool operator()(std::string::const_iterator& next,
        const std::string::const_iterator& end, std::string& tok) const
    {
        if (next == end)
        {
            return false;
        }
        const std::string::const_iterator foundToken =
            std::search(next, end, m_str.begin(), m_str.end());
        tok.assign(next, foundToken);
        next = (foundToken == end) ? end : foundToken + m_str.size();
        return true;
    }

    void reset()
    {
    }

private:
    std::string m_str;
};

after we can create

boost::tokenizer tok("some input...some other input");

and use, like a usual boost tokenizer