Spring Batch DelimitedLineTokenizer class quoteCharacter property behavior

Question

I have a item reader as below:

<beans:bean id="myItemReader" class="org.springframework.batch.item.file.FlatFileItemReader">
<beans:property name="resource" ref="myFileResource" />
    <beans:property name="lineMapper">
        <beans:bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
            <beans:property name="lineTokenizer">
                <beans:bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
                    <beans:property name="delimiter" value="|"/>
                    <beans:property name="quoteCharacter" value="~"/>
                    <beans:property name="names" value="${my_column_names}" />
                </beans:bean>
            </beans:property>
            <beans:property name="fieldSetMapper">
            <beans:bean class="${my_mapper_class_name}" />
        </beans:property>
    </beans:bean>
</beans:property>

As clear from above, I am changing the quote character from default " (double quotes) to ~ (tilt sign)

Now, Java Docs on DelimitedLineTokenizer on quote character says:

Public setter for the quoteCharacter. The quote character can be used to extend a field across line endings or to enclose a String which contains the delimiter. Inside a quoted token the quote character can be used to escape itself, thus "a""b""c" is tokenized to a"b"c.

Thus, if my data itself is containing delimiter (the pipe sign in my case), then I should surround it with the quote character - which I did. E.g. row below where third column contains the delimiter:

oneColumn|twoColumn|three~|~Column|fourColumn

However, the value which comes inside my java object for column three is "three~|~Column" and not "three|Column" as it should be.

Shouldn't the quote character used to escape a delimiter in the data be automatically handled and ignored while setting the value inside the property of the resultant java object item?

Is replacing the quote character with empty string again in the writer (PreparedStatementSetter or LineAggregator) the only solution to this ?

Thanks for reading!

Answer 1

A string is intended to be quoted if it starts/ends with quote char. In your example:

oneColumn|twoColumn|~three|Column~|fourColumn

and you'll get three|Column You have to quote string if it contains a separator, space, newline or the quoting char itself; quoting char must be doubled if part of value(see here)