What is newline character — '\n'

Question

This is a very basic concept, but something I have never been able to articulate that well. and I would like to try to spell it and see where I go wrong.

If I have t

Answer 1

I think this post by Jeff Attwood addresses your question perfectly. It takes you through the differences between newlines on Dos, Mac and Unix, and then explains the history of CR (Carriage return) and LF (Line feed).

Answer 2

sed can be put into multi-line search & replace mode to match newline characters \n.

To do so sed first has to read the entire file or string into the hold buffer ("hold space") so that it then can treat the file or string contents as a single line in "pattern space".

To replace a single newline portably (with respect to GNU and FreeBSD sed) you can use an escaped "real" newline.

# cf. http://austinmatzko.com/2008/04/26/sed-multi-line-search-and-replace/
echo 'California
Massachusetts
Arizona' | 
sed -n -e '
# if the first line copy the pattern to the hold buffer
1h
# if not the first line then append the pattern to the hold buffer
1!H
# if the last line then ...
$ {
# copy from the hold to the pattern buffer
g
# double newlines
s/\n/\
\
/g
s/$/\
/
p
}'

# output
# California
#
# Massachusetts
#
# Arizona
#

There is, however, a much more convenient was to achieve the same result:

echo 'California
Massachusetts
Arizona' | 
   sed G

Answer 3

From the sed man page:

Normally, sed cyclically copies a line of input, not including its terminating newline character, into a pattern space, (unless there is something left after a "D" function), applies all of the commands with addresses that select that pattern space, copies the pattern space to the standard output, appending a newline, and deletes the pattern space.

It's operating on the line without the newline present, so the pattern you have there can't ever match. You need to do something else - like match against $ (end-of-line) or ^ (start-of-line).

Here's an example of something that worked for me:

$ cat > states
California
Massachusetts
Arizona
$ sed -e 's/$/\
> /' states
California

Massachusetts

Arizona

I typed a literal newline character after the \ in the sed line.

Answer 4

I see a lot of sed answers, but none for vim. To be fair, vim's treatment of newline characters is a little confusing. Search for \n but replace with \r. I recommend RTFM: :help pattern in general and :help NL-used-for-Nul in particular.

To do what you want with a :substitute command,

:%s/\_$/\r

although I think most people would use something like

:g/^/put=''

for the same effect.

Here is a way to find the answer for yourself. Run your file through xxd, which is part of the standard vim distribution.

:%!xxd

You get

0000000: 4361 6c69 666f 726e 6961 0a4d 6173 7361  California.Massa
0000010: 6368 7573 6574 7473 0a41 7269 7a6f 6e61  chusetts.Arizona
0000020: 0a                                       .

This shows that 46 is the hex code for C, 61 is the code for a, and so on. In particular, 0a (decimal 10) is the code for \n. Just for kicks, try

:set ff=dos

before filtering through xxd. You will see 0d0a (CRLF) as the line terminator.

:help /\_$
:help :g
:help :put
:help :!
:help 23.4

Answer 5

Escape characters are dependent on whatever system is interpreting them. \n is interpreted as a newline character by many programming languages, but that doesn't necessarily hold true for the other utilities you mention. Even if they do treat \n as newline, there may be some other techniques to get them to behave how you want. You would have to consult their documentation (or see other answers here).

For DOS/Windows systems, the newline is actually two characters: Carriage Return (ASCII 13, AKA \r), followed by Line Feed (ASCII 10). On Unix systems (including Mac OSX) it's just Line Feed. On older Macs it was a single Carriage Return.

Answer 6

NewLine (\n) is 10 (0xA) and CarriageReturn (\r) is 13 (0xD).

Different operating systems picked different end of line representations for files. Windows uses CRLF (\r\n). Unix uses LF (\n). Older Mac OS versions use CR (\r), but OS X switched to the Unix character.

Here is a relatively useful FAQ.