Removing Windows newlines on Linux (sed vs. awk)

Question

Have some delimited files with improperly placed newline characters in the middle of fields (not line ends), appearing as ^M in Vim. They originate from freebcp (on Centos 6

Answer 1

You can use the command line tool dos2unix

dos2unix input

Or use the tr command:

tr -d '\r' output

---

Actually, you can do the file-format switching in vim:

Method A:

:e ++ff=dos
:w ++ff=unix
:e!

Method B:

:e ++ff=dos
:set ff=unix
:w

---

EDIT

If you want to delete the \r\n sequences in the file, try these commands in vim:

:e ++ff=unix           " <-- make sure open with UNIX format
:%s/\r\n//g            " <-- remove all \r\n
:w                     " <-- save file

Your awk solution works fine. Another two sed solutions:

sed '1h;1!H;$!d;${g;s/\r\n//g}' input
sed ':A;/\r$/{N;bA};s/\r\n//g' input