How to remove trailing whitespaces with sed?

Question

I have a simple shell script that removes trailing whitespace from a file. Is there any way to make this script more compact (without creating a temporary file)?

<         


        

Answer 1

You can use the in place option -i of sed for Linux and Unix:

sed -i 's/[ \t]*$//' "$1"

Be aware the expression will delete trailing t's on OSX (you can use gsed to avoid this problem). It may delete them on BSD too.

If you don't have gsed, here is the correct (but hard-to-read) sed syntax on OSX:

sed -i '' -E 's/[ '$'\t'']+$//' "$1"

Three single-quoted strings ultimately become concatenated into a single argument/expression. There is no concatenation operator in bash, you just place strings one after the other with no space in between.

The $'\t' resolves as a literal tab-character in bash (using ANSI-C quoting), so the tab is correctly concatenated into the expression.

Answer 2

It is best to also quote $1:

sed -i.bak 's/[[:blank:]]*$//' "$1"

Answer 3

Just for fun:

#!/bin/bash

FILE=$1

if [[ -z $FILE ]]; then
   echo "You must pass a filename -- exiting" >&2
   exit 1
fi

if [[ ! -f $FILE ]]; then
   echo "There is not file '$FILE' here -- exiting" >&2
   exit 1
fi

BEFORE=`wc -c "$FILE" | cut --delimiter=' ' --fields=1`

# >>>>>>>>>>
sed -i.bak -e's/[ \t]*$//' "$FILE"
# <<<<<<<<<<

AFTER=`wc -c "$FILE" | cut --delimiter=' ' --fields=1`

if [[ $? != 0 ]]; then
   echo "Some error occurred" >&2
else
   echo "Filtered '$FILE' from $BEFORE characters to $AFTER characters"
fi

Answer 4

var1="\t\t Test String trimming   "
echo $var1
Var2=$(echo "${var1}" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')
echo $Var2

Answer 5

I have a script in my .bashrc that works under OSX and Linux (bash only !)

function trim_trailing_space() {
  if [[ $# -eq 0 ]]; then
    echo "$FUNCNAME will trim (in place) trailing spaces in the given file (remove unwanted spaces at end of lines)"
    echo "Usage :"
    echo "$FUNCNAME file"
    return
  fi
  local file=$1
  unamestr=$(uname)
  if [[ $unamestr == 'Darwin' ]]; then
    #specific case for Mac OSX
    sed -E -i ''  's/[[:space:]]*$//' $file
  else
    sed -i  's/[[:space:]]*$//' $file
  fi
}

to which I add:

SRC_FILES_EXTENSIONS="js|ts|cpp|c|h|hpp|php|py|sh|cs|sql|json|ini|xml|conf"

function find_source_files() {
  if [[ $# -eq 0 ]]; then
    echo "$FUNCNAME will list sources files (having extensions $SRC_FILES_EXTENSIONS)"
    echo "Usage :"
    echo "$FUNCNAME folder"
    return
  fi
  local folder=$1

  unamestr=$(uname)
  if [[ $unamestr == 'Darwin' ]]; then
    #specific case for Mac OSX
    find -E $folder -iregex '.*\.('$SRC_FILES_EXTENSIONS')'
  else
    #Rhahhh, lovely
    local extensions_escaped=$(echo $SRC_FILES_EXTENSIONS | sed s/\|/\\\\\|/g)
    #echo "extensions_escaped:$extensions_escaped"
    find $folder -iregex '.*\.\('$extensions_escaped'\)$'
  fi
}

function trim_trailing_space_all_source_files() {
  for f in $(find_source_files .); do trim_trailing_space $f;done
}

Answer 6

In the specific case of sed, the -i option that others have already mentioned is far and away the simplest and sanest one.

In the more general case, sponge, from the moreutils collection, does exactly what you want: it lets you replace a file with the result of processing it, in a way specifically designed to keep the processing step from tripping over itself by overwriting the very file it's working on. To quote the sponge man page:

sponge reads standard input and writes it out to the specified file. Unlike a shell redirect, sponge soaks up all its input before writing the output file. This allows constructing pipelines that read from and write to the same file.

https://joeyh.name/code/moreutils/