How to trim whitespace from a Bash variable?

Nice programing

How to trim whitespace from a Bash variable?

nicepro 2020. 9. 28. 10:04

How to trim whitespace from a Bash variable?

I have a shell script with this code:

var=`hg st -R "$path"`
if [ -n "$var" ]; then
    echo $var
fi

But the conditional code always executes, because hg st always prints at least one newline character.

Is there a simple way to strip whitespace from $var (like trim() in PHP)?

Is there a standard way of dealing with this issue?

I could use sed or AWK, but I'd like to think there is a more elegant solution to this problem.

Let's define a variable containing leading, trailing, and intermediate whitespace:

FOO=' test test test '
echo -e "FOO='${FOO}'"
# > FOO=' test test test '
echo -e "length(FOO)==${#FOO}"
# > length(FOO)==16

How to remove all whitespace (denoted by [:space:] in tr):

FOO=' test test test '
FOO_NO_WHITESPACE="$(echo -e "${FOO}" | tr -d '[:space:]')"
echo -e "FOO_NO_WHITESPACE='${FOO_NO_WHITESPACE}'"
# > FOO_NO_WHITESPACE='testtesttest'
echo -e "length(FOO_NO_WHITESPACE)==${#FOO_NO_WHITESPACE}"
# > length(FOO_NO_WHITESPACE)==12

How to remove leading whitespace only:

FOO=' test test test '
FOO_NO_LEAD_SPACE="$(echo -e "${FOO}" | sed -e 's/^[[:space:]]*//')"
echo -e "FOO_NO_LEAD_SPACE='${FOO_NO_LEAD_SPACE}'"
# > FOO_NO_LEAD_SPACE='test test test '
echo -e "length(FOO_NO_LEAD_SPACE)==${#FOO_NO_LEAD_SPACE}"
# > length(FOO_NO_LEAD_SPACE)==15

How to remove trailing whitespace only:

FOO=' test test test '
FOO_NO_TRAIL_SPACE="$(echo -e "${FOO}" | sed -e 's/[[:space:]]*$//')"
echo -e "FOO_NO_TRAIL_SPACE='${FOO_NO_TRAIL_SPACE}'"
# > FOO_NO_TRAIL_SPACE=' test test test'
echo -e "length(FOO_NO_TRAIL_SPACE)==${#FOO_NO_TRAIL_SPACE}"
# > length(FOO_NO_TRAIL_SPACE)==15

How to remove both leading and trailing spaces--chain the seds:

FOO=' test test test '
FOO_NO_EXTERNAL_SPACE="$(echo -e "${FOO}" | sed -e 's/^[[:space:]]*//' -e 's/[[:space:]]*$//')"
echo -e "FOO_NO_EXTERNAL_SPACE='${FOO_NO_EXTERNAL_SPACE}'"
# > FOO_NO_EXTERNAL_SPACE='test test test'
echo -e "length(FOO_NO_EXTERNAL_SPACE)==${#FOO_NO_EXTERNAL_SPACE}"
# > length(FOO_NO_EXTERNAL_SPACE)==14

Alternatively, if your bash supports it, you can replace echo -e "${FOO}" | sed ... with sed ... <<<${FOO}, like so (for trailing whitespace):

FOO_NO_TRAIL_SPACE="$(sed -e 's/[[:space:]]*$//' <<<${FOO})"

A simple answer is:

echo "   lol  " | xargs

Xargs will do the trimming for you. It's one command/program, no parameters, returns the trimmed string, easy as that!

Note: this doesn't remove the internal spaces so "foo bar" stays the same. It does NOT become "foobar".

There is a solution which only uses Bash built-ins called wildcards:

var="    abc    "
# remove leading whitespace characters
var="${var#"${var%%[![:space:]]*}"}"
# remove trailing whitespace characters
var="${var%"${var##*[![:space:]]}"}"   
echo "===$var==="

Here's the same wrapped in a function:

trim() {
    local var="$*"
    # remove leading whitespace characters
    var="${var#"${var%%[![:space:]]*}"}"
    # remove trailing whitespace characters
    var="${var%"${var##*[![:space:]]}"}"   
    echo -n "$var"
}

You pass the string to be trimmed in quoted form. e.g.:

trim "   abc   "

A nice thing about this solution is that it will work with any POSIX-compliant shell.

Reference

Remove leading & trailing whitespace from a Bash variable (original source)

Bash has a feature called parameter expansion, which, among other things, allows string replacement based on so-called patterns (patterns resemble regular expressions, but there are fundamental differences and limitations). [flussence's original line: Bash has regular expressions, but they're well-hidden:]

The following demonstrates how to remove all white space (even from the interior) from a variable value.

$ var='abc def'
$ echo "$var"
abc def
# Note: flussence's original expression was "${var/ /}", which only replaced the *first* space char., wherever it appeared.
$ echo -n "${var//[[:space:]]/}"
abcdef

In order to remove all the spaces from the beginning and the end of a string (including end of line characters):

echo $variable | xargs echo -n

This will remove duplicate spaces also:

echo "  this string has a lot       of spaces " | xargs echo -n

Produces: 'this string has a lot of spaces'

Strip one leading and one trailing space

trim()
{
    local trimmed="$1"

    # Strip leading space.
    trimmed="${trimmed## }"
    # Strip trailing space.
    trimmed="${trimmed%% }"

    echo "$trimmed"
}

For example:

test1="$(trim " one leading")"
test2="$(trim "one trailing ")"
test3="$(trim " one leading and one trailing ")"
echo "'$test1', '$test2', '$test3'"

Output:

'one leading', 'one trailing', 'one leading and one trailing'

Strip all leading and trailing spaces

trim()
{
    local trimmed="$1"

    # Strip leading spaces.
    while [[ $trimmed == ' '* ]]; do
       trimmed="${trimmed## }"
    done
    # Strip trailing spaces.
    while [[ $trimmed == *' ' ]]; do
        trimmed="${trimmed%% }"
    done

    echo "$trimmed"
}

For example:

test4="$(trim "  two leading")"
test5="$(trim "two trailing  ")"
test6="$(trim "  two leading and two trailing  ")"
echo "'$test4', '$test5', '$test6'"

Output:

'two leading', 'two trailing', 'two leading and two trailing'

You can trim simply with echo:

foo=" qsdqsd qsdqs q qs   "

# Not trimmed
echo \'$foo\'

# Trim
foo=`echo $foo`

# Trimmed
echo \'$foo\'

From Bash Guide section on globbing

To use an extglob in a parameter expansion

 #Turn on extended globbing  
shopt -s extglob  
 #Trim leading and trailing whitespace from a variable  
x=${x##+([[:space:]])}; x=${x%%+([[:space:]])}  
 #Turn off extended globbing  
shopt -u extglob

Here's the same functionality wrapped in a function (NOTE: Need to quote input string passed to function):

trim() {
    # Determine if 'extglob' is currently on.
    local extglobWasOff=1
    shopt extglob >/dev/null && extglobWasOff=0 
    (( extglobWasOff )) && shopt -s extglob # Turn 'extglob' on, if currently turned off.
    # Trim leading and trailing whitespace
    local var=$1
    var=${var##+([[:space:]])}
    var=${var%%+([[:space:]])}
    (( extglobWasOff )) && shopt -u extglob # If 'extglob' was off before, turn it back off.
    echo -n "$var"  # Output trimmed string.
}

Usage:

string="   abc def ghi  ";
#need to quote input-string to preserve internal white-space if any
trimmed=$(trim "$string");  
echo "$trimmed";

If we alter the function to execute in a subshell, we don't have to worry about examining the current shell option for extglob, we can just set it without affecting the current shell. This simplifies the function tremendously. I also update the positional parameters "in place" so I don't even need a local variable

trim() {
    shopt -s extglob
    set -- "${1##+([[:space:]])}"
    printf "%s" "${1%%+([[:space:]])}" 
}

so:

$ s=$'\t\n \r\tfoo  '
$ shopt -u extglob
$ shopt extglob
extglob         off
$ printf ">%q<\n" "$s" "$(trim "$s")"
>$'\t\n \r\tfoo  '<
>foo<
$ shopt extglob
extglob         off

With Bash's extended pattern matching features enabled (shopt -s extglob), you can use this:

{trimmed##*( )}

to remove an arbitrary amount of leading spaces.

I've always done it with sed

  var=`hg st -R "$path" | sed -e 's/  *$//'`

If there is a more elegant solution, I hope somebody posts it.

You can delete newlines with tr:

var=`hg st -R "$path" | tr -d '\n'`
if [ -n $var ]; then
    echo $var
done

# Trim whitespace from both ends of specified parameter

trim () {
    read -rd '' $1 <<<"${!1}"
}

# Unit test for trim()

test_trim () {
    local foo="$1"
    trim foo
    test "$foo" = "$2"
}

test_trim hey hey &&
test_trim '  hey' hey &&
test_trim 'ho  ' ho &&
test_trim 'hey ho' 'hey ho' &&
test_trim '  hey  ho  ' 'hey  ho' &&
test_trim $'\n\n\t hey\n\t ho \t\n' $'hey\n\t ho' &&
test_trim $'\n' '' &&
test_trim '\n' '\n' &&
echo passed

You can use old-school tr. For example, this returns the number of modified files in a git repository, whitespaces stripped.

MYVAR=`git ls-files -m|wc -l|tr -d ' '`

This worked for me:

text="   trim my edges    "

trimmed=$text
trimmed=${trimmed##+( )} #Remove longest matching series of spaces from the front
trimmed=${trimmed%%+( )} #Remove longest matching series of spaces from the back

echo "<$trimmed>" #Adding angle braces just to make it easier to confirm that all spaces are removed

#Result
<trim my edges>

To put that on fewer lines for the same result:

text="    trim my edges    "
trimmed=${${text##+( )}%%+( )}

There are a lot of answers, but I still believe my just-written script is worth being mentioned because:

it was successfully tested in the shells bash/dash/busybox shell
it is extremely small
it doesn't depend on external commands and doesn't need to fork (->fast and low resource usage)
it works as expected:
- it strips all spaces and tabs from beginning and end, but not more
- important: it doesn't remove anything from the middle of the string (many other answers do), even newlines will remain
- special: the "$*" joins multiple arguments using one space. if you want to trim & output only the first argument, use "$1" instead
- if doesn't have any problems with matching file name patterns etc

The script:

trim() {
  local s2 s="$*"
  # note: the brackets in each of the following two lines contain one space
  # and one tab
  until s2="${s#[   ]}"; [ "$s2" = "$s" ]; do s="$s2"; done
  until s2="${s%[   ]}"; [ "$s2" = "$s" ]; do s="$s2"; done
  echo "$s"
}

Usage:

mystring="   here     is
    something    "
mystring=$(trim "$mystring")
echo ">$mystring<"

Output:

>here     is
    something<

# Strip leading and trailing white space (new line inclusive).
trim(){
    [[ "$1" =~ [^[:space:]](.*[^[:space:]])? ]]
    printf "%s" "$BASH_REMATCH"
}

# Strip leading white space (new line inclusive).
ltrim(){
    [[ "$1" =~ [^[:space:]].* ]]
    printf "%s" "$BASH_REMATCH"
}

# Strip trailing white space (new line inclusive).
rtrim(){
    [[ "$1" =~ .*[^[:space:]] ]]
    printf "%s" "$BASH_REMATCH"
}

# Strip leading and trailing white space (new line inclusive).
trim(){
    printf "%s" "$(rtrim "$(ltrim "$1")")"
}

# Strip leading and trailing specified characters.  ex: str=$(trim "$str" $'\n a')
trim(){
    if [ "$2" ]; then
        trim_chrs="$2"
    else
        trim_chrs="[:space:]"
    fi

    [[ "$1" =~ ^["$trim_chrs"]*(.*[^"$trim_chrs"])["$trim_chrs"]*$ ]]
    printf "%s" "${BASH_REMATCH[1]}"
}

# Strip leading specified characters.  ex: str=$(ltrim "$str" $'\n a')
ltrim(){
    if [ "$2" ]; then
        trim_chrs="$2"
    else
        trim_chrs="[:space:]"
    fi

    [[ "$1" =~ ^["$trim_chrs"]*(.*[^"$trim_chrs"]) ]]
    printf "%s" "${BASH_REMATCH[1]}"
}

# Strip trailing specified characters.  ex: str=$(rtrim "$str" $'\n a')
rtrim(){
    if [ "$2" ]; then
        trim_chrs="$2"
    else
        trim_chrs="[:space:]"
    fi

    [[ "$1" =~ ^(.*[^"$trim_chrs"])["$trim_chrs"]*$ ]]
    printf "%s" "${BASH_REMATCH[1]}"
}

# Strip leading and trailing specified characters.  ex: str=$(trim "$str" $'\n a')
trim(){
    printf "%s" "$(rtrim "$(ltrim "$1" "$2")" "$2")"
}

Building upon moskit's expr soulution...

# Strip leading and trailing white space (new line inclusive).
trim(){
    printf "%s" "`expr "$1" : "^[[:space:]]*\(.*[^[:space:]]\)[[:space:]]*$"`"
}

# Strip leading white space (new line inclusive).
ltrim(){
    printf "%s" "`expr "$1" : "^[[:space:]]*\(.*[^[:space:]]\)"`"
}

# Strip trailing white space (new line inclusive).
rtrim(){
    printf "%s" "`expr "$1" : "^\(.*[^[:space:]]\)[[:space:]]*$"`"
}

# Strip leading and trailing white space (new line inclusive).
trim(){
    printf "%s" "$(rtrim "$(ltrim "$1")")"
}

I've seen scripts just use variable assignment to do the job:

$ xyz=`echo -e 'foo \n bar'`
$ echo $xyz
foo bar

Whitespace is automatically coalesced and trimmed. One has to be careful of shell metacharacters (potential injection risk).

I would also recommend always double-quoting variable substitutions in shell conditionals:

if [ -n "$var" ]; then

since something like a -o or other content in the variable could amend your test arguments.

var='   a b c   '
trimmed=$(echo $var)

I would simply use sed:

function trim
{
    echo "$1" | sed -n '1h;1!H;${;g;s/^[ \t]*//g;s/[ \t]*$//g;p;}'
}

a) Example of usage on single-line string

string='    wordA wordB  wordC   wordD    '
trimmed=$( trim "$string" )

echo "GIVEN STRING: |$string|"
echo "TRIMMED STRING: |$trimmed|"

Output:

GIVEN STRING: |    wordA wordB  wordC   wordD    |
TRIMMED STRING: |wordA wordB  wordC   wordD|

b) Example of usage on multi-line string

string='    wordA
   >wordB<
wordC    '
trimmed=$( trim "$string" )

echo -e "GIVEN STRING: |$string|\n"
echo "TRIMMED STRING: |$trimmed|"

Output:

GIVEN STRING: |    wordAA
   >wordB<
wordC    |

TRIMMED STRING: |wordAA
   >wordB<
wordC|

c) Final note:
If you don't like to use a function, for single-line string you can simply use a "easier to remember" command like:

echo "$string" | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//'

Example:

echo "   wordA wordB wordC   " | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//'

Output:

wordA wordB wordC

Using the above on multi-line strings will work as well, but please note that it will cut any trailing/leading internal multiple space as well, as GuruM noticed in the comments

string='    wordAA
    >four spaces before<
 >one space before<    '
echo "$string" | sed -e 's/^[ \t]*//' | sed -e 's/[ \t]*$//'

Output:

wordAA
>four spaces before<
>one space before<

So if you do mind to keep those spaces, please use the function at the beginning of my answer!

d) EXPLANATION of the sed syntax "find and replace" on multi-line strings used inside the function trim:

sed -n '
# If the first line, copy the pattern to the hold buffer
1h
# If not the first line, then append the pattern to the hold buffer
1!H
# If the last line then ...
$ {
    # Copy from the hold to the pattern buffer
    g
    # Do the search and replace
    s/^[ \t]*//g
    s/[ \t]*$//g
    # print
    p
}'

Use AWK:

echo $var | awk '{gsub(/^ +| +$/,"")}1'

Here's a trim() function that trims and normalizes whitespace

#!/bin/bash
function trim {
    echo $*
}

echo "'$(trim "  one   two    three  ")'"
# 'one two three'

And another variant that uses regular expressions.

#!/bin/bash
function trim {
    local trimmed="$@"
    if [[ "$trimmed" =~ " *([^ ].*[^ ]) *" ]]
    then 
        trimmed=${BASH_REMATCH[1]}
    fi
    echo "$trimmed"
}

echo "'$(trim "  one   two    three  ")'"
# 'one   two    three'

This does not have the problem with unwanted globbing, also, interior white-space is unmodified (assuming that $IFS is set to the default, which is ' \t\n').

It reads up to the first newline (and doesn't include it) or the end of string, whichever comes first, and strips away any mix of leading and trailing space and \t characters. If you want to preserve multiple lines (and also strip leading and trailing newlines), use read -r -d '' var << eof instead; note, however, that if your input happens to contain \neof, it will be cut off just before. (Other forms of white space, namely \r, \f, and \v, are not stripped, even if you add them to $IFS.)

read -r var << eof
$var
eof

To remove spaces and tabs from left to first word, enter:

echo "     This is a test" | sed "s/^[ \t]*//"

cyberciti.biz/tips/delete-leading-spaces-from-front-of-each-word.html

This will remove all the whitespaces from your String,

 VAR2="${VAR2//[[:space:]]/}"

/ replaces the first occurrence and // all occurrences of whitespaces in the string. I.e. all white spaces get replaced by – nothing

Assignments ignore leading and trailing whitespace and as such can be used to trim:

$ var=`echo '   hello'`; echo $var
hello

This is the simplest method I've seen. It only uses Bash, it's only a few lines, the regexp is simple, and it matches all forms of whitespace:

if [[ "$test" =~ ^[[:space:]]*([^[:space:]].*[^[:space:]])[[:space:]]*$ ]]
then 
    test=${BASH_REMATCH[1]}
fi

Here is a sample script to test it with:

test=$(echo -e "\n \t Spaces and tabs and newlines be gone! \t  \n ")

echo "Let's see if this works:"
echo
echo "----------"
echo -e "Testing:${test} :Tested"  # Ugh!
echo "----------"
echo
echo "Ugh!  Let's fix that..."

if [[ "$test" =~ ^[[:space:]]*([^[:space:]].*[^[:space:]])[[:space:]]*$ ]]
then 
    test=${BASH_REMATCH[1]}
fi

echo
echo "----------"
echo -e "Testing:${test}:Tested"  # "Testing:Spaces and tabs and newlines be gone!"
echo "----------"
echo
echo "Ah, much better."

Python has a function strip() that works identically to PHP's trim(), so we can just do a little inline Python to make an easily understandable utility for this:

alias trim='python -c "import sys; sys.stdout.write(sys.stdin.read().strip())"'

This will trim leading and trailing whitespace (including newlines).

$ x=`echo -e "\n\t   \n" | trim`
$ if [ -z "$x" ]; then echo hi; fi
hi

#!/bin/bash

function trim
{
    typeset trimVar
    eval trimVar="\${$1}"
    read trimVar << EOTtrim
    $trimVar
EOTtrim
    eval $1=\$trimVar
}

# Note that the parameter to the function is the NAME of the variable to trim, 
# not the variable contents.  However, the contents are trimmed.


# Example of use:
while read aLine
do
    trim aline
    echo "[${aline}]"
done < info.txt



# File info.txt contents:
# ------------------------------
# ok  hello there    $
#    another  line   here     $
#and yet another   $
#  only at the front$
#$



# Output:
#[ok  hello there]
#[another  line   here]
#[and yet another]
#[only at the front]
#[]

I found that I needed to add some code from a messy sdiff output in order to clean it up:

sdiff -s column1.txt column2.txt | grep -F '<' | cut -f1 -d"<" > c12diff.txt 
sed -n 1'p' c12diff.txt | sed 's/ *$//g' | tr -d '\n' | tr -d '\t'

This removes the trailing spaces and other invisible characters.

I created the following functions. I am not sure how portable printf is, but the beauty of this solution is you can specify exactly what is "white space" by adding more character codes.

    iswhitespace()
    {
        n=`printf "%d\n" "'$1'"`
        if (( $n != "13" )) && (( $n != "10" )) && (( $n != "32" )) && (( $n != "92" )) && (( $n != "110" )) && (( $n != "114" )); then
            return 0
        fi
        return 1
    }

    trim()
    {
        i=0
        str="$1"
        while (( i < ${#1} ))
        do
            char=${1:$i:1}
            iswhitespace "$char"
            if [ "$?" -eq "0" ]; then
                str="${str:$i}"
                i=${#1}
            fi
            (( i += 1 ))
        done
        i=${#str}
        while (( i > "0" ))
        do
            (( i -= 1 ))
            char=${str:$i:1}
            iswhitespace "$char"
            if [ "$?" -eq "0" ]; then
                (( i += 1 ))
                str="${str:0:$i}"
                i=0
            fi
        done
        echo "$str"
    }

#Call it like so
mystring=`trim "$mystring"`

참고URL : https://stackoverflow.com/questions/369758/how-to-trim-whitespace-from-a-bash-variable

'Nice programing' 카테고리의 다른 글

.css () 함수로 추가 된 스타일을 제거하려면 어떻게해야합니까? (0)	2020.09.28
함수 이름 앞에 JavaScript 더하기 기호 (0)	2020.09.28
How to fix “Headers already sent” error in PHP (0)	2020.09.28
How to prevent buttons from submitting forms (0)	2020.09.28
How to check a radio button with jQuery? (0)	2020.09.28

현재글How to trim whitespace from a Bash variable?

nicepro

How to trim whitespace from a Bash variable?

How to trim whitespace from a Bash variable?

Reference

Strip one leading and one trailing space

Strip all leading and trailing spaces

'Nice programing' 카테고리의 다른 글

'Nice programing'의 다른글

티스토리툴바

How to trim whitespace from a Bash variable?

How to trim whitespace from a Bash variable?

Reference

Strip one leading and one trailing space

Strip all leading and trailing spaces

'Nice programing' 카테고리의 다른 글

'Nice programing'의 다른글

관련글

티스토리툴바