What do double-asterisk wildcards mean?

Question

I\'ve tried the following command but I am having trouble interpreting it:

ls **

but I\'m not sure exactly what it is outputting and why th

Answer 1

It's probably using a special feature of some shells that lets ** do a recursive match, as opposed to a single *, which matches only in the current directory.

The wildcard * matches any file or directory (whose name doesn't start with .) in the current directory.

Depending on which shell you're using, and with which settings, ** may be equivalent to *; it could match zero or more characters followed by zero or more characters, which is the same as matching zero or more characters just once.

But with some shells, with some settings, ** is a recursive version of *, matching all files and directories in the current directory and subdirectories.

Quoting the bash manual:

`*'
Matches any string, including the null string. When the `globstar' shell option is enabled, and `*' is used in a filename expansion context, two adjacent `*'s used as a single pattern will match all files and zero or more directories and subdirectories. If followed by a `/', two adjacent ``*'s will match only directories and subdirectories.

This works only if the globstar option is set via:

shopt -s globstar

(it's disabled by default) and only in relatively recent versions of bash.

I believe zsh also supports this syntax.

It's important to keep in mind that wildcards are expanded by the shell, not by the ls command. If you type ls **, or ls *.txt, the ls command itself never sees the * characters; it only sees an expanded list of files, just as if you had typed the entire list on the command line.

Answer 2

Globbing

By using the double asterisk (**), you are using a glob to list files on a filesystem. A glob is a string of literal or wildcard characters used for matching the file paths. Using one or more globs for locating files on a filesystem is called globbing.

Apart from Linux shells, globbing is also used in various configuration files to specify the list of files to locate. For example: files and folders to ignore in the .gitignore file, files and include options in tsconfig.json file in Typescript projects etc.

Following are some of the most important aspects of the globbing and double asterisk (**) is one of them:

---

Segments and Separators (/)

The separator is always the / character. A segment is everything that comes between the two separators.

Example: Tests/HelloWorld.js

Here, Tests and HelloWorld.js are the segments and / is the separator.

---

Single Asterisk (*)

Single Asterisk (*) matches zero or more characters within one segment. It is used for globbing the files within one directory.

Example: *.js

This glob will match files such as HelloWorld.js but not files like Tests/HelloWorld.js or Tests/UI/HelloWorld.js

---

Double Asterisk (**)

Double Asterisk (**) matches zero or more characters across multiple segments. It is used for globbing files that are in nested directories.

Example: Tests/**/*.js

Here, the file selecting will be restricted to the Tests directory. The glob will match the files such as Tests/HelloWorld.js, Tests/UI/HelloWorld.js, Tests/UI/Feature1/HelloWorld.js.

---

Question Mark(?)

Question mark(?) matches a single character within one segment. When some files or directories differ in their name by just one character, you can use the ?.

Example: tests/?at.js

This will match files such as tests/cat.js, test/Cat.js, test/bat.js etc.

---

Square Brackets ([abc])

Square Brackets ([...]) globs the files with a single character mentioned in the square brackets.

Example: tests/[CB]at.js

This glob will match files like tests/Cat.js or tests/Bat.js

---

Square Brackets Range ([a-z])

Square Brackets Range ([a-z]), matches one character specified in the range.

Example: tests/feature[1-9]/HelloWorld.js

This glob will match files like tests/feature1/HelloWorld.js, test/feature2/HelloWorld.js and so on... upto 9.

---

Negation (!)

Negation (!) can be used for excluding some files.

Example 1: tests/[!C]at.js

This will exclude the file tests/Cat.js and will match files like tests/Bat.js, tests/bat.js, tests/cat.js.

Negation is also used in configuration files inside an array to negate or exclude some files.

Example 2: ['Tests/**/*.js', '!Tests/UI/**']

This will exclude all files and folders from Tests/UI directory.

---

That's it! Hope that helps!

Answer 3

The exact behavior of this particular wildcard has been well covered by the other answers, but information on the general case may be useful.

This behavior is not limited to ls, and is referred to as "globbing", which is the expansion of patterns based on matches with existing filenames. It is important to note that these patterns do not use regular expression syntax.

The shell pre-processes the arguments before they are sent to the program. There are generally multiple levels of expansion, some of these involve globbing.

A great resource for more information on the other wildcards available in a file glob pattern is the unix manpage. A online version for glob can be found here.

Finally, a simple example of what this can do for you, especially when combined with other shell expansion goodies, in this case those provided by the bash shell. Information about the expansions used in this example can be found in the Bash Guide for Beginners - which is my goto resource, despite the title.

ls *{01..04}.{txt,csv} becomes ls *01.txt *01.csv *02.txt *02.csv *03.txt *03.csv *04.txt *04.csv

Which could output something like this:

input_01.txt input_02.txt input_03.txt input_04.txt output_01.csv output_02.csv output_03.csv output_04.csv

While skipping these:

input_05.txt input_06.txt input_07.txt input_08.txt input_09.txt input_10.txt output_05.csv output_06.csv output_07.csv output_08.csv output_09.csv output_10.csv

A trivial example, but if you know that this behavior is not specific to ls, then you can imagine the utility when coupled with mv, cp, rsync, etc.