.gitignore whitelist on directory and its contents

Question

I'm trying to whitelist the directory (and its contents) SupplierName in my Zend Framework 2 vendor directory.

The original .gitignore file in /vendor looks like this:

# Add here the vendor path to be whitelisted
# Ex: for composer directory write here "!composer" (without quotes)
!.gitignore
*

Now I'd like to whitelist the directory SupplierName which shouldn't be too hard I thought. I have read the docs on gitignore and tried the following configurations:

First try, add !SupplierName right after the comment which says that I have to add the whitelisted path here.

# Add here the vendor path to be whitelisted
!SupplierName
# Ex: for composer directory write here "!composer" (without quotes)
!.gitignore
*

Right after that I executed git status which didn't show the vendor/SupplierName directory. git add vendor/SupplierName showed the following message:

The following paths are ignored by one of your .gitignore files: vendor/SupplierName

Second try

# Add here the vendor path to be whitelisted
# Ex: for composer directory write here "!composer" (without quotes)
!SupplierName
!.gitignore
*

Right after that I executed git status which didn't show the vendor/SupplierName directory. git add vendor/SupplierName showed the following message:

The following paths are ignored by one of your .gitignore files: vendor/SupplierName

Third try

# Add here the vendor path to be whitelisted
# Ex: for composer directory write here "!composer" (without quotes)
!.gitignore
*
!SupplierName

Right after that I executed git status which didn't show the vendor/SupplierName directory. git add vendor/SupplierName seems to work. But now, when I want to add the Module.php file (and some other files, subdirectories, etc) the following happens. git add vendor/SupplierName/Module.php -->

The following paths are ignored by one of your .gitignore files: vendor/SupplierName/Module.php

# Add here the vendor path to be whitelisted
# Ex: for composer directory write here "!composer" (without quotes)
*
!.gitignore
!SupplierName
!SupplierName/
!SupplierName/*

Allows me to add files directly in vendor/SupplierName, but git add vendor/SupplierName/config/module.config.php still results in

The following paths are ignored by one of your .gitignore files: vendor/SupplierName/config/module.config.php

I've been searching for problems regarding recursive whitelisting, because that seems to be the problem, but nothing came up.

Answer 1

You can use 2 .gitignore files to achieve the desired result:

# vendor/.gitignore
*
!.gitignore
!SupplierName/
!SupplierName/*

# vendor/SupplierName/.gitignore
!*

I tested this with a test repo and seems to work for me in adding files as many levels deep underneath the vendor/SupplierName directory.

$ git add .

$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#   modified:   vendor/.gitignore
#   new file:   vendor/SupplierName/.gitignore
#   new file:   vendor/SupplierName/a
#   new file:   vendor/SupplierName/b
#   new file:   vendor/SupplierName/c
#   new file:   vendor/SupplierName/d
#   new file:   vendor/SupplierName/dir1/d
#   new file:   vendor/SupplierName/dir1/dir4/dir5/dir6/dir7/dir8/dir9/dir10/somefile
#   new file:   vendor/SupplierName/dir1/dir4/f1
#   new file:   vendor/SupplierName/dir1/dir4/f2
#   new file:   vendor/SupplierName/dir1/dir4/f3
#   new file:   vendor/SupplierName/dir1/dir4/f4
#   new file:   vendor/SupplierName/dir1/e
#   new file:   vendor/SupplierName/dir1/f
#   new file:   vendor/SupplierName/dir3/dir6/f5
#   new file:   vendor/SupplierName/dir3/dir6/f6
#   new file:   vendor/SupplierName/dir3/dir6/f7
#   new file:   vendor/SupplierName/dir3/dir7/f8
#   new file:   vendor/SupplierName/e
#

Answer 2

You can also achieve this with only one .gitignore file (in your project root):

/*
!/.gitignore
!/vendor
/vendor/*
!/vendor/SupplierName

Answer 3

Found an interesting article: https://jasonstitt.com/gitignore-whitelisting-patterns

All credits to Jason Stitt. Texts are copied from the site above:

Ignore everything, then add specific subtrees

# Ignore everything
*
# But descend into directories
!*/
# Recursively allow files under subtree
!/subtree/**
# You can be specific with these rules
!/some/other/deep/path/**
!.gitignore 

The !*/ rule un-ignores all directories. But Git does not track directories, only files, so !*/ by itself will only allow descent into the full directory tree; it won’t actually allow anything into the repo. With that rule in place, you only need one rule using the ** recursive wildcard in order to include a subtree.

If you didn’t use !*/, you would need additional rules to un-ignore /subtree/ and its child directories.

Not everyone likes !*/ because it means that if any other rule allows a filename pattern found inside some directory you don’t want in the repo, the directory itself will not be blocked. You need to use specific rules for files to include with this one.

Ignore the root directory, then add whole subtrees

# Ignore everything in the root
/*
# Un-ignore all of subtree
!/subtree/
!.gitignore 

This pattern is somewhat coarser than the previous one. The /* rule will only ignore items in the root of the repo’s directory structure, so as soon as you whitelist a directory, all of the directory’s contents will be allowed as well, even without using the * or ** wildcards.

Ignore everything in a directory, but keep the empty directory

*
!.gitignore

Git does not want to include an empty directory in a repo, because it tracks files. Put a hidden file (such as .gitignore) into the directory, and it will be saved. But to keep the directory empty, even if you have files in there for testing/development purposes, it’s a good idea to ignore everything except for the .gitignore file itself.

Answer 4

I had a similar issue when moving from CVS to Git.

Unlike CVS, Git doesn't look at directories, it focuses on files.

For example you can't not-ignore directory "a" but you can not-ignore all files in directory "a" like so: !a/*

The same is true for subdirectories.

If directory "a" has a subdirectory "b" and you ignore "!a/*" then you will still get all files in "a/b".

So you then have to ignore that too "!a/b/*" and so on for all subdirectories that you want to white list.

You only need one .gitignore file.

so you end up with something like:

# ignore everything
*
# except for .gitignore files in the current directory
!.gitignore
# and all files in directory a
!a/*
#and all files in subdirectory b
!a/b/*

With this you would still get files from a/c and a/b/c. I'm not sure if there is a workaround for recursion down subdirectories.

Answer 5

You should include everything in the blacklist first, then make every directory and subdirctory in the whitelist. For example, I only want to put DIR /opt/source/, DIR /opt/nginx/ and FILE /home/test/.opt/hello.txt in whitelist, could write .gitignore file like this to make it work:

/*
!/.gitignore
!/opt
/opt/*
!/opt/source/
!/opt/nginx/
!/home
/home/*
!/home/test
/home/test/*
!/home/test/.opt
/home/test/.opt/*
!/home/test/.opt/hello.txt

Answer 6

I've created a simple JS snipscript that can run in Node to generate a whitelist rule as I found manually writing rules a bit confusing and I wanted to be able to modify the rule later if I forgot how to hand-write it.

'use strict';

// Generating a "whitelist" wherein you only want a specific folder to be
// affected requires following .gitignore-style rules.
// https://github.com/prettier/prettier/issues/3328
//
// Handcrafting these rules is hard to reason about and error-prone, so I'm
// going to generate them.
// See: https://github.com/prettier/prettier/issues/3328
// And: https://git-scm.com/docs/gitignore
//

const path = require('path');

const whitelistDir = '/themes/simple/src/';

function generateIgnoreRule(dir) {
    let output = '# Auto-generated by ' + path.basename(__filename) + '\n';
    output += '# Exclude everything except `' + dir + '`\n';
    // Add exclude everything rule
    output += '/*' + '\n';

    // Split by path
    const parts = dir.split('/');
    if (parts[0] === '') {
        // Remove first item if its blank
        parts.shift();
    }
    if (parts[parts.length - 1] === '') {
        // Remove last item if its blank
        parts.pop();
    }

    let totalPart = '';
    for (let part of parts) {
        totalPart += '/' + part;
        output += '!' + totalPart + '\n';
        if (part !== parts[parts.length - 1]) {
            output += totalPart + '/*' + '\n';
        }
    }
    return output;
}

console.log(generateIgnoreRule(whitelistDir));
console.log(
    '\nCopy the above rules out of the console output and paste into your .gitignore / .prettierignore'
);

Answer 7

This is related to the first technique from @Simon Lang (with credit to Jason Stitt), but is applicable to a wider variety of cases.

Suppose you want to have just one .gitignore in your project root, and suppose you want to ignore the bulk of some subdirectory of your project, but allow in some small portion. For example, suppose your repo includes a /vendor/ directory with a 3rd party dependency that you've elected to keep largely intact, but to which you'll make some light adjustments that you want git to track, perhaps to assist with back porting when the inevitable new version appears.

You can then do the following in your root .gitignore file:

# ignore all files and folders in `/vendor/` and in its descendants
/vendor/**

# negated pattern to un-exclude all *folders* at any depth relative to `vendor/`
# as always git will ultimately ignore the folders for which you don't
# subsequently un-exclude any *files*.
!/vendor/**/

# negated pattern to un-exclude all files and folders at any depth below the deep
# path that contains files you want in git
!/vendor/supplier/version/module/component/src/**

Each instance of the double asterisk is critical. For example, a trailing vendor/** matches everything inside -- all files and directories in vendor with infinite depth, and so excludes all of its descendants.

In a "normal" wildcard gitignore rule like *.exe or *, the glob metacharacter match discovers any file in any directory, since the path is unconstrained. The path being unconstrained is doing the heavy lifting here, not the wildcard expansion. As soon as you try to restrict the scope of the ignore to a subdirectory of your root, that heavy lifter is gone and you immediately bump into the inability of * to descend into the tree.

Simon's answer works by keeping the tip of the initial exclude path unconstrained. It initially excludes every file (at any depth) in the directory that contains the .gitignore file. Mine gives you the flexibility to apply this exclude-all approach to subdirectories, by using ** to do globstar matching of /. You could achieve the same effect by having multiple .gitignore files and using Simon's technique in a .gitignore file located at /vendor/.gitignore, but that's more to maintain.

I don't know if there are any performance implications to using ** as I've done here. That might be testable by comparing the performance of a .gitignore that looked like

*
!*/
!a/b/c/d/**

to one that looked like this

/**
!**/
!a/b/c/d/**

in a tree having a lot of files. I believe the same content would be included and excluded, but the implementation under the hood may differ, as may the performance.