Why does CSS Grid layout add extra gaps between cells?

Question

Can’t figure out why CSS grid layout adds an unwanted extra space to content around the vertical cells, though margin and padding are zeroed out:

Answer 1

Note, this answer where posted prior to the latest update, and as such, answer the initial code sample (the one within the question) differently. The edited version of the question is brilliantly answered by Michael.

First, the inner space has nothing to do with CSS Grid, margin or padding, it comes from the fact that an img is an inline element, which like characters has a white space, for descenders, below the baseline.

One solution for that is to add display: block to the .grid figure img rule.

.grid figure img {
  display: block;
  width: 100%;
}

---

Second, the outer space is caused by the grid-auto-rows, so remove that from the .grid rule.

.grid {
  display: grid;
  grid-template-columns: 13fr 11fr 4fr 20fr;
  /*grid-auto-rows: 12fr;                      remove this  */
  grid-gap: 3px;
  align-items: flex-start;
}

Stack snippet

.grid {
  display: grid;
  grid-template-columns: 13fr 11fr 4fr 20fr;
  /*grid-auto-rows: 12fr;                      removed  */
  grid-gap: 3px;
  align-items: flex-start;
}
.grid figure {
  outline: 1px solid red;
  margin: 0;
  padding: 0;
}
.grid figure img {
  width: 100%;
  display: block;
}
.grid .gi13x12 {
  grid-column-start: 1;
  grid-column-end: 2;
  grid-row-start: 1;
  grid-row-end: 13;
}
.grid .gi11x6.one {
  grid-column-start: 2;
  grid-column-end: 3;
  grid-row-start: 1;
  grid-row-end: 7;
}
.grid .gi11x6.two {
  grid-column-start: 2;
  grid-column-end: 3;
  grid-row-start: 7;
  grid-row-end: 13;
}
.grid .gi4x4.one {
  grid-column-start: 3;
  grid-column-end: 4;
  grid-row-start: 1;
  grid-row-end: 5;
}
.grid .gi4x4.two {
  grid-column-start: 3;
  grid-column-end: 4;
  grid-row-start: 5;
  grid-row-end: 9;
}
.grid .gi4x4.three {
  grid-column-start: 3;
  grid-column-end: 4;
  grid-row-start: 9;
  grid-row-end: 13;
}
.grid .gi20x12 {
  grid-column-start: 4;
  grid-column-end: 5;
  grid-row-start: 1;
  grid-row-end: 13;
}

---

Regarding your Fiddle sample (and added screen dump), where neither matching the inner code sample, and after the above adjustments, there is still a small inner space, which is caused by the fact that the sum of the smaller elements height does not match the highest one, nor does its images, as the highest image is 123px, but smaller images are 120px (2*60) and 117px (3*39).