Pandas groupby multiple aggregations with exclusion of the focal group

Question

I have a toy dataset like the one below.

    Building    Department  feature1    feature2
0   A   1   14  28
1   A   1   11  26
2   A   1   29  19
3   A   2