I realize that the align.plots function from the ggExtra package has been deprecated and removed. However, I am using my own version as it seems to
ggplot2 now has ggplotGrob(), which may help with this.
First, we need to update the code used to generate the plots:
p1a = ggplot(df2, aes(x=x, y=y, colour=group)) + geom_point() +
scale_x_continuous(limits=range) + scale_y_continuous(limits=range)
p1 = p1a + coord_equal(ratio=1) + theme_minimal() + theme(legend.position='none')
p2 = ggplot(df, aes(x=x, y=group, colour=group)) + geom_point() +
scale_x_continuous(limits=range) + theme_minimal() + theme(legend.position='none')
p3 = ggplot(df, aes(x=group, y=y, colour=group)) + geom_point() +
scale_y_continuous(limits=range) + theme_minimal() + theme(legend.position='none')
p4 <- ggplot(df, aes(x = group, y = y)) +
geom_blank() +
theme(line = element_blank(),
rect = element_blank(),
text = element_blank(),
title = element_blank())
p4 will be blank; we just need the grob to draw.
Then we load the grid package and draw the grobs in a list arranged in rows and columns using cbind() and rbind().
library(grid)
grid.newpage()
grid.draw(
cbind(
rbind(ggplotGrob(p3), ggplotGrob(p4), size = "first"),
rbind(ggplotGrob(p1), ggplotGrob(p2), size = "first"),
size = "first"))

I'm not sure if this method will let you plot p3 in a different width and p2 in a different height, as you have them in the original example; I normally need a grid of similarly-sized graphs, and haven't needed to figure out different sizes.
This answer is partially based on partially based on https://stackoverflow.com/a/17463184/393354