FFMPEG: Overlaying one video on another one, and making black pixels transparent

Question

I\'m trying to use FFMPEG to create a video with one video overlayed on top another.

I have 2 MP4s. I need to make all BLACK pixels

Answer 1

The notional way to do this is to chroma-key the black out and then overlay, But as @MoDJ said, this likely won't produce satisfactory results. Neither will the method I suggest below, but it's worth a try.

ffmpeg -i 1.mp4 -i 2.mp4 -filter_complex
"[1]split[m][a];
 [a]geq='if(gt(lum(X,Y),16),255,0)',hue=s=0[al];
 [m][al]alphamerge[ovr];
 [0][ovr]overlay"
output.mp4

Above, I duplicate the overlay video stream, then use the geq filter to manipulate the luma values so that any pixel with luma greater than 16 (i.e. not pure black) has its luma set to white, else zero. Since I haven't provided expressions for the two color channels, geq falls back on the luma expression. We don't want that, so I use the hue filter to nullify those channels. Then I use the alphamerge filter to merge this as an alpha channel with the first copy of the overlay video. Then, the overlay. Like I said, this may not produce satisfactory results. You can tweak the value 16 in the geq filter to change the black threshold. Suggested range is 16-24 for limited-range (Y: 16-235) video files.