Encode a video into h264 using bufferedimages?

Question

Im attempting to translate a large set of bufferedimages (pre-saved images created on the fly by my application) into a video using java and hopefully a library that can help with the process.

I've explored a number of different options, such as jcodec (there was no documentation on how to use it). Xuggler (couldn't get it to run due to compatibility issues with jdk5 and its related libraries). and a number of other libraries that had very poor documentation.

I'm trying to find a library that I can use that uses java to (1) create h264 videos by writing bufferedimages frame by frame and (2) has documentation so that I can actually figure out how to use the dam thing.

Any ideas on what I should be looking into?

If pure java source code exists somewhere that can achieve this I would be VERY interested in seeing it. Because I would love to see how the person has achieved the functionality and how I could use it!

Thanks in advance...

Answer 1

Here's how you can do it with JCodec:

public class SequenceEncoder {
    private SeekableByteChannel ch;
    private Picture toEncode;
    private RgbToYuv420 transform;
    private H264Encoder encoder;
    private ArrayList<ByteBuffer> spsList;
    private ArrayList<ByteBuffer> ppsList;
    private CompressedTrack outTrack;
    private ByteBuffer _out;
    private int frameNo;
    private MP4Muxer muxer;

    public SequenceEncoder(File out) throws IOException {
        this.ch = NIOUtils.writableFileChannel(out);

        // Transform to convert between RGB and YUV
        transform = new RgbToYuv420(0, 0);

        // Muxer that will store the encoded frames
        muxer = new MP4Muxer(ch, Brand.MP4);

        // Add video track to muxer
        outTrack = muxer.addTrackForCompressed(TrackType.VIDEO, 25);

        // Allocate a buffer big enough to hold output frames
        _out = ByteBuffer.allocate(1920 * 1080 * 6);

        // Create an instance of encoder
        encoder = new H264Encoder();

        // Encoder extra data ( SPS, PPS ) to be stored in a special place of
        // MP4
        spsList = new ArrayList<ByteBuffer>();
        ppsList = new ArrayList<ByteBuffer>();

    }

    public void encodeImage(BufferedImage bi) throws IOException {
        if (toEncode == null) {
            toEncode = Picture.create(bi.getWidth(), bi.getHeight(), ColorSpace.YUV420);
        }

        // Perform conversion
        transform.transform(AWTUtil.fromBufferedImage(bi), toEncode);

        // Encode image into H.264 frame, the result is stored in '_out' buffer
        _out.clear();
        ByteBuffer result = encoder.encodeFrame(_out, toEncode);

        // Based on the frame above form correct MP4 packet
        spsList.clear();
        ppsList.clear();
        H264Utils.encodeMOVPacket(result, spsList, ppsList);

        // Add packet to video track
        outTrack.addFrame(new MP4Packet(result, frameNo, 25, 1, frameNo, true, null, frameNo, 0));

        frameNo++;
    }

    public void finish() throws IOException {
        // Push saved SPS/PPS to a special storage in MP4
        outTrack.addSampleEntry(H264Utils.createMOVSampleEntry(spsList, ppsList));

        // Write MP4 header and finalize recording
        muxer.writeHeader();
        NIOUtils.closeQuietly(ch);
    }
}

Answer 2

jcodec now (jcodec-0.1.9.jar) includes SequenceEncoder that directly permits writing of BufferedImages to a video stream.

I spent a while fixing the default import of this new class into Eclipse. After removing the first import, attempting (as I said above, I could not locate some of the classes) to create my own using Stanislav's code and reimporting I spotted the mistake:

import org.jcodec.api.awt.SequenceEncoder;
//import org.jcodec.api.SequenceEncoder;

The second is completely deprecated with no documentation directing me to the latter.

The commensurate method is then:

private void saveClip(Trajectory traj) {
    //See www.tutorialspoint.com/androi/android_audio_capture.htm
    //for audio cap ideas.
    SequenceEncoder enc;
    try {
        enc = new SequenceEncoder(new File("C:/Users/WHOAMI/today.mp4"));
        for (int i = 0; i < BUFF_COUNT; ++i) {
            BufferedImage image = buffdFramToBuffdImage(frameBuff.get(i));
            enc.encodeImage(image);
        }
        enc.finish();
    } catch (IOException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}