How to add external WebVTT subtitles into HTTP Live Stream on iOS client

Question

We have videos encoded via bitmovin.com and provided as HTTP Live Streams (Fairplay HLS), but subtitles although in WebVTT format are exposed separately as direct URLs for the whole file, not individual segments and are not part of the HLS m3u8 playlist.

I am looking for the way how an external .vtt file downloaded separately can still be included in the HLS stream and be available as a subtitle in AVPlayer.

I know Apple's recommendation is to include segmented VTT subtitles into the HLS playlist, but I can't change the server implementation right now, so I want to clarify if it is even possible to provide the subtitle to AVPlayer to play along with the HLS stream.

The only valid post on this subject claiming it is possible is this: Subtitles for AVPlayer/MPMoviePlayerController. However, the sample code loads local mp4 file from bundle and I am struggling to make it work for m3u8 playlist via AVURLAsset. Actually, I am having problem to get videoTrack from the remote m3u8 stream as the asset.tracks(withMediaType: AVMediaTypeVideo) returns empty array. Any ideas if this approach can work for real HLS stream? Or is there any other way to play separate WebVTT subtitle with HLS stream without including them into HLS playlist on the server? Thanks.

func playFpsVideo(with asset: AVURLAsset, at context: UIViewController) {

    let composition = AVMutableComposition()

    // Video
    let videoTrack = composition.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: kCMPersistentTrackID_Invalid)

    do {

        let tracks = asset.tracks(withMediaType: AVMediaTypeVideo)

        // ==> The code breaks here, tracks is an empty array
        guard let track = tracks.first else {
            Log.error("Can't get first video track")
            return
        }

        try videoTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, asset.duration), of: track, at: kCMTimeZero)

    } catch {

        Log.error(error)
        return
    }


    // Subtitle, some test from the bundle..
    guard let subsUrl = Bundle.main.url(forResource: "subs", withExtension: "vtt") else {
        Log.error("Can't load subs.vtt from bundle")
        return
    }

    let subtitleAsset = AVURLAsset(url: subsUrl)

    let subtitleTrack = composition.addMutableTrack(withMediaType: AVMediaTypeText, preferredTrackID: kCMPersistentTrackID_Invalid)

    do {

        let subTracks = subtitleAsset.tracks(withMediaType: AVMediaTypeText)

        guard let subTrack = subTracks.first else {
            Log.error("Can't get first subs track")
            return
        }

        try subtitleTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, asset.duration), of: subTrack, at: kCMTimeZero)

    } catch {

        Log.error(error)
        return
    }


    // Prepare item and play it
    let item = AVPlayerItem(asset: composition)

    let player = AVPlayer(playerItem: item)

    let playerViewController = AVPlayerViewController()
    playerViewController.player = player

    self.playerViewController = playerViewController

    context.present(playerViewController, animated: true) {
        playerViewController.player?.play()
    }
}

Answer 1

I wrote a little post about this which links to some proof of concept on GitHub since I had to deal with a similar issue. Hope this helps.

https://jorisweimar.com/programming/supporting-external-webvtt-subtitles-in-avplayer/