How Do We Stream Movies to Apple TV in HUDTube?

One of the main features of HUDTube 1.5 is the ability to stream basically any type of movie to Apple TV. Unfortunately, Apple TV understands a very limited number of formats. In general, you could assume that if something plays in QuickTime Player (without any additional codecs installed, like Perian or Flip4Mac), it should also play on AppleTV. This basically leaves MPEG-2 and H.264 as the only supported formats. As a result, a lot of media created using codecs like DivX, XviD, etc. are not supported. The situation is a bit better with audio, because most movies use either MP3 or AAC. If you add to this mix many different containers (one of the most popular being AVI), you may have an Apple TV streaming problem with many types of media from outside of iTunes.

Some of the solutions (also suggested on Apple’s Support page) work by converting the movie of your choice to a format that is supported by Apple TV, and although that’s an option, it can take a lot of time to change a file into a usable format. While it heavily depends on settings, it’s hard to get a much shorter conversion time than the duration of the movie itself without huge quality tradeoffs, and even then, the amount of time it takes just to prepare the movie means that you have to think at least one ro two hours in advance before you could watch an average-length movie.

But there’s an alternative to this—live streaming. It’s similar to the re-encoding you would do for the entire movie, but doing it live, while watching parts of the movie that have already been re-encoded. The best and easiest way to do this is to use Apple’s own proposed format—HTTP Live Streaming (or HLS for short). It allows you to split the content into convenient chunks, which are played and treated just like a whole movie, but can be re-encoded one at a time, when requested, instead of re-encoding all of them ahead of time.

The HLS has a quite few advantages over other methods of streaming, like the fact that it’s easier to use with firewalls and proxy servers, but from the developer’s standpoint, it’s good because it’s simple. The protocol is well-documented since it just uses standard HTTP transactions while still providing most options you would need for streaming. There are only three basic calls that have to be handled to make use of HLS:

  1. Request for an index file with available streams
  2. Request for the playlist for the selected stream
  3. Requests for single chunks of movie stream

Index file

The index file is also known in Apple’s documentation draft as the Master Playlist file. It contains a list of available alternative streams for the same content. Streams can be separated depending on resolution, bandwidth, or/and to provide redundant servers in case of failure of a previously used server. Clients should be programmed in a way to choose the best stream that fits its need and capabilities. He can also change playlists if the playlist or files in it fail to load.

An example of such a file would be (providing two separate servers for each resolution/bandwidth):

#EXTM3U

#EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=200000, RESOLUTION=720x480
http://alpha.mycompany.com/lo/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=200000, RESOLUTION=720x480    
http://beta.mycompany.com/lo/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=500000, RESOLUTION=1920x1080
http://alpha.mycompany.com/md/prog_index.m3u8

#EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=500000, RESOLUTION=1920x1080
http://beta.mycompany.com/md/prog_index.m3u8

You can also include specifically which codecs are used, separated by comma. This way, if you want to inform that the stream you’re providing has its audio encoded with AAC-LC and video content encoded with H.264 High Profile level 4.0, you would have to add CODECS stream info.

#EXT-X-STREAM-INF:PROGRAM-ID=1, BANDWIDTH=300000,CODECS="mp4a.40.2,avc1.640028"

The more information you provide about the stream, the better the decision the client will be able to make when choosing a stream to play.

Playlist File

Index file points to a playlist file that consists of a list of chunks that contains the whole movie. It’s an Extended M3U Playlist file in which you can provide information about the whole list and specific files in that playlist. The minimal amount of information provided by this playlist consists of:

#EXTM3U

which just defines playlist format and always has to be the first line of file;

#EXT-X-TARGETDURATION: 5

which provides the maximal duration of a single segment on that list (not the duration of the whole playlist, which is calculated by just summing up the duration of all segments on that playlist!);

#EXT-X-PLAYLIST-TYPE:VOD

which defines the type of playlist—EVENT if you’re actually live-streaming and the stream isn’t complete yet or VOD if the playlist describes content that is already available as a whole;

#EXT-X-ALLOW-CACHE:YES

which gives information about caching and states whether the client may or may not cache the content that is downloading;

#EXTINF:5,

/stream/file1.ts

which describes a single chunk—EXTINF contains two parameters: the duration of the chunk in seconds (here five seconds) and its title after the comma (which is optional, as in provided example). The second line contains the path to the file itself. This part is repeated for all segments available for the selected stream;

#EXT-X-ENDLIST

which informs that the downloaded version is the final version of the playlist. If this line isn’t present, the client may try to refresh the playlist at any given time to check if any new information or playlist items are available. If the list lacks this line, it tells the client that the list isn’t complete and he should try to reload it until the list with the #EXT-X-ENLIS is received.

Encoded file

Lastly, we have to provide the re-encoded chunk of movie data. Audio and video codecs used to make each chunk should reflect those provided in the index file itself. The container for multimedia data should be MPEG Transport Stream. 

In HUDTube 1.5, we cut the movie into segments of the same length. You should keep those fairly small, and Apple suggests to keep them under or around 10 seconds per chunk, which should provide a nice balance for most broadcasting needs. The reason for keeping those segments small is that those files have to be downloaded (just like a regular file over HTTP) by the client to start playing it. The longer the chunks are, the longer it will take to buffer the first one (or first few of them) to start playing. Of course, you should keep in mind the overhead of starting the re-encoding process for each chunk—that’s why making one-second segments won’t be the best idea in most cases. In our tests, five seconds turned out to be a good balance to have quick renders and to be able to stream as quickly as possible. Also keep in mind that the playlist is visible by the user as a normal movie, not as a list of five-second files, and as such, it’s natural to rewind/fast forward to specific place in the movie. Chunks are not being loaded along the process, but the longer the chunks are, the longer the buffering will take when user presses “Play” again—even if you try to render it ahead of time when you have free resources available (which we do in HUDTube).

The process, tips, and tricks of actually re-encoding the movie, splitting into chunks, and choosing and adapting options for that process will be covered soon in a separate post, so stay tuned!

Finishing touches

Once you have that done, you should have a functional streaming service. During the development, we’ve built a working prototype of this in Node.js (for rapid development and not worrying about server implementation), and when we had that working and tested, we built it on top of Objective-C HTTP server implementation. You can test those streams by using QuickTime Player on your Mac or even HTML5 video tag on Safari. Pushing that content to Apple TV is a bit more complicated since Apple didn’t release any official API for Apple TV and there’s only unofficial documentation of a web-based protocol used to partially control the device.

In general it involves discovering your Apple TV via Bonjour, making two connections to the device (the second one is used by the server to push information to client), logging in if it’s password protected, and pushing the HTTP address of our streaming server (path to the index path described above), which it should play.

The rest will be handled by Apple TV itself.

Now you know one of our secrets (with more to come). If you want to see it in action, go and get a fresh copy from the Mac App Store, and watch some movies on your Apple TV.

Curvatures and iOS7 Icon Shape
Understanding Frame