More Video Processing

HOWTO: converting mkv files to play on the PS3

(Originally found at Julien Simon's website.)

There's no doubt that Matroska is a great open source container that can store anything (and the kitchen sink too). Unfortunately, as of today, it's not well supported by consumer devices. So, unless you decide to use your computer for viewing, you need to convert mkv files into something that your device can play.

The purpose of this article is to show how to do that for the PS3. It may very well work on other devices, but don't blame if it doesn't :)

This article is a work in progress. Are covered so far:

• Example #1: (mkv file, H.264, AC3) --> (MPEG-2 TS file, H.264, AC3)
• Example #2: (mkv file, H.264, AC3) --> (MPEG-2 PS file, H.264, AC3)
• Example #3: (mkv file, H.264, AC3) --> (MP4 file, H.264, AAC)
  
• Example #4: (mkv file, H.264 level 5.1, DTS 5.1) --> (MP4 file, H.264 level 4.1, AAC)
  
• Example #5: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3)
• Example #6: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3 5.1)
• Example #7: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MP4 file, H.264 video, AAC)

1) Pre-requisites

Before we start, let's prepare our toolbox:

• mediainfo : the best tool to learn everything there is know on the true nature of your media files. I showed you how to build it in a previous article.
  
• mkvtoolnix : a collection of tools to inspect, extract and build mkv files.
  
ubuntu% sudo apt-get install mkvtoolnix
• ffmpeg : the best general-purpose audio & video transcoding tool. I also showed you how to build it in a previous article.
  
• MP4Box : a nice MP4 manipulation tool, which we'll use for multiplexing.
  
  ubuntu% sudo apt-get install gpac

• tsMuxer : another great tool for multiplexing. You can download a pre-compiled version here. Just copy the tsMuxer binary in /usr/local/bin.

2) Audio & video formats supported by the PS3
Let's take at a look at what the PS3 actually supports, That way, we will avoid any unnecessary transcoding. Remember that the content we're dealing has already been compressed, so any additional processing would certainly degrade quality. Also, video encoding takes a LOT of time, especially with HD files...

According to the online manual, the PS3 supports:

• MPEG-1 video, with MPEG-1 layer 2 audio
  
• MP4 container : H.264/MPEG-4 AVC High Profile video, with AAC LC audio
  
• MPEG-2 PS container (aka VOB file): MPEG-2 video, with either one of these audio formats : MPEG-2 Audio Layer 2, AAC LC, AC3 (Dolby Digital), LPCM audio
  
• MPEG-2 TS container :
• MPEG-2 video, with either one of these audio formats : MPEG-2 Audio Layer 2, AAC LC, AC3 (Dolby Digital)
• H.264/MPEG-4 AVC video, with AAC LC audio

• AVCHD, AVI, DivX and VC-1 (WMV), which are less likely to be encountered in mkv files.

So what does this mean? Well, here are a few rules:

• Obviously, the PS3 does not support the mkv container. Any input file in this format must be at least remuxed.
  
• The PS3 does support DTS audio, but only when played from actual disks, i.e. not from files. Thus, any input stream in this format must be transcoded.
  
• MP3 audio, Vorbis audio and so on are not supported and must be transcoded.
  
• If you want AC3 audio, you must use a MPEG-2 TS/PS container.
  
• If you want to stick with the MP4 container, you really have no options but H.264 video and AAC audio.

3) Getting things done
Now that we know what is supported and what isn't, we can pick the right solution, i.e. preserve the quality of the input file and save time.

The approach will always be the same:

• inspect the mkv file and identify the nature of its streams,
  
• extract the video stream and the audio stream (possibly among several available ones)
  
• depending on what your audio setup supports, select a format for the audio output (AC3 will do you no good if you don't have a Dolby Digital setup)
  
• based on the previous item and on what the PS3 supports, transcode the audio and/or video stream
  
• remux the audio and the video stream into a container supported by the PS3
  
• play the file :)

Let's look at real-life examples and sort them out!

4) Example #1: (mkv file, H.264, AC3) --> (MPEG-2 TS file, H.264, AC3)
Let's look at our input file with mediainfo (edited for brevity):

ubuntu% mediainfo video.mkv
General
Complete name : video.mkv
Format : Matroska
File size : 1.09 GiB
Duration : 41mn 48s
Overall bit rate : 3 740 Kbps

Video
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L3.1
Codec ID : V_MPEG4/ISO/AVC
Width : 1 280 pixels
Height : 720 pixels
Frame rate : 23.976 fps

Audio
Format : AC-3
Format/Info : Audio Coding 3
Codec ID : A_AC3
Channel(s) : 6 channels
Channel positions : Front: L C R, Surround: L R, LFE
You can also use mkvinfo (part of the mkvtoolnix package):

ubuntu% mkvinfo video.mkv
| + Track number: 1
| + Track type: video
| + Codec ID: V_MPEG4/ISO/AVC
| + Video track
| + Pixel width: 1280
| + Pixel height: 720

| + Track number: 2
| + Track type: audio
| + Codec ID: A_AC3
So what is this file? It's a 2-track mkv file: track 1 is H.264 video (1280x720), track 2 is AC3 6-channel audio. Both formats are supported by the PS3, and since they're high quality it would be a shame to transcode them. Let's try a simple remuxing into... no, not MP4 because it doesn't support AC3: we have to use MPEG-2 TS.

This is where tsMuxer comes in. The good thing about it is that we don't have to extract the individual streams, this will be handled for us.

Although tsMuxer is a command-line tool, it can only read its parameters from a file (let's call it tsmuxer.meta):

MUXOPT --no-pcr-on-video-pid --new-audio-pes --vbr
V_MPEG4/ISO/AVC, video.mkv, level=4.1, insertSEI, contSPS, track=1, lang=eng
A_AC3, video.mkv, track=2, lang=eng

Ignore the first line and look at the next two. On each of them, we're describing a stream and telling tsMuxer what format it is and where to find it (file name and track number). Don't worry, this information is dumped by mediainfo or mkvinfo (go back and check), so you shouldn't have any problem writing the file :)

Now, we're ready to remux:

ubuntu% tsMuxeR tsmuxer.meta video.m2tsoutput removed
Mux successful complete.
Muxing time: 5 min 25 sec
That's all: this file plays fine on the PS3 (I'm streaming it right now using mediatomb). Total time: about 10 minutes. And we preserved the original quality of the input file.

5) Example #2: (mkv file, H.264, AC3) --> (MPEG-2 PS file, H.264, AC3)
Here, we'll use the same input file as example #1, but we'll remux it to MPEG2-PS (aka the VOB format). This is also interesting if you want to do without tsMuxer.
Wouldn't it be nice to be able to just do this?

ubuntu% ffmpeg -i video.mkv -acodec copy -vcodec copy -r 23.976 -f vob video.mpg[NULL @ 0x8076d00]error, non monotone timestamps 15030 >= 7470
av_interleaved_write_frame(): Error while opening file

As you can see, it doesn't work. Not sure why... Let's use a two-step solution instead. First, let's extract the streams from the mkv file:

ubuntu% mkvextract tracks video.mkv 1:video.h264 2:audio.ac3
Now, let's remux the streams into a MPEG2-PS container:

ubuntu% ffmpeg -i video.h264 -i audio.ac3 -map 0.0:0 -map 1.0:0 -acodec copy -vcodec copy -r 23.976 -f vob video.mpg
That's it. This solution is a little bit longer than the previous one, but it works and doesn't require you to write the tsMuxer meta file :)
6) Example #3: (mkv file, H.264, AC3) --> (MP4 file, H.264, AAC)
Here, we'll also use the same input file as example #1. For whatever reason, we need to use the MP4 container (maybe to be able to play that file on another device). This means that we must also drop AC3 in favor of AAC.

We have already inspected the file, so let's extract the streams:

ubuntu% mkvextract tracks video.mkv 1:video.h264 2:audio.ac3

Then, we need to transcode the audio stream to AAC (let's use high-quality VBR):

ubuntu% ffmpeg -i audio.ac3 -acodec libfaac -aq 255 -ar 44100 -ac 2 -async 1 -vsync 1 audio.aac
Now, we'll use MP4Box to remux the initial video stream and the new audio stream into an MP4 container:

ubuntu% MP4Box -new -add audio.aac -add video.h264 -fps 23.976 video.mp4AAC import - sample rate 44100 - MPEG-4 audio - 2 channels
AVC-H264 import - frame size 1280 x 720 at 23.976 FPS
Import results: 60146 samples - Slices: 1027 I 36894 P 22225 B - 1 SEI - 968 IDR
Stream uses B-slice references - max frame delay 2
Saving video-ps3.mp4: 0.500 secs Interleaving

That's it :)

7) Example #4: (mkv file, H.264 level 5.1, DTS 5.1) --> (MP4 file, H.264 level 4.1, AAC)
Here, we'll use a new sample file (for the sake of brevity, I will just display the relevant lines):

ubuntu% mediainfo video2.mkv
General
Complete name                    : video2.mkv
Format                           : Matroska

Video
Format                           : AVC
Format/Info                      : Advanced Video Codec
Format profile                   : High@L5.1
Muxing mode                      : Container profile=Unknown@5.1
Codec ID                         : V_MPEG4/ISO/AVC
Frame rate                       : 23.976 fps

Audio
Format                           : DTS
Format/Info                      : Digital Theater Systems
Codec ID                         : A_DTS

ubuntu% mkvinfo video2.mkv

| + A track
|  + Track number: 1
|  + Track type: video
|  + Codec ID: V_MPEG4/ISO/AVC

| + A track
|  + Track number: 2
|  + Track type: audio
|  + Codec ID: A_DTS

OK, so track 1 is H.264 video and track 2 is DTS audio. Let's extract them:

ubuntu% mkvextract tracks video2.mkv 1:video.h264 2:audio.dts
According to what I've said earlier, H.264 is supported, so all we have to do is to transcode the audio... right?

Wrong :) Take another look at the video stream: format profile is "High @ Level 5.1" (note: this has nothing to do with 5.1 audio!). Unfortunately, the PS3 cannot play anything above Level 4.1, so this video stream needs to be fixed. Does this mean we have to reencode it and lose quality?

Nope :) We're going to change the level right in the binary file. For this purpose, you need an hexadecimal editor. I recommend ghex2, which can easily be installed with 'sudo apt-get install ghex'.

just go 'ghex2 video.h264' and look at the first 8 bytes of the file: they should read '00 00 00 01 67 64 00 33'. Hmm... 0x33 in decimal is '51', which means 'level 5.1' : let's change this byte to 0x29 ('41' in decimal) to 'downgrade' the video to level 4.1 and save the file.

Now, let's take care of the audio stream. As above, we'll use high quality VBR:

ubuntu% ffmpeg -i audio.dts -acodec libfaac -aq 255 -ar 44100 -ac 2 -async 1 -vsync 1 audio.aac
The last operation is to remux the initial video stream and the new audio stream into an MP4 container:

ubuntu% MP4Box -new -add audio.aac -add video.x264 -fps 23.976 video.mp4
That's it. I'm not sure the level hack will always work, but it did in this case and we avoided any video transcoding.

8) Example #5: (mkv file, MPEG-2 video, DTS 5.1+ AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3)
Let's take a look at this one (output edited for brevity)

ubuntu% mediainfo video3.mkv 
General
Complete name                    : video3.mkv
Format                           : Matroska
File size                        : 73.9 MiB
Duration                         : 50s 240ms
Overall bit rate                 : 12.3 Mbps

Video
Format                           : MPEG Video
Format version                   : Version 2
Format profile                   : Main@High
Format settings, Matrix          : Default
Codec ID                         : V_MPEG2
Width                            : 1 920 pixels
Height                           : 1 080 pixels

Audio #1
Format                           : DTS
Format/Info                      : Digital Theater Systems
Codec ID                         : A_DTS
Language                         : Russian

Audio #2
Format                           : AC-3
Format/Info                      : Audio Coding 3
Codec ID                         : A_AC3
Language                         : English

ubuntu% mkvinfo video3.mkv 
|  + Track number: 1
|  + Track type: video
|  + Codec ID: V_MPEG2

|  + Track number: 2
|  + Track type: audio
|  + Codec ID: A_DTS

|  + Track number: 3
|  + Track type: audio
|  + Codec ID: A_AC3

Track #1 is full HD MPEG-2 video, track #2 is DTS audio in Russian and track #3 is AC3 audio in English. Let's extract them:

ubuntu% mkvextract tracks video3.mkv 1:video.mpeg 2:audio.dts 3:audio.ac3

If like me you don't speak Russian, you'll pick track #3 :) If you do speak Russian, please refer to the next example for DTS-->AC3 conversion.

One option would be to remux tracks #1 and #3 into an MPEG-2 TS container (like we did in example #1). Unfortunately, tsMuxer chokes on the video stream ('Can't detect stream type').

Let's try to remux in an MPEG-2 PS container. For this, we'll use ffmpeg but without any transcoding:

ubuntu% ffmpeg -i video.mpeg -i audio.ac3 -map 0.0:0 -map 1.0:0 -acodec copy -vcodec copy -f vob video.mpg
That's it!

9) Example #6: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MPEG-2 PS file, MPEG-2 video, AC3 5.1)
We'll use the same input file as in example #5, but this time we'll use convert the DTS audio stream and convert it to AC3.

Let's take another look at the DTS stream:

ubuntu% mediainfo audio.dts
General
Complete name : audio.dts
Format : DTS
Format/Info : Digital Theater Systems
File size : 9.04 MiB
Duration : 49s 381ms
Overall bit rate : 1 536 Kbps

Audio
Format : DTS
Format/Info : Digital Theater Systems
Bit rate mode : Constant
Bit rate : 1 536 Kbps
Channel(s) : 6 channels
Channel positions : Front: L C R, Surround: L R, LFE
Sampling rate : 48.0 KHz
Resolution : 16 bits
Since the DTS stream has 6 channels, we can either transcode it to 2-channel AC3 or 6-channel AC3. Let me show you both:

ubuntu% ffmpeg -i audio.dts -acodec ac3 -ar 48000 -ab 448k -ac 2 audio-2ch.ac3

ubuntu% ffmpeg -i audio.dts -acodec ac3 -ar 48000 -ab 448k -ac 6 audio-6ch.ac3

448 KBit/s is the maximum authorized AC3 bitrate for DVDs. However, with the Playstation 3, we could go up to 640Kbit/s.

Now, let's remux the video and audio streams (I'll use 6-channel AC3 here):

ubuntu% ffmpeg -i video.mpeg -i audio-6ch.ac3 -map 0.0:0 -map 1.0:0 -acodec copy -vcodec copy -f vob video.mpg
That's it.

10) Example #7: (mkv file, MPEG-2 video, DTS 5.1 + AC3) --> (MP4 file, H.264 video, AAC)

We'll use the same input file as in example #5. Let's assume that - like in example #3 - we need to use the MP4 container. This means that we must convert the video stream to H.264 and our preferred audio stream (AC3 for this purpose) to AAC.

Please note that this is not the best option! The video compression will be lossy and all the longer that we're dealing with a 1920x1080 video stream. As far as audio is concerning, we're moving from 6-channel AC3 to stereo AAC. So, unless you really need the MP4 container, I strongly suggest that you use the solution described in example #6.

Anyway, let's get started and extract the MPEG-2 and AC3 streams:

ubuntu% mkvextract tracks video3.mkv 1:video.mpeg 3:audio.ac3
Then, let's encode the audio stream to AAC:

ubuntu% ffmpeg -i audio.ac3 -acodec libfaac -aq 255 -ar 44100 -ac 2 -async 1 -vsync 1 audio.aac
Now, the big one: encoding the MPEG-2 stream to H.264:

ubuntu% ffmpeg -i video.mpeg -f rawvideo - | x264 --level 4.1 --crf 21 --bframes 16 --b-pyramid --ref 4 --mixed-refs --weightb --partitions all --threads 2 -o video.h264 - 1920x1080
Let's explain this line a bit: as you can see, we're using ffmpeg to decode the video stream to raw video and we're piping the output into the x264 encoder. The reason for this is avoid any H.264 default value that could be set by ffmpeg: indeed, piping guarantees that all H.264 flags will be set by x264.

What about the x264 flags? '--level 4.1' sets the H.264 level (the PS3 can't play anything above 4.1, please see example #4 for more information). '--crf 21' select Constant Rate Factor encoding, with default quality. '--bframes 16 --b-pyramid --ref 4 --mixed-refs --weightb --partitions all' are frame options which seem to strike the right balance between quality and encoding time (please see this article for more information). Finally, '--threads 2' will create 2 encoding threads.

FYI, on a dual-core PC @ 1.83 GHz, this command took 8 minutes to encode a 50-second video! Now you understand why you don't want to do this unless you have no other option.

Now, the final step: remuxing into an MP4 container

ubuntu% MP4Box -new -add video.h264 -add audio.aac -fps 25 video.mp4
That's it!