Extracting Subtitles from TS files

[doc]

I expect I already know the answer to this

Does anyone know if there is a way to extract the subtitles out of a TS file?

I've found lots of utilities that should do it with mpg files. I know the subtitles are included in the TS file as I can see them in VLC but I'd really like to find a way to extract them.

I've tried converting the recordings to mpg but they then lose the subtitle stream.

Any idea's anyone?

[emveepee]

ProjectX is best for extracting Teletext and graphic subtitles. CCExtractor is easier for extracting North American closed captions.

Martin

[doc]

I was a bit vague with my post, I'm in New Zealand using DVB-S cards.

I've tried ProjectX but couldn't get it to work. I don't know if it will extract the streams or not. I'll play some more over the next few days and see how I get on.

[doc]

I'm getting somewhere slowly....

The subtitles are stored as teletext pages on page 801.

I can extract them to a .sub file in ProjectX, and they are in the format below

1
00:00:00,000 --> 00:00:00,000
Now to the most viewed stories
on our website today.

2
00:00:00,000 --> 00:00:00,000
Labour MP Charles Chauvel says
a story about him telling children
on a plane to shut up

The log file has the following

--> using faked PTS for following data:

++> Teletext: PID 0x0243 / PesID 0xBD / SubID 0x10 :

-> export format: srt
-> Unicode (UTF-16 BE) on text export
-> decode hidden rows of teletext (id 0xFF, means offline pages)
-> temp. file: EleventhHour-7938023-0{0x0243-0xBD-0x10}.tt (122956758 bytes)
-> looking for page number 801
!> teletext stream doesn't use PTS's, sync impossible
-> take only first Audio PTS (to sync the starttime)
-> Teletext PTS: first packet 00:00:00.000, last packet 00:00:00.000
-> adjusting teletext at its own timeline
887 pages of No. 801 written...
---> new File: C:\\EleventhHour-7938023-0{0x0243-0xBD-0x10}[801].srt

As you can see the stop/start times are all set to 00:00:00,000

Does anyone know if its possible to get the times out of this too? or is what I've extracted the best I can manage?

Thanks.

[StephaneM]

Originally Posted by doc

As you can see the stop/start times are all set to 00:00:00,000 Does anyone know if its possible to get the times out of this too? or is what I've extracted the best I can manage?

The ProjectX log states that the TTX stream isn't using a PTS (presentation time stamp) and as such ProjectX is not able to deduce the time of display (in reference to video or audio stream) of the subtitle.

Actually this is not an error in the stream because TTX subtitles may not carry a PTS (TTX are supposed to be displayed when they come)

I think that my recorder would also fail to extract correctly these subtitles...

Regards,
Stéphane.

[doc]

Thanks for looking at that Stephane.

I'll put this idea at the bottom of the pile and start on a few others....

[doc]

OK, weathers been rubbish today so I got bored....

After playing around a bit more I got ProjectX to extract a timestamp with the subtitles after unchecking the option "Re-build TTX-PTS from 1st MpgAudio stream"

The timestamps were out by 73 seconds for some reason, so I've edited the SRT file with vbscript to correct it.

Ill test on a few more files to see if the 73 seconds is consistant or not - once the TV becomes free!

If it is I'll use SJQ to automatically create them after each recording finishes.

If its not consistant, I'll put this back to the bottom of the pile again!

[StephaneM]

Originally Posted by doc

The timestamps were out by 73 seconds for some reason, so I've edited the SRT file with vbscript to correct it.

73 seconds seems huge (usually DVB elementary stream timestamps are to be synched with the main stream clock, and usually this clock is the one carried by the video stream)

Just for curiosity, why do you want to extract the subtitles?

[doc]

Originally Posted by StephaneM

73 seconds seems huge (usually DVB elementary stream timestamps are to be synched with the main stream clock, and usually this clock is the one carried by the video stream)

Just for curiosity, why do you want to extract the subtitles?

Well... the first ttx page is timed at 00:00:00 and all other timestamps seem to be synced to that page, so the amount they are off varies each time. I managed somehow to get one to start 09:something - not sure why as the show started at 12:00 noon. Will have a closer look another time.

The 'customer' here want a transcript of all the subtitles for archiving purposes. To be honest the extracts I have will do, I was just hoping I could get the timelines correct so that in future you can use them for playback - at some point they may ask the question, but I hope they don't!

[StephaneM]

The 'customer' here want a transcript of all the subtitles for archiving purposes. To be honest the extracts I have will do, I was just hoping I could get the timelines correct so that in future you can use them for playback - at some point they may ask the question, but I hope they don't!

Then try my DVBViewer Recorder if you can, it will extract all the subtitles with the correct time stamp (during the recording) and as those are in SMI format they can be played back even with Windows Media Player (and of course with SageTV). Time stamps are pretty accurate: you will still have some inaccuracy because of video playback pipeline (usually with WMP subtitles will come on a little earlier than they should, it depends on the video decoder used)

[Doubletop]

Can somebody provide a simple explanation what would be required for SageTV to play closed captions when they already exist in stream? I'm not "having a go" StepanieM you've done some great stuff but I cannot see why we should need to use your DVBViewer recorder to control the tuner and have SageTV then use it as a network encoder when SageTV is perfectly capable of doing the tuning and saving the video file.

If DVBviewer can find the captions why can't a SageTV plugin be created that extract the captions during the playback and displays them? I just don't understand why we have to have these round the houses solutions. Especially when apps like DVBViewer apparently have a very high CPU overhead.

Doc; I'm in NZ as well so so if you get this sorted you would earn countless WAF points. Its the only thing missing from SageTV as far as I am concerned. We have to resort to real time for shows like "Coro" as the American in our household has understandable issues with the Manchester accent.

[doc]

I think its on the Sage 'to do' list, its just that the list is quite long I'd guess.

The main reason I haven't looked into Stephane's software is because all the freeview channels are on 2 muxes, so with 2 tuner cards and umpteen dvb enhancer virtual tuners you can record all of the freeview channels simultaneously.

I was looking at ProjectX as I was hoping to run a standalone exe against each TS file after they've recorded. ProjectX does a good job once you get the ini file correct, but the timestamps are way off for the first ttx page, which makes it pretty useless unless you manually correct them. If you just want the odd file thats fairly easy as long as you can count the seconds and then knock up a script to amend it, but its not really a viable solution.

I'm afraid my programming skills got left behind in the 90's when the old mainframe got sent to the landfill and the MOD decided to buy COSH (commercial off the shelf) applications.

I'm hoping to get back into programming in the next few months now that I've got time on my hands so if my skills ever develop I'll look into this again. I've done a lot of vbscript'ing lately but I don't call that proper programming.

My question for the programmers out there is - which language? I'd hate to spend a few months and a few courses (and pounds) learning one to find out I'd have been better learning another. I also have to consider which would be best for work if I decide to go in that direction again - I'm sick of servers and support work

DT - I live in England - just visiting friends here and go back in 5 weeks. Heading off in the car on my own in a couple of days with a tent. If you don't hear back I've probably been attacked by kiwi's ...... or a 15cm tsunami !

[StephaneM]

Originally Posted by Doubletop

Can somebody provide a simple explanation what would be required for SageTV to play closed captions when they already exist in stream?

Time to develop the feature...

I cannot see why we should need to use your DVBViewer recorder to control the tuner and have SageTV then use it as a network encoder when SageTV is perfectly capable of doing the tuning and saving the video file.

1) SageTV doesn't (by default) save the # subtitles stream, only audio and video.
2) When you ask SageTV to save the subtitles stream: it can mess the recording and SageTV will not play it (I was having terrible issues when doing so)

So basically when SageTV make the recording, you have no subtitle in the recording.

Moreover, each channel will carry multiple subtitles, so you have to choose which of them you want to display. By using a network recorder you can for each channel specify which sub to extract. This is not the same when you only have the recording file: which subtitle stream should be extracted? As you don't know the channel that where used to make the recording (unless using for instance the web server plugin to ask for it...)

Especially when apps like DVBViewer apparently have a very high CPU overhead.

You are wrong: DVBViewer doesn't have any CPU overhead. With my recorder and DVBViewer you'll not see your CPU working at all... I can record, extract sub from three HDTV recording on my old machine while playing one HDTV recording / displaying DVB subs (that are overlaid bitmap images)

So if you really need live subtitles dispalyed you should really try it...

Regards,
Stéphane.

[StephaneM]

Originally Posted by doc

The main reason I haven't looked into Stephane's software is because all the freeview channels are on 2 muxes, so with 2 tuner cards and umpteen dvb enhancer virtual tuners you can record all of the freeview channels simultaneously.

You can also do it with DVBViewer + DVBViewer Recorder... The only thing you can't do is: with one tuner, tune to encrypted channels on the same mux at the same time (from DVBViewer authors: when it is possible -it depends on the CAM used- you may loose packets, corrupt one or both recordings. That's why it is not allowed)

Regards,
Stéphane.

[doc]

Originally Posted by StephaneM

You can also do it with DVBViewer + DVBViewer Recorder...

Cool, I hadn't realised that. I should have guessed as the XML importer of yours I use is always at least one step ahead of what I want to do

I'll see if I can find time to look into this next week when I get back.