Playback duration/progress issues

[Opus4]

Once in a while, I get this issue on some channels for my HDHomeRuns... there might be stream errors during prime time, but no errors during other times. I can run the file through VideoReDo's quick stream fix (or whatever it is called) and then play it fine. After a few days, the channel has no issues again.

Andy

[mikejaner]

Originally Posted by Narflex

@mikejaner, what source are you recording these from?

HDHR Prime via SageDCT

[Narflex]

@mikejaner, OK, then the problem is likely due to the cable provider messing up the PTS timestamps when they are doing ad insertion. If the author of that program is able to detect when PTS discontinuities occur...then I can tell them how to send SageTV an event so that it stops/restart the stream at that time which would work around this issue (only downside is it'll have a half second or so gap at that point when it occurs...but that's a much better tradeoff).

[Tiki]

Originally Posted by Narflex

@mikejaner, OK, then the problem is likely due to the cable provider messing up the PTS timestamps when they are doing ad insertion. If the author of that program is able to detect when PTS discontinuities occur...then I can tell them how to send SageTV an event so that it stops/restart the stream at that time which would work around this issue (only downside is it'll have a half second or so gap at that point when it occurs...but that's a much better tradeoff).

So, would this result in the recording being broken up into multiple files (a new file each time a PTS discontinuity occurs)?

[Narflex]

Originally Posted by Tiki

So, would this result in the recording being broken up into multiple files (a new file each time a PTS discontinuity occurs)?

Yes it would. If someone does do this...the other case you want to account for is when you notice audio/video PIDs changing dynamically (mainly due to ad insertion). Normally this doesn't happen; but I noticed FIOS was doing it every now and then on some channels a couple years ago.

[mikejaner]

Originally Posted by Opus4

Once in a while, I get this issue on some channels for my HDHomeRuns... there might be stream errors during prime time, but no errors during other times. I can run the file through VideoReDo's quick stream fix (or whatever it is called) and then play it fine. After a few days, the channel has no issues again.

Andy

Cool VideoRedo quick stream fix did it for me too!!
It would be cool for others if there was an open source tool, possibly command line based which could do this same thing.

[mikejaner]

Originally Posted by Narflex

@mikejaner, OK, then the problem is likely due to the cable provider messing up the PTS timestamps when they are doing ad insertion. If the author of that program is able to detect when PTS discontinuities occur...then I can tell them how to send SageTV an event so that it stops/restart the stream at that time which would work around this issue (only downside is it'll have a half second or so gap at that point when it occurs...but that's a much better tradeoff).

I put in a question to the author, so we'll see what he says. Thanks!

[cgruelle]

Originally Posted by mikejaner

Cool VideoRedo quick stream fix did it for me too!!
It would be cool for others if there was an open source tool, possibly command line based which could do this same thing.

ProjectX may be what you are looking for.

http://www.doom9.org/index.html?/Dig...-fullguide.htm

[mikejaner]

Originally Posted by cgruelle

ProjectX may be what you are looking for.

http://www.doom9.org/index.html?/Dig...-fullguide.htm

I am actually testing that right now. Either it's not looking so good, or I don't know how to use it. Tried various settings, but cannot get it to correct the stream properly.

[Taddeusz]

I made a new thread for this here:

http://forums.sagetv.com/forums/show...578#post581578

But it seems this is really the same issue. I'm playing around with a Linux SageTV server. I have my Windows recording shares mounted via fstab. When I add them as recording directories on my Linux server the durations are all screwed up. As these files are located on a different physical machine there's no way for me to move things around. What to do?

Personally I would rather be using Linux (particularly one without GUI) as I am now since the OS has far less overhead than Windows. This poses a real problem for other people trying to move from Windows to Linux.

[turak]

You could try shutting SageTV down. Umounting the Windows shares. Starting up SageTV. Shutting down SageTV. Remount the Windows shares. Start up SageTV.

It's basically the same thing.

[lgkent]

I was able to set some break points and do some debugging. It looks like getMediaAVInf0 is getting the duration wrong when importing files larger than 1GB. If I set skip_internal_format_parser=true, ffmpeg gets the duration right, but this causes problems with audio stream detection. My intellij ide does not handle breakpoints thru jni with native code, so I need to figure out best way to debug through jni calls, any advice? I am going to try to setup eclipse to see if it will let me debug java + jni native...

My files are MPEG2-PS recored from HdHomerun tuners. I am running latest code on Ubuntu 14.04 It does not matter if they were recorded on Linux or Windows. Linux has issues detecting duration with MPEG2-PS files larger than 1GB.

Steps to repro:
1. choose a video file larger than 1GB and temporarily move it out of var/media/tv.
2. restart sagetv server so it will detect and remove video from database.
3. restore the video file from step 1.
4. restart sagetv server to re-import video and observe incorrect duration.

[Narflex]

I'd say start drilling down through the JNI code and determine which part is doing the duration calculation. That should be done by getting the first and last PTS values in the file and then just doing the difference on them (with wraparound detection since PTS values are a 33 bit field so they roll over sometimes). You probably don't need a debugger so much as just adding some printf statements in the parser code to analyze this.

[lgkent]

I have a provisional fix for the imported video duration issue. It's more of a 64bit include/compile problem with linux rather than a code issue. I am happy to fork and submit. Is there any formal bug tracking or additional process? Meanwhile I am going to give it some run time to see if anything else comes up.

The demuxer was getting nonsense values from lseek64() when getting the file length in DemuxSourceLength() for large video files. I added "#define _LARGEFILE64_SOURCE" in NativeCore.h before any includes and lseek64() returns good values now. I found it easier/faster to create a simple c program that used the .so lib for debugging in this case.

There no doubt needs to be some discussion if anyone has a more elegant solution, or other potential issues. This could fix other latent large file issues.

[Narflex]

That sounds like a very likely fix. Just go ahead and submit the patch on GitHub (after signing the Google CLA, read the CONTRIBUTING file for details).

[Taddeusz]

This fix works well for the MPEG-PS files from my HDHR's. My test Linux system is now picking up the durations correctly when I add my Windows recording shares to be recording folders in Linux. However, my HD-PVR ts files are not getting the correct duration. They are all showing double the actual duration.

[lgkent]

This sounds like another duration issue I had with a few files. It happens with files that have incorrect last modified dates. The easy fix is to 'touch' the video file (to update last modified) and sage will correct the timestamp on import.
Ideally the last modified timestamp should be the meta-data air-date + duration. For some reason I had several files that were 50 minutes off so they imported and played, but appeared to start 50 minutes before the start time and looked like they were twice as long. If the last modified timestamp is completely out of the play window, sage will auto-correct the timestamp on import.
I don't see this as a sage bug, more of a timestamp issue with the original file. If we were to make the auto timestamp correction more sensitive it would cause issues with legitimate recordings that start early or end late.

[Taddeusz]

Originally Posted by lgkent

This sounds like another duration issue I had with a few files. It happens with files that have incorrect last modified dates. The easy fix is to 'touch' the video file (to update last modified) and sage will correct the timestamp on import.
Ideally the last modified timestamp should be the meta-data air-date + duration. For some reason I had several files that were 50 minutes off so they imported and played, but appeared to start 50 minutes before the start time and looked like they were twice as long. If the last modified timestamp is completely out of the play window, sage will auto-correct the timestamp on import.
I don't see this as a sage bug, more of a timestamp issue with the original file. If we were to make the auto timestamp correction more sensitive it would cause issues with legitimate recordings that start early or end late.

This appears to be the problem. It appears that SageTV in Windows doesn't always set the modified date/time on the file properly and uses some other method to determine the duration.

[babgvant]

Originally Posted by mikejaner

I put in a question to the author, so we'll see what he says. Thanks!

SageDCT doesn't currently monitor timestamps in files as they are dumped, so that would need to be added for this approach to work.

If you're going to track/mess w/ timestamps, I think it would be a better approach to just ditch the original file's stamps and generate new ones while remuxing.

[zoop]

After upgrading to v9 (current release, ubuntu linux 16.04LTS), I have a similar problem with a slightly different manifestation. On imported videos, the duration bar is always 0:00:00 and jumps immediately to "end". The video continues to play, but any attempt at seeking causes the video to restart. This occurs on both HD300s and the placeshifter client (mac). This does not appear to be happening on recorded TV shows. I have tested mkv,m4v, and avi files, the behavior is the same... I haven't checked all of my imported files, but so far there haven't been any that behave properly. Logs are attached below... If I am reading them right, Sage thinks the file is 0:00:00 in duration, but I have no idea why that would be. I have tried the "re-import" function as well, to no avail.

I have been poking around the forums for the same problem, haven't found a thread yet. Any thoughts?

thx much