Glitches in recordings caused by sagetv

[drewg]

Hi,

I filed a support request yesterday, but I was hoping somebody here would have some insights.

I've been running SageTV for ~2 years. In all that time, through 3 versions (6.3.x, 6.4.8, 6.6.2.1) and many different Java versions, I've consistently had problems with small corruptions in recordings due to SageTV server activity. Whenever I do anything that causes the "spinning wheel" on my HD100, I'll get a glitch in any recording that is in progress. I'm talking about things like entering the recordings menu, entering the videos menu, etc.

I have a really simple setup. I'm running Ubuntu 8.04 on a dual-core 2.5GHz athlon64. Recordings are saved to a SATA based internal software Raid5 array formatted with XFS. I do all my SageTV viewing via an hd100. I'm using sun java 1.6.0_17. I'm using the default STV with the comskip plugin and malore menus enabled. I use good tuners (LG DT3303 based based Dvico Fusion HDTV5 Gold and Airstar HD50000, and a Pinnacle 800i), I'm close to the towers, and have generally excellent reception.

The other night, I set up an experiment to try to diagnose the problem. I bought an HDHR and setup MythTV in parallel, and assigned MythTV a recording directory on the same filesystem SageTV uses for recordings. I then set both to record SVU on Weds night at 9pm. I entered the recordings menu around 9:20PM and started watching. There were massive glitches in the SageTV recording corresponding to that activity, and none in the MythTV recording. For fairness, I also watched a bit of the MythTV recording (via browse files on server) around 9:40 or so -- it had no corruption that I could find. After the recording ended, I went back and checked the SageTV recording, and, sure enough, it had corruption in it corresponding to when I was browsing the server files to find the MythTV recording.

I've tried various things over the years to try to fix this:

1) Increased the java heap from 256MB to 512MB.

2) Used IBM's Java. IBM's java is much faster, so this reduces the "spinning wheel" wait time, and hence the recording corruption. But there is still some, and the IBM Java has a wonky effect on tuning. When using the IBM java, the first recording to be scheduled after the server starts on every tuner will fail. Subsequent recordings succeed. This scares me away from using the IBM Java..

3) sysctl kernel.sched_compat_yield = 1 (due to this article: http://www-01.ibm.com/support/docvie...id=swg21372909)

But nothing seems to totally eliminate the corruption. Is there something I'm missing? Some kind of buffer size I can just increase? Some other Java tuning I can do?

Thanks,

Drew


PS: As you can see, the MythTV recording is slightly larger than the SageTV one. Having watch both through, I can say the MythTV recording is flawless, but the SageTV recording has glitches corresponding to HD100 activity..

sagetv@thunder:~/server$  ls -l /var/media/tv/LawOrderSpecialVictimsUnit-Quickie-1101178-0.mpg /var/media/mythtv/1171_20100106210000.mpg
-rw-r--r-- 1 mythtv mythtv 6468698172 2010-01-06 22:00 /var/media/mythtv/1171_20100106210000.mpg
-rw-r--r-- 1 sagetv sagetv 6461399040 2010-01-06 22:00 /var/media/tv/LawOrderSpecialVictimsUnit-Quickie-1101178-0.mpg

[freedml]

maybe the tuners are 'too good' and have too much signal from the towers that you are 'close to.' a different setup with the same tuners might tell you something about the setup. a different setup with different tuners doesn't tell me much.

[drewg]

Originally Posted by freedml

maybe the tuners are 'too good' and have too much signal from the towers that you are 'close to.' a different setup with the same tuners might tell you something about the setup. a different setup with different tuners doesn't tell me much.

No, the tuners are *better* than the HDHR. All the corruption in the SageTV recordings corresponds to times when the SageTV media extender was waiting (and displaying the spinning circle). Eg, if I'm out for the night, all my recordings will be perfect. If I'm doing something in the media extender, the recordings will be glitchy with the glitching corresponding 1:1 with times when the media extender was waiting. The MythTV experiment was just to prove to myself that the problem was not general system overhead.

I'm pretty convinced that the issue is due to garbage collection overheads. If I use the IBM JRE 1.6 with their "low pause" garbage collector (-Xgcpolicy: optavgpause) the issue seems to get a lot less severe.

What I think is happening is the Java garbage collector runs, preventing the thread that's supposed to be reading bytes from /dev/dvb/adapterN/dvr1 from running. Eventually, the kernel buffer fills up, and data is lost. There is an ioctl to set the DVB ringbuffer size (DMX_SET_BUFFER_SIZE). I probably should build a debug version of the DVB drivers and see if SageTV increases the buffer size, or just uses the inadequate 1.8MB default.

Drew