Spinning Circle - somewhat fedup

[tmiranda]

Originally Posted by Islander

Ok I will try that tonight and let it run overnight.

"- Remove all of the import paths." I am assuming that this so that whenever the scan is done it is only dealing with the "recorded" media. Wouldn't it be best to leave the imported paths in (whether initially or after the recorded scan is done and the system tested) to see if it is the size of the DB that is choking it?

This is what the Db looks like:

Wizbin size 22,413kb
# of dvds and imported movie files as reports by SMW = 1,806
Music = 8,006 songs
Photos = 4,700 or so
Recorded TV 2,100 recorded shows and movies, add possibly another 200 or so imported shows

Are those numbers likely to choke the db?

Thanks.

True, by removing hte import paths you will only be dealing with recorded shows (whic hwill show up as imported because you are blowing away the wiz.bin.)

If that works (no lag) then I'd start adding back import paths (slowly) to determine if the DB size is the problem. I can also be a single imported file causing the problem because some files have been known to hang the import scanning.

Again, my suggestion is to start with a "bare" system and slowly add things one at a time until you find the problem.

BTW. My database is a similar size as yours (more of some items, less of others) and I have no problems.

[Islander]

One thing I found out going through the logs is that there was a lot of DX9 this and Dx9 that. I tried running Dxdiag and nothing came up although Vista 64 has been upgraded all the way to SP2. I am not entirely sure what that means whether DX9 is in the system and it is just the end user runtime that was missing. In any event I went ahead downloaded and installed the end user runtime and the situation seems a little better, or maybe my mind is playing tricks.

It could well be a corrupted file or entry in the wizbin because the previous server was having intermittent problems shutting down and rebooting perhaps due to a faulty memory stick. That probably was not too good for the wizbin. Unfortunately as I understand it wizbin is an all or nothing situation and impossible to repair by another method but resorting to an old backup. Given the issues with the old server I am not entirely sure that if the issue is with the wizbin that I will be able to find a clean one.

Hopefully the next version of Sagetv gets rid of this all or none situation and uses some sort of individual files even if indexed on wizbin v2.0 so that corruptions will not mess the whole system.

I am keeping an eye on the logs for consistencies on the hang situation.

[ranger]

I get the spinning circle quite frequently too, but it is almost always associated with TVE, so I have attributed it to the size of the library and the number of non-local storage locations. I played with all the image settings and only saw minor improvements.

I rarely see any problems in the SageTV recorded items list. The only really unusual item to note is that there are no delays in a PlaceShifter client on the local network. I think SageTV must handle the db in a different way for PlaceShifter.

[razrsharpe]

Originally Posted by ranger

I get the spinning circle quite frequently too, but it is almost always associated with TVE, so I have attributed it to the size of the library and the number of non-local storage locations. I played with all the image settings and only saw minor improvements.

I rarely see any problems in the SageTV recorded items list. The only really unusual item to note is that there are no delays in a PlaceShifter client on the local network. I think SageTV must handle the db in a different way for PlaceShifter.

if you have a large amount of imported tv in TVE (ie non sage recorded tv) then ya it will be slow on initial load. There is a bug in the sage core how Get/SetMediaFileMetadata is called from a client. They have said that it will be fixed in the next betas. Placeshifter doesn't have this problem because all the code is handled inside (local to) the service's jvm.

I think your issue is different then islander's because yours is associated with navigation (inside tve) whereas his seems to have no ryme or reason to it. But then again, maybe something that islander is running (in the background perhaps) is calling Get/SetMediaFileMetadata and that's where all his spinning circle of death's are coming from.....

[Islander]

I have been keeping a close eye at the spinning circle and going back to the log to see what happened just before the HANG and most of the instances are related to these:

[Fork-PRQST-1215135@a4156c] Cleaning up orphaned files...
Wed 4/14 2:12:39.767 [Fork-PRQST-1215135@a4156c] Scanning video directory \\HPSERVER1\Recorded TV
Wed 4/14 2:12:39.769 [Fork-PRQST-1215135@3e54a] Cleaning up orphaned files...
Wed 4/14 2:12:39.770 [Fork-PRQST-1215135@3e54a] Scanning video directory \\HPSERVER1\Recorded TV

[Seeker@10713de] Diskspace checking is running
Wed 4/14 2:11:41.560 [Seeker@10713de] Verifying existence of all TV media files in database fixDurs=false avoidArchive=true

[Fork-PRQST-1215135@a4156c] Scanning video directory D:\Recorded TV
Wed 4/14 2:07:10.981 [Fork-PRQST-1215135@3e54a] Scanning video directory D:\Recorded TV
Wed 4/14 2:07:10.984 [Fork-PRQST-1215135@a4156c] Scanning video directory \\HPSERVER1\Videos
Wed 4/14 2:07:10.984 [Fork-PRQST-1215135@3e54a] Scanning video directory \\HPSERVER1\Videos

[Fork-PRQST-1215135@a4156c] Scanning video directory \\HPSERVER1\Downloaded Content
Wed 4/14 2:07:15.208 [Fork-PRQST-1215135@3e54a] Scanning video directory \\HPSERVER1\Downloaded Content
Wed 4/14 2:07:15.241 [Fork-PRQST-1215135@3e54a] Deleted orphaned .my file D:\Recorded TV\Justified-TheLordofWarandThunder-1113158-0.my
Wed 4/14 2:07:15.241 [Fork-PRQST-1215135@a4156c] Deleted orphaned .my file D:\Recorded TV\Justified-TheLordofWarandThunder-1113158-0.my

It seems to be repeating the scanning and verifying procedures with some frequency, is that what is supposed to be doing?

[tmiranda]

In the default STV there is a setting that controls if the scanning is done periodically or not. I can't remember exactly where right now, somewhere under Setup->Detailed Setup->Advanced or Customize. I'm not sure how SageMC handles it. (The debug info you posted looks like it's coming from SageMC.)

Personally I'd turn off automatic scanning until you get things sorted out.

[razrsharpe]

Originally Posted by tmiranda

I'm not sure how SageMC handles it.

follows however the default stv is setup up... it is a server property, and afaik the core handles the rescanning (not the stv)

[Islander]

Exactly and that is what I do not understand because mine has always and is currently setup as being manual, i.e. on demand rather than on a stated interval.

[Opus4]

Periodically checking recordings is not the same as checking import dirs. However, the log lines included above talk of deleting orphaned .my files - I think that would have to be a customization, unless I've forgotten some core handling of such files. I believe all of those "Fork-PRQST" lines are from a customization.

- Andy

[PLUCKYHD]

Originally Posted by Opus4

Periodically checking recordings is not the same as checking import dirs. However, the log lines included above talk of deleting orphaned .my files - I think that would have to be a customization, unless I've forgotten some core handling of such files. I believe all of those "Fork-PRQST" lines are from a customization.

- Andy

Yes the .my files didn't catch that in the log. Make sure and try turning that off in sagemc to not create those don't know that those are causing the issue but none the less.

Also the forks in the log Andy is right there something is creating those and it could be causing the hangs. Again try the defautl without any imports and you shouldn't see those or the .my files issue in the logs.

[Islander]

Fresh reinstall only other action taken was that of upgrading until last official version was reached. I let the system built the wiz.bin from scratch. The resulting file was about half the size of the current one. Going by the number of "vidos" reported on the clean wiz.bin it seems that everything or close to everything made it through. No issue with sustained spinning circles. When I used the current wiz.bin with the stock ui I started seeing the spinning circles again.

I then took that clean wiz.bin and used it on the sagetv installation with all the add ins I had. I noticed no issues with the spinning circles. Couple of things were not appearing where I would have expected but I guess I can redo the process over the weekend when I have more time and see what is what.

Unfortunately, if I understand correctly I either have a backup of wiz.bin that is not corrupted or I am SOL as there is no utility or anything else that will save it or at least id any media files causing the corruption that should be deleted (assuming that this will in fact fix the corruption which I do not know); right? [I have many weeks worth of backup on the previous server but with the frequent shutdowns I am not sure I will find a non-corrupted version but keeping my fingers crossed]

Worse case scenario, if I have to let the system rebuild wiz.bin:

1. I understand that I will lose the favorites and will have to redo those, right?

2. I have seen something EPG data being lost but I am guessing that refers to the exiting one which Sage would have used to make the link to the recorded shows and EPG will get updated and populated again once the update interval arrives, right?

3. If I understand correctly now ALL media files will appear as imported. No biggie for the DVDs etc because they were already considered imported. The imported tv shows I am guessing will not fare worse. The issue will be the sage recording as they will not appear under recordings, I will lose the history of what has been recorded and favorites may end up recording them again (not a big headache in my opinion). Will the rebuild make a distinction between recorded movies and recorded tv shows so that one ends up as imported movies and the other as imported tv shows.

Any other issues I should be aware of? Thanks.

[PLUCKYHD]

You pretty much covered it all. The tv
will only be imported and marked as tv if you use bmt. Also I am not sure if bmt can relocate data on the tv files unless you rename them that will be the biggest ouch.

If you don't care about that you could simply
move all them to a "tv" named folder and use file system view.

I would keep that corrupted wiz bin just in cAse sagetv can give you a better fix.

[tmiranda]

To get the recorded TV files to show up under recordings you can go back to the bad wiz.bin, use the web interface to create .xml files, go back to the new wiz.bin, and use the web server to load the .xml files.

There may be a way to export/import the Favorites using the web interface as well, I don;t really rememebr off the top of my head.

[Fuzzy]

Yes, I believe the web interface's xml will include favorites, so you SHOULD be able to recover those into the new wiz.bin as well. Only thing you will probaly lose from the wiz.bin is history (important only if you use intelligent recording) and watched status.

[Islander]

Thanks for steering me in the right direction as usual.

Homework for the weekend!

[Islander]

Ran into an issue:

http://forums.sagetv.com/forums/show...postcount=2479

Is there any other way to extract the xml info for recordings from the wiz.bin? I believe I have seen one or more importers but do not recall seeing any exporters.

Thanks.

[Fuzzy]

Use the web interface. Pretty much any page that has any sort of recording list has an orange 'XML' icon on it.

[Islander]

That is the problem ALL xlm work BUT the one for the recordings. See that post for the error.

[Fuzzy]

I'v ehad a problem once before where I couldnt' get the XML (I think it gave a Error 500). It ended up being a SINGLE bad recording (actually, the recording was fine, but the data was missing. The name of the recording was pretty null. In the sage interface, this showed up as an 'unknown' recording. Deleting that recording fixed it and the others could be grabbed fine.