SageTV Community  

Go Back   SageTV Community > SageTV Products > SageTV Software

Notices

SageTV Software Discussion related to the SageTV application produced by SageTV. Questions, issues, problems, suggestions, etc. relating to the SageTV software application should be posted here. (Check the descriptions of the other forums; all hardware related questions go in the Hardware Support forum, etc. And, post in the customizations forum instead if any customizations are active.)

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 04-26-2015, 04:56 PM
speck55 speck55 is offline
Sage Advanced User
 
Join Date: Apr 2011
Location: Milwaukee, WI
Posts: 156
WIZ.BIN 50MB? Tips to clean up years worth of data?

Forgive the obviousness of this post, but forum searches of wiz.bin size or wiz.bin clean up and so on yielded zero results - in fact any search of wiz.bin reformatted wiz.bin to be in double-quotes. Weird.

WIZ seems to be 50MB or so on the server.

I think I have had increasing spinny circles of death, at random - long service start times, and mainly when sitting at a PC client and picking a TV recording to play. Spins for 30 seconds to a minute, then I have to hit play again, then a black window This is years worth of watched/unwatched data, and probably a couple of thousand imported movies whose paths no longer exist, and so on.

I remember a couple of years ago reading a post or two on running metadata tools and/or setting a value in .PROPERTIES to prune stuff out of WIZ, as I move toward moving Sage onto a bit more powerful (ok, way overkill) machine.

Anybody have advice on slimming that DB down?
Reply With Quote
  #2  
Old 04-27-2015, 06:30 AM
Tiki's Avatar
Tiki Tiki is offline
Sage Icon
 
Join Date: Feb 2005
Location: Southwest Florida, USA
Posts: 1,992
I don't think there is much you can do, other than to delete it and start over (which I don't recommend). Wiz.bin is a proprietary / closed database format. Sage exposes some APIs that allow developers to read/write data to/from the database, but I don't think there are any general-purpose repair/cleanup tools.

50MB really isn't much space on a media server (my wiz.bin is currently about 40MB, after running Sage for over 10 years). Consider that a typical 1 hour TV show recorded in HD takes roughly 4-6 GB (more than 1000 times as much space!), and you start to realize that space isn't a concern.

Performance is a different issue. I honestly don't know how much a large wiz.bin file impacts performance. I suspect the scheduling engine creates a bigger hit (if you have a lot of favorites, a lot of channels, and allow Sage to look at the schedule a long way in the future).

Deleting wiz.bin would be one way to see how much impact it has on performance. However, the downside is that wiz.bin stores all your favorites and tracks which shows you've watched (so it doesn't record them again). Losing this data would be a major setback for most people.
__________________
Server: Ryzen 2400G with integrated graphics, ASRock X470 Taichi Motherboard, HDMI output to Vizio 1080p LCD, Win10-64Bit (Professional), 16GB RAM
Capture Devices (7 tuners): Colossus (x1), HDHR Prime (x2)
,USBUIRT (multi-zone)
Source:
Comcast/Xfinity X1 Cable
Primary Client: Server Other Clients: (1) HD200, (1) HD300
Retired Equipment: MediaMVP, PVR150 (x2), PVR150MCE,
HDHR, HVR-2250, HD-PVR
Reply With Quote
  #3  
Old 04-27-2015, 07:34 AM
will will is offline
Sage Fanatic
 
Join Date: Jul 2009
Location: New York
Posts: 798
I believe the wiz.bin affects start up time the most. So the larger the wiz.bin the slower the service starts.

I don't know if I am making this up but I recall reading that a few years ago on the forum (your question comes up every now and then).
__________________
Will

OS: Windows 7
Hardware: Intel Core i7-920 with 12GB RAM & an Adaptec 5805 with a Chenbro 36-port SAS Expander
Case: Antec 1200 with 4 iStarUSA trayless hot-swap cages (20 drives max)
Drives: 8 Toshiba/Hitachi 2TB drives in a RAID 6 & 7 Toshiba 3TB drives in a RAID 6
Capture Cards: HDHomeRun Connect Quatro 4, Hauppauge 60 HD-PVR
Players: 5 HD300s, 2 HD200s
Reply With Quote
  #4  
Old 04-27-2015, 07:50 AM
stanger89's Avatar
stanger89 stanger89 is offline
SageTVaholic
 
Join Date: May 2003
Location: Marion, IA
Posts: 15,188
Quote:
Originally Posted by will View Post
I believe the wiz.bin affects start up time the most. So the larger the wiz.bin the slower the service starts.
And limited_carny_init resolves that:
http://forums.sagetv.com/forums/show...ted_carny_init

Only side effect of that is intelligent recordings aren't schedule for a "while" after Sage starts.

The only time I've had trouble with spinning rings and the wiz.bin is when I was using comskip plugin, and some of my paths had changed and I had files in the wiz.bin that were no longer accessible. The comskip plugin would try to find the associated comskip files and would have to wait to timeout since the path didn't exist.

You can fix that by setting:
seeker/disable_root_path_existence_check=true
http://forums.sagetv.com/forums/show...ble_root_path*

It will let Sage remove files that it can't find, regardless of if it can access the path or not (normally it won't remove files it can't access the path for, to preserve network files that are temporarily missing due to network issues).

Just make sure all your network paths are online before you try that.
Reply With Quote
  #5  
Old 04-27-2015, 10:33 AM
waynedunham waynedunham is offline
Sage Icon
 
Join Date: Sep 2004
Posts: 1,466
Wow, I'm surprised at how big your Wiz.bin is. I'm using my original Wiz.bin migrated forward through the years and it is only 44mb.
Maybe it is because I do not save much of anything. Watch it and delete it. I have a handful of ripped DVDs from my collection. Ones I watch over and over (Galaxy Quest is one) and a few personal videos recorded on my video camera, phone, etc.
Then I have about a half a page (some folders with multiple files) worth of archived recordings.

I had started down the road of ripping all my DVDs, but then sat back and thought about how many of them I watched multiple times and only ripped those DVDs rather than "wasting" a lot of HD space.
Of course I don't have kids either, so I don't need to keep thousands of hours of TV shows and movies available at a seconds notice based on a child's whim.
__________________
Wayne Dunham

Last edited by waynedunham; 04-27-2015 at 10:34 AM. Reason: Hey, it may have been the wrong word, but it passed spellcheck!
Reply With Quote
  #6  
Old 04-27-2015, 11:21 AM
Tiki's Avatar
Tiki Tiki is offline
Sage Icon
 
Join Date: Feb 2005
Location: Southwest Florida, USA
Posts: 1,992
Quote:
Originally Posted by waynedunham View Post
Wow, I'm surprised at how big your Wiz.bin is. I'm using my original Wiz.bin migrated forward through the years and it is only 44mb.
Maybe it is because I do not save much of anything. Watch it and delete it.
Sounds like your Wiz.bin is about the same size as ours (mine was ~40MB, and OP's was ~50MB).

Deleting shows after viewing won't really have any effect on wiz.bin size. Wiz.bin doesn't store the actual shows - just information/metadata, and that data gets retained for stuff you watch, even after you delete a show.
__________________
Server: Ryzen 2400G with integrated graphics, ASRock X470 Taichi Motherboard, HDMI output to Vizio 1080p LCD, Win10-64Bit (Professional), 16GB RAM
Capture Devices (7 tuners): Colossus (x1), HDHR Prime (x2)
,USBUIRT (multi-zone)
Source:
Comcast/Xfinity X1 Cable
Primary Client: Server Other Clients: (1) HD200, (1) HD300
Retired Equipment: MediaMVP, PVR150 (x2), PVR150MCE,
HDHR, HVR-2250, HD-PVR
Reply With Quote
  #7  
Old 04-27-2015, 12:28 PM
reggie14 reggie14 is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Maryland
Posts: 2,760
50MB does seem a little big. I've been running off the same wiz.bin file since 2003, and mine is still only 28MB. I've noticed the size does fluctuate a bit, but I don't think I've ever been above 50MB. But then again, I've never had a huge number of imported videos- 1000 at most.

I doubt the slowdowns are caused by the wiz.bin being large. Running out of space on the heap might be more likely. Did you increase your java heap size to 1GB?

I think you can "clean" up the wiz.bin by setting the advanced_video_file_recovery setting in sage.properties, but that's really more intended to deal with situations like when your moving Sage to a new computer.
Reply With Quote
  #8  
Old 04-27-2015, 12:58 PM
nyplayer nyplayer is offline
SageTVaholic
 
Join Date: Sep 2005
Posts: 4,799
I am not sure how Sagetv deals with the WIZ.BIN most databases have to be compressed (Compact) to recover empty space. I know on all my Sqlite DB I do a vacuum for example my SJQ DB. I do this with my NEXTPVR DB also... My Sage wiz.bin is over 50meg. It is not a case of how long you have had the Sage DB but how much you have recorded and number of favorites. Also deletions rename etc... Just because some one has had it longer really has nothing to do with the Size.

So if it is not being maintained within Sage there is really nothing you can do but rebuild it and just import the recordings. Of course you will lose your watched status etc..

if you are feeling brave.

http://forums.sagetv.com/forums/show...05&postcount=3
__________________
Windows 10 I3 Dual Core 3.0 Server, SageTV9... OpenDCT with 2 HDHRPrimes and 4 HDHRconnects for a total of 14 Tuners.

Last edited by nyplayer; 04-27-2015 at 01:20 PM.
Reply With Quote
  #9  
Old 04-27-2015, 01:14 PM
waynedunham waynedunham is offline
Sage Icon
 
Join Date: Sep 2004
Posts: 1,466
Quote:
Originally Posted by Tiki View Post
Sounds like your Wiz.bin is about the same size as ours (mine was ~40MB, and OP's was ~50MB).

Deleting shows after viewing won't really have any effect on wiz.bin size. Wiz.bin doesn't store the actual shows - just information/metadata, and that data gets retained for stuff you watch, even after you delete a show.
Then mine should be HUGE! I watch a LOT of recorded TV. It is why I NEED 5-8 tuners. Server A has an HDHR-Prime (3 tuners) 2 HD-PVR/STBs and a single original HDHR for OTA (2 rarely used tuners since I have such horrendous reception here). Server B (my old setup) has an HDHR Prime (3 tuners) and an original HDHR (again, OTA only used as a last resort)
With that, it takes a lot of manual manipulation of setting manual recordings of some favorites to make room for other favorites that don't re-air. I do a lot of that to try and keep the HD-PVR/STB and original HDHR recordings to a minimum. The OTA stuff is definitely hit and miss. One day I'll have 100% full rock solid reception from my attic antenna and another day it will barely tune that same channel. And the HD-PVRs are still kind of flakey, the more you use them the more likely they'll lock up and need a power cycle.
But on many nights all 3 Prime tuners are chock full all night long.
You can tell the summer is coming though. I've only got 50 shows in the queue over the next 72 hours on the "A" server. (and yes, I watch it ALL!)
__________________
Wayne Dunham
Reply With Quote
  #10  
Old 04-27-2015, 02:01 PM
Mark57's Avatar
Mark57 Mark57 is offline
Sage Advanced User
 
Join Date: Sep 2007
Location: OKC
Posts: 113
Quote:
Originally Posted by waynedunham View Post
Then mine should be HUGE! I watch a LOT of recorded TV. It is why I NEED 5-8 tuners. Server A has an HDHR-Prime (3 tuners) 2 HD-PVR/STBs and a single original HDHR for OTA (2 rarely used tuners since I have such horrendous reception here). Server B (my old setup) has an HDHR Prime (3 tuners) and an original HDHR (again, OTA only used as a last resort)
With that, it takes a lot of manual manipulation of setting manual recordings of some favorites to make room for other favorites that don't re-air. I do a lot of that to try and keep the HD-PVR/STB and original HDHR recordings to a minimum. The OTA stuff is definitely hit and miss. One day I'll have 100% full rock solid reception from my attic antenna and another day it will barely tune that same channel. And the HD-PVRs are still kind of flakey, the more you use them the more likely they'll lock up and need a power cycle.
But on many nights all 3 Prime tuners are chock full all night long.
You can tell the summer is coming though. I've only got 50 shows in the queue over the next 72 hours on the "A" server. (and yes, I watch it ALL!)
Thread sidebar -- regarding your attic antenna reception. My attic roofing panels have the silver mylar coating on it. I took a drill mounted rotary wire brush and scraped it off where it was in a direct line of site with the antenna farm and my attic antenna, I only needed to do a 3' x 3' area due to the proximity of my attic antenna and the inside of the roof. I used a Digiair Pro meter to test before and after and my signal did improve. All my OTA signals are great and account for 85% of my recordings.
__________________
Mark
____________________
Server: Scratch built server, 16GB RAM, Nvida GeForce 8600 GTS, 6 HD Homeruns, 32 TB Storage, Windoz 10 64 bit, Sage 64 bit 9.2.1, Sage HD-100 & 200 Extenders
Reply With Quote
  #11  
Old 04-27-2015, 04:16 PM
waynedunham waynedunham is offline
Sage Icon
 
Join Date: Sep 2004
Posts: 1,466
Quote:
Originally Posted by Mark57 View Post
Thread sidebar -- regarding your attic antenna reception. My attic roofing panels have the silver mylar coating on it. I took a drill mounted rotary wire brush and scraped it off where it was in a direct line of site with the antenna farm and my attic antenna, I only needed to do a 3' x 3' area due to the proximity of my attic antenna and the inside of the roof. I used a Digiair Pro meter to test before and after and my signal did improve. All my OTA signals are great and account for 85% of my recordings.
I only wish I had your luck. I'm on the "wrong" side of the hill. On the other side of the hill is where all the broadcast antennas are. My house is old and doesn't have any insulation, or covering on the underside of the roof. The only insulation is between the ceiling joists. Nothing in the rafters at all.
The attic antenna is better than an indoor antenna, but not by a whole lot.

I would like to look into an external mast antenna, but hesitate to spend the $$$ if it isn't going to net me a major increase in reception. To give an indication of just how bad my reception is sitting down in "the bowl" I'm in. Most of the antennas I can get a signal from are on top of a mountain only about 10 miles away!

And I have tried the antenna outside as well and it didn't make much difference at all over what I am getting in the attic. I'm going to need a mast that will get me 30-50' off the ground I"m guessing.
__________________
Wayne Dunham
Reply With Quote
  #12  
Old 04-28-2015, 10:43 AM
Monedeath Monedeath is offline
Sage Expert
 
Join Date: Sep 2009
Location: Idaho
Posts: 514
Quote:
Originally Posted by waynedunham View Post
Wow, I'm surprised at how big your Wiz.bin is. I'm using my original Wiz.bin migrated forward through the years and it is only 44mb.
The SageTV install I'm responsible for has had it wiz.bin reset once already in the past year to address the same complaints the OP has, and after the rebuild/reimport of everything it already was in the 40mb range, albeit MUCH more responsive.

Then our local provider(Cable One) started making frequent changes to our physical channel lineups once they went full digital, resulting in our CableCard(HD HomeRun Prime) becoming the only reliable means of tuning into their programming offerings.

So the HDHomeRun Prime gets replaced with a Ceton (6) tuner, while a pair of HD HomeRuns were added to grab the locals OTA. (two Hauppauge tuners (an 1800 and 2250) retired in the process) and we're back in the realm of the spining circle once more. Of course, we've gone from 9 tuners to 8 tuners to 12 tuners at this point which probably puts our SageTV Server in the category of being one of the larger pure recorders out there in terms of tuners. (Something like multirecord for SageTV would allow retirement/re-purposing for a few tuners as it's mainly record padding that spurred the justification for so many tuners)

I suspect part of our current issue is the overhead with the Ceton tuner as that seemed to be where the server took the single largest and most noticeable performance hit. 32-bit Win7 doesn't seem to have enough addressable memory to handle a large SageTV library + 12 tuners. (A Ceton PCIe (6) tuner card, another Hauppauge 2250 card, and its assorted collection of drives.) Moving to a 64 bit install and more RAM is now on that server's agenda, just have to work out the logistics on how/when I'm going to implement that shift.

In the meantime, other recordings and content are being shuffled off to a newer Plex/File Server that exists in space that SageTV is being kept unaware of to try to help bring the wiz.bin down to a more manageable size, with 26,616.28GB worth of recordings currently tracked in Sage, that might take awhile.
Reply With Quote
  #13  
Old 04-29-2015, 01:55 PM
waynedunham waynedunham is offline
Sage Icon
 
Join Date: Sep 2004
Posts: 1,466
Quote:
Originally Posted by Monedeath View Post
The SageTV install I'm responsible for has had it wiz.bin reset once already in the past year to address the same complaints the OP has, and after the rebuild/reimport of everything it already was in the 40mb range, albeit MUCH more responsive.

Then our local provider(Cable One) started making frequent changes to our physical channel lineups once they went full digital, resulting in our CableCard(HD HomeRun Prime) becoming the only reliable means of tuning into their programming offerings.

So the HDHomeRun Prime gets replaced with a Ceton (6) tuner, while a pair of HD HomeRuns were added to grab the locals OTA. (two Hauppauge tuners (an 1800 and 2250) retired in the process) and we're back in the realm of the spining circle once more. Of course, we've gone from 9 tuners to 8 tuners to 12 tuners at this point which probably puts our SageTV Server in the category of being one of the larger pure recorders out there in terms of tuners. (Something like multirecord for SageTV would allow retirement/re-purposing for a few tuners as it's mainly record padding that spurred the justification for so many tuners)

I suspect part of our current issue is the overhead with the Ceton tuner as that seemed to be where the server took the single largest and most noticeable performance hit. 32-bit Win7 doesn't seem to have enough addressable memory to handle a large SageTV library + 12 tuners. (A Ceton PCIe (6) tuner card, another Hauppauge 2250 card, and its assorted collection of drives.) Moving to a 64 bit install and more RAM is now on that server's agenda, just have to work out the logistics on how/when I'm going to implement that shift.

In the meantime, other recordings and content are being shuffled off to a newer Plex/File Server that exists in space that SageTV is being kept unaware of to try to help bring the wiz.bin down to a more manageable size, with 26,616.28GB worth of recordings currently tracked in Sage, that might take awhile.
Be careful how many network folders you have in your SageTV paths (import, recordings, etc). I noticed a lot more spinning circles before I removed some of my network paths looking at other machines (in my houses internal wired gigabit lan).
I'm not sure how that compares with any Wiz.bin size issues, but I know it helped me to limit the Network folders SageTV looks at.
__________________
Wayne Dunham
Reply With Quote
  #14  
Old 04-29-2015, 04:21 PM
Monedeath Monedeath is offline
Sage Expert
 
Join Date: Sep 2009
Location: Idaho
Posts: 514
Quote:
Originally Posted by waynedunham View Post
Be careful how many network folders you have in your SageTV paths (import, recordings, etc). I noticed a lot more spinning circles before I removed some of my network paths looking at other machines (in my houses internal wired gigabit lan).
I'm not sure how that compares with any Wiz.bin size issues, but I know it helped me to limit the Network folders SageTV looks at.
Actually not in line with some of our experience. We actually saw some slight improvements in performance once we started breaking things into smaller (sub-directory//media import directory) containers as a precursor to moving it to Plex rather than maintaining the original flat file system Sage is setup to do on its own. Of course, that may have to do with the size of the hard drive being used and the number of files present as to when that quirk can start to crop up.

Performance wise, usually the deciding factor on that is more likely to be the networked computer/devices power saving settings in regards to Hard Drive sleep modes and how long it takes the drive to spin up. I know we have disks that are getting a little long in the tooth and are starting to report 11+ second spinup times on them, and they have been linked to a number of the spinning circle encounters we've had(the media we wanted to watch was on that drive), but as the other health indicators are fine they're simply being watched rather than replaced(that and replacing 3+TB drives isn't exactly cheap in either money or time taken to transition things across).
Reply With Quote
  #15  
Old 04-29-2015, 06:25 PM
KryptoNyte's Avatar
KryptoNyte KryptoNyte is offline
SageTVaholic
 
Join Date: Dec 2006
Posts: 2,636
I wonder if the wiz.bin file could be cracked open once this goes open source. There is a lot of data in there, and it would be nice to have some semblance of access to it, or control over it. I suppose that exposing wiz.bin to users might also introduce some serious mayhem.
Reply With Quote
  #16  
Old 04-29-2015, 06:31 PM
stanger89's Avatar
stanger89 stanger89 is offline
SageTVaholic
 
Join Date: May 2003
Location: Marion, IA
Posts: 15,188
You have access to it, the SageTV API gives you access/control over all the data in it.
Reply With Quote
  #17  
Old 04-29-2015, 08:30 PM
KryptoNyte's Avatar
KryptoNyte KryptoNyte is offline
SageTVaholic
 
Join Date: Dec 2006
Posts: 2,636
Then let me ask a more basic question; why can't the metadata be natively embedded in the media file itself as it is with WMC, providing for a slightly more universal way to access that data?
Reply With Quote
  #18  
Old 04-29-2015, 09:38 PM
reggie14 reggie14 is offline
SageTVaholic
 
Join Date: Aug 2003
Location: Maryland
Posts: 2,760
Quote:
Originally Posted by KryptoNyte View Post
Then let me ask a more basic question; why can't the metadata be natively embedded in the media file itself as it is with WMC, providing for a slightly more universal way to access that data?
FWIW, I think Sage did start embedding it in the media files with v7. But I think this is mainly used to handle cases where you're transferring files and Sage needs to re-identify a file.

I think there's a lot more data about each recording in the wiz.bin than what can be stored in the embedded metadata, though.
Reply With Quote
  #19  
Old 04-30-2015, 08:47 AM
Monedeath Monedeath is offline
Sage Expert
 
Join Date: Sep 2009
Location: Idaho
Posts: 514
Quote:
Originally Posted by reggie14 View Post
FWIW, I think Sage did start embedding it in the media files with v7. But I think this is mainly used to handle cases where you're transferring files and Sage needs to re-identify a file.

I think there's a lot more data about each recording in the wiz.bin than what can be stored in the embedded metadata, though.
There is an I/O consideration also. While it is all well and good to embed the metadata into the content itself. When it comes to larger collections of media, you want that metadata to also exist in its own database, as having your media program poke its head into every single media file that might be relevant to what your looking for in order to poll the metadata contained within every time you initiate a query is horribly inefficient for more than very small media collections.
Reply With Quote
  #20  
Old 04-30-2015, 06:20 PM
KryptoNyte's Avatar
KryptoNyte KryptoNyte is offline
SageTVaholic
 
Join Date: Dec 2006
Posts: 2,636
1) If there is metadata attached to the file, how do I look at it?

2) Regardless of whether an external database serves the needs of some, it's still beneficial to have the metadata in the video file itself.
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Wow - It's been 3 years... SteveW General Discussion 26 07-03-2014 08:43 PM
Who will still be using SageTV in five years? briankeys The SageTV Community 48 07-10-2013 02:19 PM
I've been asleep for a couple of years.. Yombute Hardware Support 9 03-15-2012 05:25 AM
After 7 years with SageTV, It may just be over mightyt SageTV Software 51 06-19-2011 05:49 PM
XP,Vista to Windows 7 and SageTV save data base reinstall tips. SHS SageTV Software 0 11-16-2009 10:13 AM


All times are GMT -6. The time now is 04:37 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.