SageTV Community  

Go Back   SageTV Community > SageTV Development and Customizations > SageTV v7 Customizations > Phoenix

Notices

Phoenix This forum is for discussing the user-created Phoenix custom interface for SageTV.

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 04-27-2012, 04:53 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Easy way to speed up Fanart browsing

All,

if you're looking for speeding up the Fanart browsing on extenders by caching the whole fanart directory on a RAM Disk drive without spending a penny, then please read through:
  • Download OSFMount from here
  • Install the OSFMount with defaults, but please do not run the OSFMount as requested during the installation process.
  • Unzip the "FanartRamDisk.zip" file in attachment into your SageTV installation folder.
  • Edit the "FanartRamDisk.bat" batch file and configure your RAM Disk drive size, Fanart source directory and RAM Disk drive letter as appropriate (Default is F:\).
  • Run the "FanartRamDisk.bat" batch file (WARNING! The RAM Disk drive is formatted, so please make sure the assigned drive letter is not used already).
  • You should now have a copy of your current fanart directory into the RAM disk drive.
  • Configure your extender's fanart directory in system setup to point to the root of your newly created RAM Disk drive.
  • Browse your video collection and see the improvement.

To update the fanart RAM Disk cache, just run the batch script again or automate the process as you wish.

There is a parameter in the batch script that you need to experiment a little bit and is the "RAMDiskAllocationUnitSize". For my usage I noticed that 8192 bytes is a good compromise, but it depends on the average size of your Fanart files really. Valid values are only the one listed and must be entered as is, just change the value in the batch file, run it and browse to see the difference.

Please let me know whether it did help...

Enjoy!
Eddy
Attached Files
File Type: zip FanartRamDisk.zip (517 Bytes, 157 views)
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 04-27-2012 at 07:01 PM. Reason: Warning about formatting the RAM Disk drive
Reply With Quote
  #2  
Old 04-29-2012, 01:52 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,953
Based on the batch file, it looks like you are only redirecting the source fanart folder to the ramdrive. This is only partially useful, as all the fanart heavy UI's use scaled caching of the images, which in most cases are stored in the SageTV/userdata or SageTV/cache folders. So, while it might be faster on the very first time an image is accessed, it is then back to harddrive speeds on subsequent loads. You'd be better off leaving full fanart folder alone, and ramdisking the cache folder, or what i do, and just run a massive disk cache that is larger than pretty much everything that is accessed by sage.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #3  
Old 04-29-2012, 02:59 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Quote:
Originally Posted by Fuzzy View Post
Based on the batch file, it looks like you are only redirecting the source fanart folder to the ramdrive. This is only partially useful, as all the fanart heavy UI's use scaled caching of the images, which in most cases are stored in the SageTV/userdata or SageTV/cache folders. So, while it might be faster on the very first time an image is accessed, it is then back to harddrive speeds on subsequent loads. You'd be better off leaving full fanart folder alone, and ramdisking the cache folder, or what i do, and just run a massive disk cache that is larger than pretty much everything that is accessed by sage.
Partially true, in my case the Fanart background scale and poster scale options are disabled and all files in the Fanart directory are already of a good size so that they never get scaled (a part form the small background which get only scaled after you stop browsing for more than 1 second and stored into the Fanart cache), this way I can take full advantage of the RamDisk. I've tried to redirect the SageTV/userdata or SageTV/cache folders into the RamDisk as well, by changing the path in the Sage.properties without success as it looks like that those directories are always relative to the Sage installation dir, but very happy to be proven wrong

Bear in mind that this solution is meant to be a free option, not willing to shed $80 dollars on what your're using, which since you said you have your write cache disabled I don't understand how it could be on any help on ramdisking the Fanart cache folder, unless I completely misunderstood

However, if anyone is brave enough (and got an UPS), and want to do a similar thing as you do, you can still use this free solution like this:

- Server PC starts up in the morning
- Create the RamDisk
- Copy the entire SageTV directory into RamDisk for the day.
- Run SageTV from RamDisk.
- On the evening stop SageTV service copy the Win.biz/Wiz.bak and properties files back into HDD.
- Switch server PC off for the night.

Eddy
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 04-29-2012 at 05:23 PM.
Reply With Quote
  #4  
Old 04-29-2012, 05:42 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,953
You should be able to mount your ramdrive to the sage cache folder location, that way it is transparent to sage. Not as smooth or simple as a real ram cache, but it should work.

Still, nothing compares to a write-through mirror mode cache of an entire volume.. :-)
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #5  
Old 04-29-2012, 05:49 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Quote:
Originally Posted by Fuzzy View Post
You should be able to mount your ramdrive to the sage cache folder location, that way it is transparent to sage. Not as smooth or simple as a real ram cache, but it should work.
Are you talking about this which allows to make a Windows folder virtual? With Linux is dead easy...
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard
Reply With Quote
  #6  
Old 04-29-2012, 06:29 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
OK, I'm going to use Windows junctions to cache the "userdata\Phoenix\cache" into the RamDisk tomorrow (time to go to bed for me now) and if it works properly will post a new batch file...

Eddy
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard
Reply With Quote
  #7  
Old 04-29-2012, 06:42 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,953
I was referring to actually mounting the ramdisk itself to C:\Program Files (x86)\SageTV\SageTV\cache. Volumes don't HAVE to be mounted to drive letters, that's just the convention. They can also be mounted to any existing (empty) NTFS folder, just like in Linux.

That said, this is very inflexible, and using directory junctions would be more versatile (since you'd be able to put more than one folder on a ramdisk).

Speaking of flexible, FancyCache is 'free' while in beta - might be worth a look.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #8  
Old 04-29-2012, 07:14 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Quote:
Originally Posted by Fuzzy View Post
I was referring to actually mounting the ramdisk itself to C:\Program Files (x86)\SageTV\SageTV\cache. Volumes don't HAVE to be mounted to drive letters, that's just the convention. They can also be mounted to any existing (empty) NTFS folder, just like in Linux.
Apparently cannot be done with OSFMount, the volume produced is not shown in the "Disk Management"
Quote:
That said, this is very inflexible, and using directory junctions would be more versatile (since you'd be able to put more than one folder on a ramdisk).
Agree and this is what I'm going to try...
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 04-30-2012 at 04:51 AM.
Reply With Quote
  #9  
Old 04-30-2012, 07:06 AM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Quick update

I managed to get the Windows directory junction working, therefore now even the userdata\Phoenix\cache directory is transparently cached in the RamDisk together with fanart.

I'll do some testing tonight and report back results...

Eddy
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 04-30-2012 at 07:14 AM.
Reply With Quote
  #10  
Old 04-30-2012, 10:17 AM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
re: How to do the same with clients

I forgot to mention that this solution is currently designed for extenders only; if you want to enable the SageTV clients to use the cached Fanart as well you must enable the Windows share on the newly created RamDisk and change the Fanart UNC path obviously.

Please look here for instructions on how to do it from the command line...

Eddy
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 04-30-2012 at 10:30 AM.
Reply With Quote
  #11  
Old 04-30-2012, 10:43 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 6,490
Will this still speed things up if your Sage OS is on a SSD?
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #12  
Old 04-30-2012, 11:05 AM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Quote:
Originally Posted by wayner View Post
Will this still speed things up if your Sage OS is on a SSD?
Generally speaking nothing is faster than memory, however it all depends on:

- How good is the OFSMount implementation (no idea).
- How big are you fanart files.
- Whether you have the scaling of your fanart enabled or not (mine are all disabled as I produce the right size at the source ignoring BMT, so posters are 256x360 and backgrounds 1024x576 looking good on a 110" screen)

I had however noticeable improvement without having the Windows junction enabled yet, but doing more tests tonight with the latests script so whether the userdata being cached as well improves even further.

Eddy
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard
Reply With Quote
  #13  
Old 04-30-2012, 11:38 AM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,953
Quote:
Originally Posted by routerunner View Post
I forgot to mention that this solution is currently designed for extenders only; if you want to enable the SageTV clients to use the cached Fanart as well you must enable the Windows share on the newly created RamDisk and change the Fanart UNC path obviously.

Please look here for instructions on how to do it from the command line...

Eddy
Client caching is actually best done locally on the client. Not to say you couldn't just as easily utilize a ramdisk on the client as well (using this same script). Just have to run a sync job between the client and the server's fanart folders.

I'm actually running fancycache on my client, which does a wonderful job of caching the local copy of my server's fanart folder, which is kept in sync using windows offline storage.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #14  
Old 04-30-2012, 04:07 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
re: New batch file with junctions

Folks, as promised I've done some testing with the junctions enabled and for the time being I can consider myself happy. I believe the next room of improvement will be is to cache the "WIZ" as well, however not brave enough to mess about it, will you?

Please find in attachment my version of the batch which of course you need to adapt to you circumstances, also the batch contains junctions for the standard thumbnails as well (pictures, recordings).

You might consider to split the file in different parts to allow independent update of the different cache for instance, but is up to you.

The only extra instruction is that you need to download the junction software in attachment as well. Default installation directory of the junction is "C:\Program Files\Junction\" as the batch assumes that directory, just unzip the file content in that directory.

Please have a go and post your result good or bad, here...

Have fun
Eddy
Attached Files
File Type: zip FanartRamDisk.zip (660 Bytes, 129 views)
File Type: zip Junction.zip (77.8 KB, 151 views)
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 04-30-2012 at 04:29 PM.
Reply With Quote
  #15  
Old 04-30-2012, 04:17 PM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,953
Quote:
Originally Posted by routerunner View Post
Folks, as promised I've done some testing with the junctions enabled and for the time being I can consider myself happy. I believe the next room of improvement will be is to cache the "WIZ" as well, however not brave enough to mess about it, will you?
I run a read-only cache my entire sagetv installation (along with a few other items). It's no more dangerous than running completely cache-less. Writes (the risky part) are still done direct to disk. I have no concerns about THAT causing corruption. I have, of course, had a couple wiz.bin corruptions over the 7 years I've used the same wiz.bin, but I've always had a backup for it. These corruptions occurred long before I even thought about doing any sort of supplemental caching.

Backup strategy is important whether you cache in ram or not.
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #16  
Old 05-08-2012, 05:57 AM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
re: Update

I did more experiment and so far it looks like that the best speed on extenders is achieved by leaving the Fanart repository on HDD/SSD and just put the "cache" and "thumbnails" folder in the RAM Disk as per the second version of the "FanartRamDisk" script. This way I didn't see any "spinning circles" at all

EDIT: I reckon that a possible explanation is because when reading fanart from HDD/SSD they are cached naturally by the RAM on board of the HDD/SSD controller, however if you perform write operations (by generating the fanart cache as well) on the same drive at the same time the RAM on board of the HDD/SSD controller gets invalidated and the interface can clog up easily on SSD and on HDD there also the mechanical movement that makes this delay more evident; by moving everything on RAM disk you have a similar effect but mitigated by the fact that no mechanical part are involved, however by splitting as suggested you get the best of both worlds which has the benefit of not interfering with each other and not performing any write operation on your HDD/SSD.
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 05-08-2012 at 08:40 AM. Reason: added my explanation, + more comments
Reply With Quote
  #17  
Old 05-08-2012, 09:49 AM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
Thanks for your work on this. I'm thinking of trying this out this weekend. I have an SSD, but I still get the occasional spinning circles and it's the most annoying thing about Sage right now.

Have you updated the original post with the latest scripts? I haven't had a chance to dig into the files yet and some of the discussion here has me a little lost.
Reply With Quote
  #18  
Old 05-08-2012, 10:28 AM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
re: FanartRamDisk v3 script

As requested, please find in attachment version 3 of the FanartRamDisk script. Basically I've just commented out the part that copy the Fanart files into your RamDisk drive, the rest of the script is the same as v2. Also, despite the file size, the script is not evil at all

To summarise you need to do this:
  • Download OSFMount from here
  • Install the OSFMount with defaults, but please do not run the OSFMount as requested during the installation process.
  • Download the junction software in attachment to post #14 of this thread.
  • Install the junction software in "C:\Program Files\Junction\" as the batch assumes that directory.
  • Unzip the "FanartRamDisk v3.zip" file in attachment into your SageTV installation folder.
  • Edit the "FanartRamDisk.bat" batch file and configure your RAM Disk drive size and RAM Disk drive letter as appropriate (Default is F:\ WARNING! The RAM Disk drive will be formatted, so please make sure the assigned drive letter is not used already).
  • Run the "FanartRamDisk.bat" batch file.
  • You should have two directories in your new RAM Disk drive which will be soft-linked to their original place into your SageTV installation directory.

Please let me know how you're getting on...

Eddy
Attached Files
File Type: zip FanartRamDisk_v3.zip (666 Bytes, 129 views)
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 05-08-2012 at 02:03 PM.
Reply With Quote
  #19  
Old 05-08-2012, 02:36 PM
Dargason Dargason is offline
Sage Expert
 
Join Date: Oct 2003
Posts: 516
Excellent, thanks!

I'll give it a try this weekend (or earlier, if I get a chance.) About how much RAM does this require? I have an extra 4GB I can install if needed... just haven't had a reason to do it.
Reply With Quote
  #20  
Old 05-08-2012, 04:54 PM
routerunner's Avatar
routerunner routerunner is offline
Sage Icon
 
Join Date: May 2008
Location: Wiltshire, UK
Posts: 1,343
Quote:
Originally Posted by Dargason View Post
Excellent, thanks!

I'll give it a try this weekend (or earlier, if I get a chance.) About how much RAM does this require? I have an extra 4GB I can install if needed... just haven't had a reason to do it.
There isn't a rule, it really depends on your usage and how long you keep your server on...for instance take the size of your current "C:\Program Files\SageTV\SageTV\GeneratedThumbnails" directory and "C:\Program Files\SageTV\SageTV\userdata\Phoenix\cache\imagetrans" sum them together and multiply by two, that will give you some idea as a start.

In my case my movie collection is about a couple of thousand movies, however my server is shutting down every night and my current RAM Disk size is just 128Mb and is a huge size compared on what I'm really using just browsing my movie collection or recordings.
__________________

Automatic Power Off | Squeezeslave | DVB-S Importer | DVB Decrypter & Card Client | Tuner Preroll


Every man is a damn fool for at least five minutes every day; wisdom consists in not exceeding the limit. ~ Elbert Hubbard

Last edited by routerunner; 05-08-2012 at 05:18 PM.
Reply With Quote
Reply

Tags
cache, extender, fanart, ramdisk


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
TV Recordings - large thumbnail fanart displayed on top of poster fanart mkanet Diamond 5 03-25-2011 03:33 PM
BMT / Fanart / Themes Leveraging Fanart? Addict SageTV v7 Customizations 2 09-09-2010 05:38 AM
Fanart: Complete Fanart Collection... 4000 Movies and TV shows mkanet SageTV Customizations 24 04-20-2010 03:31 PM
Can someone who uses fanart please answer an easy question mkanet SageMC Custom Interface 5 06-15-2009 04:59 PM
SageMC, Fanart, MediaScraper & Central FanArt folder structure Ė Help requested rmac321 SageMC Custom Interface 24 06-14-2009 02:00 PM


All times are GMT -6. The time now is 03:56 PM.


Powered by vBulletin® Version 3.8.7
Copyright ©2000 - 2018, vBulletin Solutions, Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.