SageTV Community  

Go Back   SageTV Community > Information & Announcements > SageTV Downloads & Instructions

Notices

SageTV Downloads & Instructions This forum is for discussions containing Downloads and Instructions related to the Open Source version of SageTV. Please do not post questions in this forum.

Reply
 
Thread Tools Search this Thread Display Modes
  #121  
Old 05-24-2017, 07:19 AM
jusjoken jusjoken is online now
SageTVaholic
 
Join Date: Dec 2005
Location: Strathmore, AB
Posts: 2,564
Quote:
Originally Posted by rickgillyon View Post
The 9.1.1.651 files are there now, but are missing the .exe extensions.
ooops. Sorry. My script has been updated and is re-running so the files with the .exe should be there in the next few hours.

k
__________________
If you wish to see what I am up to and support my efforts visit my Patreon page
Reply With Quote
  #122  
Old 05-24-2017, 07:20 AM
jusjoken jusjoken is online now
SageTVaholic
 
Join Date: Dec 2005
Location: Strathmore, AB
Posts: 2,564
Quote:
Originally Posted by gordyp View Post
That might not be all they are missing (the .exe part was easy enough to fix, just add it to the downloaded file name) - I had problems with the server download - I was getting a bunch of Java errors (I suspect the java.jar file was missing) and had to roll back to 9.1.0.648 on the server. The 9.1.1.651 client install seems fine, aside from the missing file extension...
Perhaps your download from BinTray corupted it or something else is wrong. I tested the ServerSetup and is works fine on my test system. The missing .exe is being repaired as we speak.

k
__________________
If you wish to see what I am up to and support my efforts visit my Patreon page
Reply With Quote
  #123  
Old 05-24-2017, 08:42 AM
rickgillyon's Avatar
rickgillyon rickgillyon is offline
Sage Icon
 
Join Date: Sep 2005
Location: Whitley Bay, England
Posts: 1,950
Quote:
Originally Posted by jusjoken View Post
ooops. Sorry. My script has been updated and is re-running so the files with the .exe should be there in the next few hours.
No real need, I'm sure everyone could work out to add the .exe. Thanks for the efforts, much appreciated.
__________________
unRAID Server: Intel Core i5 7600K, 48GB DDR4, 2x512GB PCIe M.2 Cache Pool, 2x10TB SATA3 Parity Drive, 3x8TB SATA Array, 1x hdHomeRun DVB-T2 Quattro, IPTV via xTeVe, unRAID 6.8.3, tvHeadEnd for recording back end, Emby
Clients: 3 Nvidia Shields, 3 FireTV, 3 Win10 Pro PC Clients
Reply With Quote
  #124  
Old 05-24-2017, 10:10 AM
sflamm sflamm is offline
Sage Icon
 
Join Date: Mar 2009
Posts: 1,653
Out of curiosity- have you measured how much affect string de-duplication has on SageTV? It doesn't seem to me that there would be a lot of shared strings ... just the titles of TV series ... what else? And how much memory were the strings taking before and after? Curious if you have any statistics to share.
Reply With Quote
  #125  
Old 05-24-2017, 10:15 AM
Fuzzy's Avatar
Fuzzy Fuzzy is offline
SageTVaholic
 
Join Date: Sep 2005
Location: Jurupa Valley, CA
Posts: 9,957
He discussed the memory benefits a bit in the github issue: https://github.com/google/sagetv/issues/282
__________________
Buy Fuzzy a beer! (Fuzzy likes beer)

unRAID Server: i7-6700, 32GB RAM, Dual 128GB SSD cache and 13TB pool, with SageTVv9, openDCT, Logitech Media Server and Plex Media Server each in Dockers.
Sources: HRHR Prime with Charter CableCard. HDHR-US for OTA.
Primary Client: HD-300 through XBoxOne in Living Room, Samsung HLT-6189S
Other Clients: Mi Box in Master Bedroom, HD-200 in kids room
Reply With Quote
  #126  
Old 05-24-2017, 10:17 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
Quote:
Originally Posted by sflamm View Post
Out of curiosity- have you measured how much affect string de-duplication has on SageTV? It doesn't seem to me that there would be a lot of shared strings ... just the titles of TV series ... what else? And how much memory were the strings taking before and after? Curious if you have any statistics to share.
It would be interesting if people share their experiences with Java heap size usage. This could be a huge benefit to people with 3+ extenders. I went to the unRAID/Linux version to solve this problem myself.

Should we be upgrading our Java to the latest V8, assuming that one is on V8 in the first place?

edit - Maybe not so new as Java 8 Update 20 was released in mid 2014 - there have been something like 50 updates since then.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server

Last edited by wayner; 05-24-2017 at 10:21 AM.
Reply With Quote
  #127  
Old 05-24-2017, 10:58 AM
sflamm sflamm is offline
Sage Icon
 
Join Date: Mar 2009
Posts: 1,653
Quote:
He discussed the memory benefits a bit in the github issue: https://github.com/google/sagetv/issues/282
The results are interesting... the source of the duplication was unexpected (to me) but makes a lot of sense.

Looks like the strings that are being repeated are the:

STV files
client configurations files
channel guide
fanart (show descriptions, ....)
etc.

that are being used across clients... most of the string values in these are the same and the GC dedup is pointing all the String classes to the same immutable char array substantially reducing the memory footprint.
This is pretty neat - as writing the SageTV to use shared references across clients normally would be very difficult and this is essentially being done behind the scenes for free.

Last edited by sflamm; 05-24-2017 at 11:08 AM.
Reply With Quote
  #128  
Old 05-24-2017, 11:00 AM
sflamm sflamm is offline
Sage Icon
 
Join Date: Mar 2009
Posts: 1,653
Quote:
Should we be upgrading our Java to the latest V8, assuming that one is on V8 in the first place?
There are multiple threads that point out everything works on Java 8 but a few items related to certain Jetty usages (not core to the product)
Reply With Quote
  #129  
Old 05-24-2017, 11:05 AM
sflamm sflamm is offline
Sage Icon
 
Join Date: Mar 2009
Posts: 1,653
Quote:
Narflex commented 18 days ago
Nice. There's already a bunch of string.intern() calls in the existing code
for areas where I spotted this as a major problem previously
These string.intern() calls should be removed. They make the deduplication less efficient. (made sense before the built-in feature)
Reply With Quote
  #130  
Old 05-24-2017, 11:45 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
Quote:
Originally Posted by sflamm View Post
There are multiple threads that point out everything works on Java 8 but a few items related to certain Jetty usages (not core to the product)
I was referring to the fact that this dedup required Java 8 update 20 - but we are WAY beyond that now so I assume that everyone on Java 8 should be good.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #131  
Old 05-24-2017, 11:50 AM
sflamm sflamm is offline
Sage Icon
 
Join Date: Mar 2009
Posts: 1,653
The fanart (images) that are used for each client device are generated by the server. I assume these take a huge amount of the server-side process memory usage.

The images are static... the only processing that is done on them is scaling. And the fanart is shared across all clients.

Would it be possible to store them internally in-memory as a String after being scaled? The data can be streamed from the String to the the clients... benefiting again greatly from the dedup....

Last edited by sflamm; 05-24-2017 at 12:06 PM.
Reply With Quote
  #132  
Old 05-24-2017, 12:50 PM
EnterNoEscape's Avatar
EnterNoEscape EnterNoEscape is offline
SageTVaholic
 
Join Date: Jun 2010
Location: Harrisburg, PA
Posts: 2,657
Quote:
Originally Posted by sflamm View Post
These string.intern() calls should be removed. They make the deduplication less efficient. (made sense before the built-in feature)
That is likely a bad idea. The deduplication works on the char[] arrays, not the String objects themselves (and they consume space per each allocated String, just some of them now share their char[] array with other String objects). intern() where we know it's a good idea is still better. This just helps us sweep up all of those corner cases and apparently there are a lot.

Quote:
Originally Posted by wayner View Post
It would be interesting if people share their experiences with Java heap size usage.
You should understand that the numbers I came up with were assembled by nudging the JVM to perform GC and deduplicate. What I have seen when you just let it do this on it's own without any coercion, is it will deduplicate, but it may or may not always clear out all of the char[] arrays at the same time; you won't really see the benefits until you're near the heap limits. So your heap allocation size isn't really going to be a good indicator of anything. What will be a better indicator is how many more miniclients you can use or how much longer everything keeps working before you see OOM errors.

For reference, on a stock SageTV install and ~60MB database, I was able to run multiple miniclients with a max heap size of only 256MB which to me is crazy low. When I looked at the objects on the heap in that configuration, it was clear the deduplication was working very effectively.

Quote:
Originally Posted by wayner View Post
I was referring to the fact that this dedup required Java 8 update 20 - but we are WAY beyond that now so I assume that everyone on Java 8 should be good.
You are correct that there is a minimum Java 8 version before this was introduced. If you try to run this on a version that's 8, but older than 20 it will just complain about how it doesn't know what to do the parameter. It should not actually prevent the software from working. Also in either case if you're on Java 8, I recommend keeping up with the latest fixes. This is a well established feature, so I don't have any worries about it needing to mature and it's easy to remove from the code if we decide it causing some horrible unforeseen issues.
__________________
SageTV v9 Server: ASRock Z97 Extreme4, Intel i7-4790K @ 4.4Ghz, 32GB RAM, 6x 3TB 7200rpm HD, 2x 5TB 7200rpm HD, 2x 6TB 7200rpm HD, 4x 256GB SSD, 4x 500GB SSD, unRAID Pro 6.7.2 (Dual Parity + SSD Cache).
Capture: 1x Ceton InfiniTV 4 (ClearQAM), 2x Ceton InfiniTV 6, 1x BM1000-HDMI, 1x BM3500-HDMI.

Clients: 1x HD300 (Living Room), 1x HD200 (Master Bedroom).
Software: OpenDCT :: WMC Live TV Tuner :: Schedules Direct EPG
Reply With Quote
  #133  
Old 05-24-2017, 01:21 PM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
Quote:
Originally Posted by EnterNoEscape View Post
You should understand that the numbers I came up with were assembled by nudging the JVM to perform GC and deduplicate. What I have seen when you just let it do this on it's own without any coercion, is it will deduplicate, but it may or may not always clear out all of the char[] arrays at the same time; you won't really see the benefits until you're near the heap limits. So your heap allocation size isn't really going to be a good indicator of anything. What will be a better indicator is how many more miniclients you can use or how much longer everything keeps working before you see OOM errors.
It will be interesting to see how this performs. On Windows with a heap size of 1024 I could run 3-4 miniclients - plus the server client (although I didn't really do anything on the server client). But over time I would start to get warnings about running out of space and and the system would slow down and it sometimes it would just completely crash. There also may have been some other memory leak issues with Windows as this seemed to get worse if the server had been running for weeks.

I now run with a heap size of 3072 and I never even reach 2000 of usage, even when trying by turning on every client I can find. But as you (or someone else mentioned) the Linux version of SageTV seems to be more efficient in memory usage. But I can probably try turning my heap size down to about 1000 and see how it does. Presumably unRAID can use the memory for something else.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #134  
Old 05-24-2017, 01:38 PM
stuckless's Avatar
stuckless stuckless is offline
SageTVaholic
 
Join Date: Oct 2007
Location: London, Ontario, Canada
Posts: 9,711
Quote:
Originally Posted by sflamm View Post
The fanart (images) that are used for each client device are generated by the server. I assume these take a huge amount of the server-side process memory usage.

The images are static... the only processing that is done on them is scaling. And the fanart is shared across all clients.

Would it be possible to store them internally in-memory as a String after being scaled? The data can be streamed from the String to the the clients... benefiting again greatly from the dedup....
No... this would not help and might actually hurt things

Images in the sagetv image system are loaded and cached and evicted from the cache... which is why sometimes you'll see spinning circles when you navigate a very UI intensive menu. If we stored them in strings... that would actually take more memory, since, we'd be keeping the image bytes in the string, but they's have to be uncompressed before you can actually work with them. Today, we only keep the uncompressed images in memory (not the raw bytes from the files). (We also cache the filenames, in phoenix, but that's different than the image contents).
Reply With Quote
  #135  
Old 05-28-2017, 03:57 PM
FreshOne FreshOne is offline
Sage Advanced User
 
Join Date: Dec 2007
Posts: 80
Quote:
Originally Posted by jusjoken View Post
Latest release available
  • The following may no longer be required after version 9.0.13.536_R1.010 and above as this may have been fixed.
    • Some users have seen that the installer FAILs to register a number of libraries. This may be related to a missing KB fix needed for VC2015 runtime. IF you run into this then see this KB article and install the related patch and then re-run the installer.
I installed 9.1.1.651 today and had like 7 or 8 files that did not register properly on my Windows 7 SP1 install. After I DLed and ran the KB fix, the Sage Install was finally able to complete successfully.

So the issue hasn't been fixed as far as I can tell.
Reply With Quote
  #136  
Old 05-29-2017, 07:26 AM
PeteCress's Avatar
PeteCress PeteCress is offline
Sage Fanatic
 
Join Date: Jan 2009
Location: Paoli, PA - USA, AKA "The Navel Of The Universe"
Posts: 842
Quote:
Originally Posted by jusjoken View Post
Is your win7 current with updates? We know this works on window 7....
I am going to make a logical leap and say that the installer probably does not work on XP - mainly because it is failing on my XP box and I do not see mention of any Windows versions except 7 and 10 in the threads I have skimmed so far.

Am I on the right track?

If so, would anybody opine whether I should "Upgrade" my XP box to 7 or 10?

I haven't heard anything good about 10, but it seems to be where things are headed.... OTOH, I know and trust 7.

Also, once I get an acceptable OS on the box, am I going to run into trouble because my SageTV is installed on D and not C: ?

The rationale for D: is that D: is for Data and C: is for System.... so I can image/re-image C: without losing data like Wiz.bin or SageTV.Properties.
__________________
Server: SageTV 9, Windows 10, i5 NUC
Clients: HD200*3 over Cat5e Ethernet + 1 slightly flakey HD 300 + 1 HD200 remote at another residence
Plugins: (none yet, looking for recommendations)
Storage: NetGear Ultra-6 NAS 10 TB total w/dual redundancy. Plus 5tb QNAP for RecordedTV.
Capture: 3 Silicon Dust HomeRun tuner boxes (6 tuners total)
Program Source: OTA antenna

Last edited by PeteCress; 05-29-2017 at 07:31 AM.
Reply With Quote
  #137  
Old 05-29-2017, 07:44 AM
wayner wayner is offline
SageTVaholic
 
Join Date: Jan 2008
Location: Toronto, ON
Posts: 7,373
Quote:
Originally Posted by PeteCress View Post
If so, would anybody opine whether I should "Upgrade" my XP box to 7 or 10?

I haven't heard anything good about 10, but it seems to be where things are headed.... OTOH, I know and trust 7.
You asked for opinions - my opinion is that you should upgrade and to Win10 as Win7 is getting pretty long in the tooth. Some folks have complained about having forced reboots on Win10 but I don't think that I have seen that happen.

Other than that I don't see a downside to Win10 and a bunch of things on the upside, such as being newer and having some features that Win7 doesn't have.

For a further opinion - all of the cool kids are moving to unRAID Linux.
__________________
New Server - Sage9 on unRAID 2xHD-PVR, HDHR for OTA
Old Server - Sage7 on Win7Pro-i660CPU with 4.6TB, HD-PVR, HDHR OTA, HVR-1850 OTA
Clients - 2xHD-300, 8xHD-200 Extenders, Client+2xPlaceshifter and a WHS which acts as a backup Sage server
Reply With Quote
  #138  
Old 05-29-2017, 08:14 AM
PeteCress's Avatar
PeteCress PeteCress is offline
Sage Fanatic
 
Join Date: Jan 2009
Location: Paoli, PA - USA, AKA "The Navel Of The Universe"
Posts: 842
Quote:
Originally Posted by wayner View Post
...you should upgrade and to Win10 as Win7 is getting pretty long in the tooth. Some folks have complained about having forced reboots on Win10 but I don't think that I have seen that happen.

Other than that I don't see a downside to Win10 and a bunch of things on the upside, such as being newer and having some features that Win7 doesn't have.

For a further opinion - all of the cool kids are moving to unRAID Linux.
Well, that resolves the 7-vs-10 question for me.

But for a hundred bucks (10 OEM price), maybe I need to re-visit the unRAID solution. Somebody had me going on it a couple weeks ago, but when I tried to create a USB stick to do the deed, I could not make it happen.

A strong appeal to me of the UnRAID solution is the USB stick: it sounds like I can leave the XP install in place and just try UnRAID without losing anything - once I get a USB stick working.... -)

Perhaps more relevant is that it seems like The-Powers-That-Be-SageTV-wise are all using UnRAID....

No problem with Un-RAID and 3-TB drives, right?
__________________
Server: SageTV 9, Windows 10, i5 NUC
Clients: HD200*3 over Cat5e Ethernet + 1 slightly flakey HD 300 + 1 HD200 remote at another residence
Plugins: (none yet, looking for recommendations)
Storage: NetGear Ultra-6 NAS 10 TB total w/dual redundancy. Plus 5tb QNAP for RecordedTV.
Capture: 3 Silicon Dust HomeRun tuner boxes (6 tuners total)
Program Source: OTA antenna
Reply With Quote
  #139  
Old 05-29-2017, 09:11 AM
graywolf's Avatar
graywolf graywolf is offline
Sage Icon
 
Join Date: Oct 2009
Location: NC
Posts: 1,366
Quote:
Originally Posted by PeteCress View Post
Also, once I get an acceptable OS on the box, am I going to run into trouble because my SageTV is installed on D and not C: ?
Just when you run the Installer, choose Options before doing the install so you can ensure it is going to the path you want. When I did the V6 > V7 > V9 upgrade on my test image, the V9 Installer did the Default (C:\Program Files (x86)\SageTV) path instead of detecting it was on D: drive.
Reply With Quote
  #140  
Old 05-29-2017, 01:12 PM
waylo waylo is offline
Sage Advanced User
 
Join Date: Feb 2006
Posts: 238
Quote:
Originally Posted by PeteCress View Post
Well, that resolves the 7-vs-10 question for me.

But for a hundred bucks (10 OEM price), maybe I need to re-visit the unRAID solution. Somebody had me going on it a couple weeks ago, but when I tried to create a USB stick to do the deed, I could not make it happen.

A strong appeal to me of the UnRAID solution is the USB stick: it sounds like I can leave the XP install in place and just try UnRAID without losing anything - once I get a USB stick working.... -)

Perhaps more relevant is that it seems like The-Powers-That-Be-SageTV-wise are all using UnRAID....

No problem with Un-RAID and 3-TB drives, right?
I think the ONLY reason to stay with Win7 is Windows Media Center. It's been discontinued past Win 8.1 Pro. If you have any need to watch DRM protected channels using a CableCard solution, that's your best bet still... unfortunately.

I myself have my HTPC/server with Win7 because I still want that option...

SageTV isn't the only DVR I have installed, but it is by far the most used.
__________________

Server: SageTV 9.2.6x64 on Win10 x64 Home, i7-2600, 8gb RAM, OpenDCT, Java 1.8, 20TB storage
Display: EVGA GT 1030 fanless to Sony LED TV via DVI-HDMI
Capture Devices: HDHR Prime for Comcast, HDHR Dual for OTA (retired)
Clients: Nvidia Shield Android miniclient
Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Open Source Windows Installer jusjoken SageTV Github Development 461 07-17-2019 01:54 PM
Windows installer and running as a service Opus4 SageTV Github Development 13 01-01-2016 07:33 AM
Any news on the Windows installer? Damstas SageTV Github Development 2 09-03-2015 10:09 AM
Web Interface - where is the Windows installer ckewinjones SageTV Customizations 2 01-09-2011 10:36 AM


All times are GMT -6. The time now is 11:22 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2022, vBulletin Solutions Inc.
Copyright 2003-2005 SageTV, LLC. All rights reserved.