Basic Linux V9 Build and Walkthrough - V2

[SageWizdom]

Seeing all the great work on Sage v9, I wanted to try and document a basic Linux install and setup. This is Version 2. It is still a work in progress (automation hopefully next). I will continue to make edits and updates to this post. Suggestions, fixes, ideas are welcome and appreciated. Thanks!

NOTE: This will install a command line only server (No GUI). This is based on Ubuntu Server 14.04.3 LTS - 64 bit. The SageTV GUI can be easily managed and configured via a client on another machine once this install is complete. I have attempted to share and document standard linux command line calls in this and the below walkthroughs. My test server boots in about 20 seconds and has very few extra processes running, dedicating as much ram/processor time to Sage and any transcoding tasks.

Note: A posting about basic linux commands is available here (here)

1) Backup your current install
Backup anything you want to save on your computer (insert link to the upgrade sage thread). This is typically your entire sagetv directory, your media directory and any custom configurations (ir remote install, primenet encoder, sagedct, etc).

2) Install Linux
See the post below for how (http://forums.sagetv.com/forums/show...92&postcount=2)


After installing Linux:
** the remainder of this walkthough is being replaced by an automated install script for both Sage and encoders available here (http://forums.sagetv.com/forums/showthread.php?p=585773)

To install Manually, continue with the below instructions:

3) Check for and install updates
Log in as the user you created during installation and with the configured password. Run the below commands.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo reboot

You will be asked to type in your password to validate your sudo privileges.

To explain the commands:
sudo or "superuser do" executes the following command as a super user. You must have sudo privileges to do this. by default the user created during install will have sudo privileges.

apt-get is the ubuntu package manager. This command allows you to install, manage or remove linux software packages in linux. In this command, we are using the install keyword to install several packages. apt-get will automatically identify and install any missing dependancies for the software you ask it to install. It will also validate with you, after it has figured out what it needs to do, but before it installs anything.

sudo apt-get update will update the internal database to find out what software is available and what needs to be updated.

sudo apt-get upgrade will update all of the software already on your computer / already installed

sudo apt-get dist-upgrade will install any needed updates / security patches that your computer needs.

Once complete, reboot the system to ensure all changes take effect.


4) Check your network settings
By default, the install will use DHCP. This should be fine for most/all dealings in the system. If you would like to configure a static ip, see the original walkthrough for details. I use a static IP for two reasons: A) it makes it easier to SSH into my system knowing where it is and B) this makes access to the mobile web interface easier. If your Sage box has a DNS name... none of this may matter (let me know in the comments if you all want me to put the static ip config back in this document)

To find your current IP address type:

>> ifconfig

eth0      Link encap:Ethernet  HWaddr 12:ad:6c:25:30:c1  
          inet addr:192.168.0.187  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:169 errors:0 dropped:18 overruns:0 frame:0
          TX packets:62 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:30311 (30.3 KB)  TX bytes:8517 (8.5 KB)

This will display all the network connections on your computer. Your connection should be labeled as eth0. eth for ethernet, 0 as this is your first ethernet card. If you have multiple ethernet connections you will have multiple entries. There will also be one labeled "lo" for loop back.


5) Install Java
Following directions from the Linux SageTV thread, install Java
http://forums.sagetv.com/forums/showthread.php?t=62324

Open a terminal
Select "Activities" -> "Show Applications" -> "Byobu Terminal"

sudo add-apt-repository ppa:webupd8team/java
sudo apt-get update
sudo apt-get install -y oracle-java8-installer

The first command adds the java repository to your computer so that apt-get knows how to get files from it.

Add Java Environment Variables
Installing Java does not appear to add the needed environment variables. This will add them to your ".bashrc" file. The ".bashrc" file is automatically run every time you create a new bash shell (it is your default login shell). If you use currently or choose to use a different logon shell, you should add these commands to a startup script for that shell.

echo "" >> .bashrc
echo "export JDK_HOME=/usr/lib/jvm/java-8-oracle/" >> .bashrc
echo "export JAVA_ARCH=amd64" >> .bashrc
echo "" >> .bashrc

6) Install Sage Dependancies
Install ubuntu SageTV Dependancies
(From the linux build thread http://forums.sagetv.com/forums/showthread.php?t=62324)

This is one very long line, make sure to copy the whole thing.

sudo apt-get install libx11-dev libxt-dev libraw1394-dev libavc1394-dev libiec61883-dev libfreetype6-dev yasm autoconf libtool build-essential libaudio-dev libpulse-dev libasound-dev madplay libasound2 xorg libfaad-dev

Reboot again to make sure all the libraries load appropriately

sudo reboot

7) Install Sage

This install uses the .deb package created by stuckless (awesome!)

Download the SageTV .deb package by running the below command.
http://forums.sagetv.com/forums/showthread.php?t=62422
NOTE: The below link is correct as of 23Feb2016. If a new version has come out, simply check the above thread for the latest download.

Download SageTV 9 JAR

# Download the SageTV .deb package
> wget https://bintray.com/artifact/download/opensagetv/sagetv/sagetv/9.0.4.232/sagetv-server_9.0.4_amd64.deb

# Install the package to /opt/sagetv/server/
> sudo dpkg -i sagetv-server_9.0.4_amd64.deb

Note1: It will ask for your sage license key as part of this. If you have it handy, put it in now to get access to the tv listing server. If you do not, skip this step and you can enter it later

Note2: The .deb currently launches the SageTV executable as part of the install... because this takes over the foreground terminal, you will need to use the ctrl-c (break) combination to stop it. We will reboot in a few minutes after which Sage will automatically launch in the background at boot time.

Note3: As new versions become available, you can re-download the .deb and re-install it to get the latest updates... Make sure to backup your database before hand (TODO: Document this / or better yet, automate this)

8) Use an existing SageTV Key for guide updates
If you have a valid Window or Linux SageTV license key, and did not enter it in the previous step.
Create a file named "activkey" in the sagetv folder (/opt/sagetv/server/) containing your key
(from the following thread http://forums.sagetv.com/forums/show...406#post577406)

echo "your key" > activkey

for example (but not a valid key) (32 characters letters and numbers)

echo "DF68I5XXXXXXXXXXXXXXXXXXXXXXXXXX" > activkey

9) Open network ports
By default the system does not have a firewall enabled. If you do nothing, it should stay this way. If you would like to enable a firewall...

To allow the client to talk to the server, you need to open ports on the server. Ubuntu is using the ufw to control the iptables firewall. Once complete, you will need to restart to ensure all changes have taken.

NOTE you MUST edit the first 3 octets below to match you home network subnet.

sudo ufw allow proto tcp from 192.168.0.0/24 to any port 22
sudo ufw allow proto tcp from 192.168.0.0/24 to any port 7760
sudo ufw allow proto tcp from 192.168.0.0/24 to any port 7818
sudo ufw allow proto tcp from 192.168.0.0/24 to any port 8018
sudo ufw allow proto tcp from 192.168.0.0/24 to any port 8080
sudo ufw allow proto udp from 192.168.0.0/24 to any port 8271
sudo ufw allow proto tcp from 192.168.0.0/24 to any port 31099
sudo ufw allow proto udp from 192.168.0.0/24 to any port 31100
sudo ufw allow proto tcp from 192.168.0.0/24 to any port 42024

sudo iptables -I INPUT -p udp -m udp --sport 65001 -j ACCEPT

sudo reboot

The last command should allow the "hdhomerun_config discover" command to work by accepting packets on the correct source port (thanks Boolah)

10) Configure Sage
Once your system has rebooted, point a Sage Client or Placeshifter at the server and you should be able to configure it.

Walkthroughs of Network Encoder Installs are below

HDHomeRun Prime with PrimeNetEncoder
http://forums.sagetv.com/forums/show...267#post581267

HDHomeRun Prime with OpenDCT
http://forums.sagetv.com/forums/show...21&postcount=1


Bonus) Adding Another HD
** Clean up the below when I'm actually awake**
Note the below is draft thoughts and comments
Given the low price of storage, here is how to add more.
A) Install another hard drive

B) Find it on your system.
- run "parted" to find your attached storage devices
-- Run Parted "sudo parted"
-- List attached devices "(parted) print devices"... find your device name based on size (ex. /dev/sdb)
-- Select the device you want to work with "select /dev/sdb"
-- Create a partition table "mklabel gpt" This specifically makes a guid based partition table (good for larger TB sized drives)
-- exit and save

use gdisk to create a partition
"sudo gdisk" ("sudo apt-get install gdisk" if needed)
create a new partition by typing "n"
if you want to use the whole drive, just use the defaults (keep hitting enter)
once back to the main command input, type "w" to write to disk (select Y if you are sure)

Format your partition
"sudo mkfs -t ext4 /dev/sdb1"
This makes a couple assumptions. This assumes you want an ext4 filesystem (vs fat32 or similar). This also assumes your drive is located at the above example. Replace this with the real path to your disk.

auto mount your drive
"sudo nano /etc/fstab"
create a new line at the bottom to tell you computer to auto mount the drive at boot. It should look like:
"/dev/sdb1 /mnt/storage ext4 errors=remount-ro 0 2"

create the mount point
"sudo mkdir /mnt/storage"

mount the drive
"sudo mount /dev/sdb1"

You should now be able to access the drive "cd /mnt/storage"
next add directories and configure sage to use this storage space.

[rmac321]

Just wanted to post a quick thanks for taking the time to work on the installation guide.

I had played with installing the open source a month or two ago, but never got it working.

A couple notes.
I initially tried installing on a Debian box I use for tinkering. If I recall correctly, I tried on both Wheezy and Jessie. There is apparently just enough of a difference in Debian and Ubuntu that the .deb file installation won't work without additional tinkering and I never had time to figure out what the problem was.
Made some room for an additional os, added Ubuntu and ran through your install steps and was up and running.

Looking forward to getting it set up to take over primary duty from my aging XP SageServer.

Thanks Again

[rmac321]

Looking through the threads, I've seen this question asked, but don't recall seeing an answer. If I missed a thread, feel free to point me in the right direction.

With the Windows version of SageTV we were encouraged to have several different recording drives to spread the load when recording more than one program at the same time. Is this still the case with the open source Linux version?

It seems to me that if that is the case, a good solution would be to set up a separate volume group just for recording drives and then make each drive a distinct logical volume within the volume group. Would be easy to swap out a bad drive or add drives and separate the load.

[SageWizdom]

Originally Posted by rmac321

With the Windows version of SageTV we were encouraged to have several different recording drives to spread the load when recording more than one program at the same time. Is this still the case with the open source Linux version?

It seems to me that if that is the case, a good solution would be to set up a separate volume group just for recording drives and then make each drive a distinct logical volume within the volume group. Would be easy to swap out a bad drive or add drives and separate the load.

Whew! Great question, I am decidedly the wrong guy to ask that question to. I just let everything write to the big drive I threw in the server. I will eagerly watch to see if someone else has a good answer.

[SageWizdom]

** Update: I uploaded an updated script as the first version had hard coded values for the firewall local network. See below for instructions how to remove these values.

** Update 2: Updated script again. I'm going to work on getting it into the code base per a great suggestion and then it will be just two commands, one to download and one to run.

** Update 3: I've copied this to a new thread, to put all development in a standalone area. http://forums.sagetv.com/forums/showthread.php?p=585773

First try and an automated install script

-- Only tested with Ubuntu 14.04.3 LTS - 64bit Server Edition
-- Attempts to do some validation / testing, but no promises (aka don't use this on a production machine)

This will attempt to probe for, then download and install the latest SageTV .deb file.

This will attempt to prove for, then download and install the latest opendct .deb file.

This will attempt to set up your firewall (but will not turn it on).

In theory, at the conclusion of this run you should be able to open a Sage client, connect and view video remotely

*NOTE: This does not install the local sage client, this assumes a headless computer.

How to use this
1) Install Ubuntu Linux - the 64 bit SERVER version of 14.04.3 LTS as per http://forums.sagetv.com/forums/show...92&postcount=2

2) Copy this file to your ubuntu box. The easiest way is probably to use SFTP. Ubuntu as installed above should already have SFTP installed on it. If you are a windows user, you can use PSFTP as an application on windows to send the file. The SFTP exchange will likely look like:

sftp <new sage user>@<new sage ip>
put installsage.txt
quit

3) Log into your remote Linux box. The easiest way should be SSH. If you are using windows, PuTTY is a good SSH tool.

ssh <new sage user>@<new sage ip>

4) At the command prompt, rename the script to installsage.sh

mv installsage.txt installsage.sh

5) Make the file executable

chmod a+x installsage.sh

6) Run the script with admin privalidges

sudo ./installsage.sh

It should now run, attempting to install everything and telling you as it goes. You will be prompted several times relating to Java and the install of SageTV (to enter your license key).

Known Issues:

#1 When installing, for some reason, after sage installs, it does not return, even though it should. To handle this:
- Type Ctrl-Z to have the current application Sleep
- Type the command "sudo service sagetv stop"
- When this is done type the command "fg" to return the sleeping processes to the foreground.
#2 the running of console only does not work for some reason right now in the script. you have to stop the opendct service, run this once and then start the service... I'll have to look tomorrow again at it.


I've attempted to have it auto install as much as is reasonable. Please holler with any errors.

Thanks

[ranger]

I built an Ubuntu 14.04 VM from scratch. No updates, no customizations except for install openssh-server. I ran your script as is and it installed all required software and then installed SageTV. The only errors I saw were a few associated with the apt-get upgrade which I have seen before, so probably nothing to do with your script.

Sage runs and I connected with a client. I'll try a few recordings, throw on some plugins for further testing, but so far so good!

One other comment in regards to firewall rules. On my own installs on both Ubuntu and Centos, I have had to whitelist the IP Addresses for my HDHomerun tuners. If not they fail to work after reboot. I'll try it out with your install just to see if you got something I missed...

[SageWizdom]

Originally Posted by ranger

I built an Ubuntu 14.04 VM from scratch. No updates, no customizations except for install openssh-server. I ran your script as is and it installed all required software and then installed SageTV. The only errors I saw were a few associated with the apt-get upgrade which I have seen before, so probably nothing to do with your script.

Sage runs and I connected with a client. I'll try a few recordings, throw on some plugins for further testing, but so far so good!

One other comment in regards to firewall rules. On my own installs on both Ubuntu and Centos, I have had to whitelist the IP Addresses for my HDHomerun tuners. If not they fail to work after reboot. I'll try it out with your install just to see if you got something I missed...

Just for completeness I'd love to here what the errors were in case its something we need to track. I know there are some outputs related to when it adds the java ppt archive... you will get notices talking about adding some keys.

Technically, I do not turn on the firewall in the script. If you do and run a status, I believe it will look something like...

$ sudo ufw status
Status: active

To Action From
-- ------ ----
22/tcp ALLOW 192.168.0.0/24
7760/tcp ALLOW 192.168.0.0/24
7818/tcp ALLOW 192.168.0.0/24
8018/tcp ALLOW 192.168.0.0/24
8080/tcp ALLOW 192.168.0.0/24
8271/udp ALLOW 192.168.0.0/24
31099/tcp ALLOW 192.168.0.0/24
31100/udp ALLOW 192.168.0.0/24
42024/tcp ALLOW 192.168.0.0/24
9000:9100/tcp ALLOW Anywhere
8300:8500/udp ALLOW Anywhere
7818/tcp ALLOW Anywhere
8271/udp ALLOW Anywhere
9000:9100/tcp (v6) ALLOW Anywhere (v6)
8300:8500/udp (v6) ALLOW Anywhere (v6)
7818/tcp (v6) ALLOW Anywhere (v6)
8271/udp (v6) ALLOW Anywhere (v6)

Which reminds me right off the bat that the script is hard coded to use the wrong IP's as noted above 192.168.0.0 which will most certainly be wrong for your network. I'll fix and repost in a minute. To delete these rules, the easiest way is to..

1) run the command "sudo ufw status numbered" to provide a listed number next to each command.

2) run the command "sudo ufw delete <rule number>" which will prompt you for confirmation and allow you to delete each rule one at a time. Don't forget, if you are deleting rule 1, rule 2 becomes the new rule 1.

3) if you want to re-add the commands... use the command in the script, without the "from 192.168.0.0/24" section.

Updated post in a few minutes

[ranger]

It appears that OpenDCT did not install from the script. Not sure why, but the service did not exist. I had walked away for a while when the script was running and it looks like the script defaulted to no OpenDCT install or may have failed the install.

rich@ubu-VirtualBox:~$ service opendct status
opendct: unrecognized service

I went ahead and removed the ufw rules with the 192 address and re-added them without it. Once the firewall was activated, I lost connection to the HDHomeruns. Taking a look through your script I confirmed my suspicions. Your rules on the HDHomerun are specific to OpenDCT. The raw HDHomerun dynamically assigns a port and it changes after a reboot, so whitelisting their IP Addresses seems to be the best solution in that case.

It won't be until later in the week, but I'll try and run the script from scratch again and collect the console output for review.

[ranger]

OK, I know why the script never installed OpenDCT the first time I ran it. The script hung at "Main is starting" after SageTV was installed and I came back after a while and remotely re-booted the server. (Sorry I forgot that from the first install).
This time pressed ctrl-c and after an error, the script continued.

Executing pre-scripts
Starting server mode
Main is starting



^Cdpkg: error processing package sagetv-server (--install):
 subprocess installed post-installation script was interrupted
Processing triggers for ureadahead (0.100.0-16) ...
ureadahead will be reprofiled on next reboot
Errors were encountered while processing:
 sagetv-server

Press any key to continue, install will auto resume in 5 seconds
========= Add Firewall Entries ========

Running and testing the install now. Not having used OpenDCT before, I'm not sure if SageTV is using the OpenDCT tuners or the native HDHomerun tuners.

[EnterNoEscape]

Originally Posted by ranger

Running and testing the install now. Not having used OpenDCT before, I'm not sure if SageTV is using the OpenDCT tuners or the native HDHomerun tuners.

If you don't see an IP address next to the capture device in its name, you're using the built in support.

[SageWizdom]

As an update to the above thread, there is now an Alpha automated install for both SageTV and OpenDCT. As noted above, it starts after item thre


** See automated install script for Sage and OpenDCT here http://forums.sagetv.com/forums/showthread.php?p=585773)

[wayner]

I am thinking of diving into installing a Linux server - is this walkthrough still current or is there a more recent update.

[SageWizdom]

This walkthrough is a lot less current. I ended up automating it for ubuntu 14.04 LTS. I also posted a script for ubuntu 16.04 LTS. The 16.04 does not yet fully install the openDCT support (the old one worked out of the box) but it does get sage otherwise up and running. These are certainly good places to start, and I'm around to try and help. also feel free to pm me if you get stuck.

[SageWizdom]

Link to post with latest script. The linux walk through should still be good and I believe works for 16.04 as well as it did for 14.04. I'll take a look and verify.

https://forums.sagetv.com/forums/sho...4&postcount=84

[wayner]

I will likely be using unRAID/Docker - is this applicable for that?

[Fuzzy]

Originally Posted by wayner

I will likely be using unRAID/Docker - is this applicable for that?

Not really. You almost don't need any instructions for installing the Docker if you've got the Community Applications plugin installed on your unRAID machine. Just search SageTV, and click it.

[jgs14]

Hello,

This walkthrough lists that we need to open source-port 65001/udp in a linux install to get HDHomeRun detection working.

On my recent install, I needed to also open source-port 5004/udp in order to get video/tuning working. Without this open I was unable to do a channel scan or view video.

I suggest we add this to the walkthrough, as it took me quite a while to figure out and should be of use to others.

Regards,

Joe

[larryf]

Is anyone using Ubuntu Server with Sage in a VM? I'm getting an error message, "EXT4-fs (sda2) unable to read Superblock" in the console.

Strangely enough, everything appears to be working, but I see the error 5 or 6 times in the console at every boot.

Tried installing many times, used the script in another post, tried manually, tried recreating the VM... The message only comes up after installing Sage, soon after I enter the key (even tried not entering the key since I don't really need it).

VMware 6.5 Free, assigning 4G memory and 100G disk space. Using the latest Ubuntu ISO LTS from this post.

Thanks,
Larry

[ASegredo]

These efforts are greatly appreciated but I have one bone to pick. When you say "Linux" you mean Ubuntu. For instance the widely used RedHat Fedora distribution uses rpm instead of aptget and Ubuntu is rather unique in not having a root login which is where most distros do admin work.

The title should be Basic Ubuntu V8 Build and Walkthrough, because everything is specific to Ubuntu.

Sorry to be pedantic, but you get my hopes up that V9 builds on anything other than Ubuntu.

I am a retired software developer and I understand and appreciate the effort that has gone into this project.

[stuckless]

Originally Posted by ASegredo

These efforts are greatly appreciated but I have one bone to pick. When you say "Linux" you mean Ubuntu. For instance the widely used RedHat Fedora distribution uses rpm instead of aptget and Ubuntu is rather unique in not having a root login which is where most distros do admin work.

The title should be Basic Ubuntu V8 Build and Walkthrough, because everything is specific to Ubuntu.

Sorry to be pedantic, but you get my hopes up that V9 builds on anything other than Ubuntu.

I am a retired software developer and I understand and appreciate the effort that has gone into this project.

I get the point, but, I suspect that while the steps are Ubuntu ( or even Debian) specific, I'd like to think that anyone that's been around linux for more than a few months could convert the instructions, mentally, to their distro. I run Ubuntu and Fedora workstations, and I've installed sagetv on both of those. The .tgz downloads can pretty much run on any 64 bit linux, the real challenge is just getting the prereqs, such as java, installed.