XenServer hypervisor build walkthrough

[SageWizdom]

I'm going to post this in case it is useful to anyone else. In order to easily manage and support several projects, I wanted to virtualize my home hosting. Looking at how little cpu my SageTV server uses, I'm going to also be running a virtualized Sage box as part of this... a grand experiment.

When choosing a virtualization platform, there are lots of options. This walkthrough specifically details how to install the XenServer hypervisor. XenServer is the OpenSource Citrix version of the Xen Project hypervisor. There is no cost for personal use.

I'm throwing this on a server with an 8 core i7 processor, 8 GB ram and at least 1 TB of storage. I will likely bump up storage and possibly the ram if this works going forward.

1) Download the XenServer Installer
Go to link

http://xenserver.org/overview-xenserver-open-source-virtualization/download.html

Download the XenServer v6.5 Installer ISO and burn it to disk.

2) Install XenServer

• Check your bios and ensure that VT (virtualization) extensions are turned on
• Insert and boot from the XenServer install disk
• press enter to boot from disk (or do nothing and it will auto boot)
• Select your keyboard
• Click OK to continue (this will erase all data on computer)
• Accept EULA
• If VT is not turned on your your processor doesn't support virtualization you will now get an error warning of limited functionality.
  • This is a good time to back to your bios and enable VT support if you have it. If your processor does not, you will be limited in what you can actually do.
• Select hard drive (Storage Device ex sda) to install to. This will be completely overwritten!
• Select source media (I used local)
• Do you want to install any supplemental packs (we will do this later)? I said no
• Would you like to test your media… You should do this.
• When complete, a dialog will be displayed saying no errors were found.
  • If you are told there are errors, re download the image and start over
• Enter a root password for the XenServer
• Set up networking (I use/recommend a static ip)
• Set up DNS and hostname (I don't have DNS pointing at this machine, but I gave it a simple hostname anyway)
• Select your time zone
• Use NTP? This is recommended
• Click Install
• When install is complete, the disk will be ejected. remove it and press enter to reboot

3) Configure XenServer
There are several ways to configure your server. If you have a windows laptop/computer, you can install XenCenter to directly control and manage your system. As I do not have a windows machine, an alternative was needed.

I am using Xen Orchestra (https://xen-orchestra.com/). They offer prebuilt VM that you can use for free and easily install directly install into your system. I will try to add a how to at a later date of how to build from source, which gives you additional features.

Details here: link

https://github.com/vatesfr/xo/blob/master/doc/installation/xoa_installation.md

• Go to the XenOrchestra web site and register
• Once you are logged in, go to the download section and download the (.xva to your computer)
• Copy this file to your virtual host
  • From the directory where you downloaded the image
  • On Linux or a Mac, open a terminal and "sftp root@<your virtual host ip>"
  • On Windows you will need an sftp client.
  • Once you log in to your remote server, upload the file with the command "put xoa_free_4.12.xva"
  • To exit, type "quit"
• SSH into your virtual host
  • On Linux or a Mac, open a terminal and "ssh root@<your virtual host ip>"
  • On Windows you will need an ssh client. One option is putty (http://www.putty.org/)
• Add the VM
  • xe vm-import filename=xoa_free_4.12.xva
• Start the VM
  • Get a list of running VMs with the command "xe vm-list"
  • Find the uuid of the vm labeled as "XOA 4.12 Free"
  • Start the VM with the command "xe vm-start uuid=<vm uuid>"
• Get the IP of your VM with the command
  

  xe vm-param-get uuid=<uuid of your vm> param-name=networks

• Connect to the server with a web browser
• Default username is "admin@admin.net" with the password "admin".

There are more steps to creating a vm, I can detail this if anyone is having trouble.

[SageWizdom]

Placeholder for future adds

[Greg2dot0]

I got this set up over the weekend. Thanks for the idea...seemed silly migrating my Sage install to a dedicated server only to have it utilize about 5% of the resources. Curious, what configs are you running your Sage VM with?

I've got 20gb partition, 4gb RAM, and 4 vCPU (2 sockets, 2 cores). Host is an i5 processor at 31 or something like that.

I am noticing that things like comskip seem to run much slower and sort of expect this as part of the trade-off for virtualizing the system.

[SageWizdom]

Originally Posted by Greg2dot0

I got this set up over the weekend. Thanks for the idea...seemed silly migrating my Sage install to a dedicated server only to have it utilize about 5% of the resources. Curious, what configs are you running your Sage VM with?

I've got 20gb partition, 4gb RAM, and 4 vCPU (2 sockets, 2 cores). Host is an i5 processor at 31 or something like that.

I am noticing that things like comskip seem to run much slower and sort of expect this as part of the trade-off for virtualizing the system.

I've been totally low-balling the system for now (I really need to upgrade my ram). It currently has 2GB or ram and 2 CPU cores on an i7.... I need to upgrade the machine ram, and have been trying to get a feel for how Xen does its processor sharing (it does not appear to allow overcommiting of resources at the level I would appreciate (vmware does some crazy stuff). I've been looking at trying the container version, but have not gotten to that yet. I am not re-installed comskip, so I dont know how much that would affect it.

[Greg2dot0]

Originally Posted by SageWizdom

I am not re-installed comskip, so I dont know how much that would affect it.

I actually have the whole think running pretty well w/2gb and 2 cores including COMSKIP on Unbuntu 64 server 16.04. I am continuing to monitor (I'm running Windows as one of my desktops, so have XenCenter loaded) the overall performance. I have only 16gb in the whole system (using 6 on 3 different machines). I am tending to start low and increase resources as they become constrained. So far, 2gb seems to be the sweet spot.

[SageWizdom]

Awesome. I realize I should have said, I have not yet installed comskip. So much for typing at night.

[Greg2dot0]

No problem. I actually have increased my ram to 3GB because of some pauses when loading (and possibly related to some video glitches). I'm looking at the logs and it seems that Java does the memory management and eats up whatever you allocate. Tough to nail down the right number.

[hemicuda]

How would this option compare to Oracle VM VirtualBox? I've had good luck with it over the years. One caveat though is that I haven't tried running my Sage server inside it.

[Greg2dot0]

I think it would be comparable as virtual technologies have more or less matured. The one thing is that VirtualBox is more of a client virtualization technology while XenServer is more server based.

[KarylFStein]

The problem I had with VirtualBox was it didn't stop/start VMs on shutdown/boot of the host. There is a third party application that provided this and worked well for me, but if you ever want to open the VirtualBox application you have to disable the other program, (which halts the VMs).

I'm using Hyper-V now and like it. It let's you do things like dynamic memory and CPU, (no guessing on sizing although you can put limits on how many resources a guest can claim), and has shutdown/startup on host boot functionality built-in. BUT it doesn't let you pass a PCIe card to the guest unless you're running a Windows Server OS.

Anyway, they're both a different beast than XenServer.

[SageWizdom]

Originally Posted by KarylFStein

The problem I had with VirtualBox was it didn't stop/start VMs on shutdown/boot of the host. There is a third party application that provided this and worked well for me, but if you ever want to open the VirtualBox application you have to disable the other program, (which halts the VMs).

I'm using Hyper-V now and like it. It let's you do things like dynamic memory and CPU, (no guessing on sizing although you can put limits on how many resources a guest can claim), and has shutdown/startup on host boot functionality built-in. BUT it doesn't let you pass a PCIe card to the guest unless you're running a Windows Server OS.

Anyway, they're both a different beast than XenServer.

HyperV should totally work, I was mostly trying to avoid a standalone server for Sage when I have a bunch of others all virtualized. I would expect the right answer is always "whatever you are most comfortable with." That said, I'm thinking I need to go play with the docker image soon... it seems to do much better memory sharing (as you noted above) than xen server.

[Greg2dot0]

Just wanted to do an update here that I ditched Xen in favor of Hyper-v. I happen to have a technet license available for Windows Server 2012 R2 so figured I'd fire it up. While it took me a lot longer to get this configured than Xen (due to driver signing issues), I feel that this hypervisor is much more mature.. Dynamic memory allocation is awesome.

While Performance Monitor captures a ton of metrics, I do feel like reporting is one area where Xen has Hyper-v beat.

[SageWizdom]

The dynamic memory allocation thing in Xen is very frustrating. Apparently it can do some amount of dynamic memory allocation, but it involves several command line options and non obvious configuration items and is not automatic.

Was there anything special you had to do to get it to work? It would be worth capturing for anyone else using Hyper-V

[Olias]

I am going to convert my SageTV server into a virtual machine soon and am looking a various hypervisors. My main requirement is USB passthrough to access two HD-PVRs.
That seems to rule out Hyper-v. I know that VMWare allows for USB passthrough, does anyone know if Xen has this feature?

[BobPhoenix]

Originally Posted by Olias

I am going to convert my SageTV server into a virtual machine soon and am looking a various hypervisors. My main requirement is USB passthrough to access two HD-PVRs.
That seems to rule out Hyper-v. I know that VMWare allows for USB passthrough, does anyone know if Xen has this feature?

When unRAID had Xen as it's hypervisor I used it with pass through for one of my SageTV servers. Now that unRAID is on KVM I'm using that for one of my servers. After this weekend it will be two of them.

[Greg2dot0]

Originally Posted by SageWizdom

Was there anything special you had to do to get it to work? It would be worth capturing for anyone else using Hyper-V

Just check a box...that easy.

[SageWizdom]

Originally Posted by Olias

I am going to convert my SageTV server into a virtual machine soon and am looking a various hypervisors. My main requirement is USB passthrough to access two HD-PVRs.
That seems to rule out Hyper-v. I know that VMWare allows for USB passthrough, does anyone know if Xen has this feature?

Perhaps I'm doing it wrong, but Xen appears to only run headless. AKA, I can not really run a gui on the local box. I have enough linux experience that this didn't kill me, but did take a while to get it to the point that it all was working easy and convenient.

As some folks here have noted... I'm strongly thinking about a next version where I get a synology NAS and use the docker container. Not sure if others have thoughts on this or not.

[stuckless]

Originally Posted by SageWizdom

Perhaps I'm doing it wrong, but Xen appears to only run headless. AKA, I can not really run a gui on the local box. I have enough linux experience that this didn't kill me, but did take a while to get it to the point that it all was working easy and convenient.

As some folks here have noted... I'm strongly thinking about a next version where I get a synology NAS and use the docker container. Not sure if others have thoughts on this or not.

If you are looking at a NAS... I'd strongly urge you to look at unRAID. You can setup a 30 day trial version and play with it. The docker image will run on synology, I'm pretty sure, but on unRAID... it just works and can be installed with a couple clicks using the "Community Applications" plugin. Depending on the type of synology you buy... not sure it's cheaper than building an unRAID server, either. Something to think about

[SageWizdom]

I have decidedly been following the threads on unraid. I am feeling a little lazy as of late and was at least considering (assuming $$) that a solution where I could just "buy it" might be nice. Between this, some of the automation, some of the auto plugin install, I'm trying to get to a point where I can just spin up a sage backend for myself (or someone else) very easily, access it via web configuration and then front end it with sage or other.