Virtualize with ESXi 4.0

[harrijay]

I have upgraded my setup now which you can see what I have below . I am considering switching my setup to ESXi since I am familiar with it at my work and I want to be able to run other test OS's on the occasion. I also have plenty of horsepower that is not being used. I have done a test run and I can pass my A180 and a NIC directly to a VM. I think this will give me the best chances of working and the least amount of hiccups. I want to also connect my 1.5TB drive directly to the VM rather than creating a virtual disk file. Does anyone have any experience with passing hardware like this to a VM? Anything I should avoid before making the jump?

TIA,
Jayson

[kbyrd]

There is at least one other thread about doing this, although it was in the context of using ESX to get Solaris to manage the storage, the poster wanted ZFS.

As long as the tuner is a network or USB tuner, this should work just fine. I'm very familiar with VMware as well and while I haven't done this yet, I'm thinking about it.

[aflat]

I've had issues directly attaching storage, but that was a couple of years ago, they hopefully have worked out the kinks now. Mounting a virtual disk as a physical drive still has issues, but they are getting less and less every revision as well.

[jpappas]

Originally Posted by harrijay

I have upgraded my setup now which you can see what I have below . I am considering switching my setup to ESXi since I am familiar with it at my work and I want to be able to run other test OS's on the occasion.

ESXi provides less control from the CLI than ESX, at the gain of being runnable from a USB flash drive or the like (Lite-footprint) and being "free" of course.

Originally Posted by harrijay

I also have plenty of horsepower that is not being used. I have done a test run and I can pass my A180 and a NIC directly to a VM.

I will have to re-visit PCI pass-thru, AFAIK ESX3.5 could not pass a PCI device thru (maybe experimentally, and USB was hit/miss). If that works, I may look at this too. Just have to check which white-box motherboards work the best with ESXi4.

Originally Posted by harrijay

I want to also connect my 1.5TB drive directly to the VM rather than creating a virtual disk file. Does anyone have any experience with passing hardware like this to a VM?

RDM (Raw Disk Mapping) will pass Raw Disks to VMs.

IN such a config, you would have a VMFS partition (presumably on the smaller drive) with the Sage OS VMDK, and then a second virtual drive would be added that would be RDM (virtual if you want snapshots and the like, physical if you want SCSI commands to passed directly to the drive; virtual is best-practice if not using MSCS or other clustering)

Originally Posted by harrijay

Anything I should avoid before making the jump?

I can definitely see the advantages of this config. I am thinking the only hangups would be hardware compatibility and device pass-thru. If pass-thru works without too much overhead; the I/O is fast enough for HD capture requirements; and the target system is HCL "close" then it would be a reasonably clean config.

Keep us posted!
John

[kbyrd]

The RDM path is well tested, it works fine in the field. The common problem people run into running ESX at home is making sure your storage is recognized. Historically, ESX hasn't had great support for SATA or SAS on the motherboard. Recently that's gotten much better. If you can make it a datastore, then it'll work.


The PCI passthru will only work with motherboards which have an IOMMU. If you have an network attached encoder (like the A180 or the HDHR), this isn't a problem.

[mikejaner]

I have been testing this extensively (raw disk mapping) over the last few months. It works fine for passing the disks through to every os I threw at it (Ubuntu, Windows 7, 2008, XP, 2003) with one exception. You cannot successfully create a software based RAID array with Windows 7 or 2008 R2. There seems to be a compatibility issue with ESX and how it lets the guest os write to "disks". Note this is also true if you are mapping virtual disks, not just raw disks, to these OS's. RAID works fine in Ubuntu and 2008 (R1) and below.

Here's a link to a quick and dirty tutorial for mapping raw LUN's:
http://www.vm-help.com/forum/viewtopic.php?f=14&t=1025

BTW: I have also tested SageTV by just using standard virtual drives, and recording from a HDHR over the network. It works very well, with no glitches so far (two weeks). I do not have VT-H capability so all of my testing can only happen with my HDHR's and Hava HD Wireless.

[DigitalMan]

If you created your RDM volumes using that guide, perhaps you should try using -z instead of -r to create physical compatibility mode mappings rather than virtual mode.

I have an ESXi 4 setup and its running perfectly. I'm using virtual NIC's and performance has been spectacular, so I see no need to use passthru on them. I do have my arrays set up using RDM (virtual mode). One of these days I would love to see how many tuners and clients it would be able to handle. Tuners and clients are on separate physical NICs and the Sage VM has 3 virtual NICs in it.

Originally Posted by mikejaner

I have been testing this extensively (raw disk mapping) over the last few months. It works fine for passing the disks through to every os I threw at it (Ubuntu, Windows 7, 2008, XP, 2003) with one exception. You cannot successfully create a software based RAID array with Windows 7 or 2008 R2. There seems to be a compatibility issue with ESX and how it lets the guest os write to "disks". Note this is also true if you are mapping virtual disks, not just raw disks, to these OS's. RAID works fine in Ubuntu and 2008 (R1) and below.

Here's a link to a quick and dirty tutorial for mapping raw LUN's:
http://www.vm-help.com/forum/viewtopic.php?f=14&t=1025

BTW: I have also tested SageTV by just using standard virtual drives, and recording from a HDHR over the network. It works very well, with no glitches so far (two weeks). I do not have VT-H capability so all of my testing can only happen with my HDHR's and Hava HD Wireless.

[mikejaner]

Thanks for the tip, I will give it a try. Didn't know about -z. Do you think it will make a difference if VMDKs will not do RAID either in 2008 R2 or Win7?

[Update: That worked, thanks again!! I'm gonna test performance now]

[stanger89]

OK guys, I put ESXI 4 u1 on a spare PC so I could play with it, but I've run into a snag that has me utterly baffled.

I'm attempting to install Linux Mint 8 (KDE) in a VM. I downloaded the ISO, uploaded it to an ESXi datastore, setup a VM (picked ubuntu 64 as the linux version).

The LiveCD boots just fine into KDE and it all appears to be running fine. Problem is, whenever I try to "install" Mint, my ESXi machine (the whole machine) just powers off. Somewhere between it creating the file system and copying the files.

Like I said I'm baffled, I haven't run it a lot, but I've put a few hours on another VM with Windows Server 2003 (enough to install Sage and a SP2). Anyone have any ideas what would cause the whole system to pack up?

[mikejaner]

Originally Posted by stanger89

OK guys, I put ESXI 4 u1 on a spare PC so I could play with it, but I've run into a snag that has me utterly baffled.

I'm attempting to install Linux Mint 8 (KDE) in a VM. I downloaded the ISO, uploaded it to an ESXi datastore, setup a VM (picked ubuntu 64 as the linux version).

The LiveCD boots just fine into KDE and it all appears to be running fine. Problem is, whenever I try to "install" Mint, my ESXi machine (the whole machine) just powers off. Somewhere between it creating the file system and copying the files.

Like I said I'm baffled, I haven't run it a lot, but I've put a few hours on another VM with Windows Server 2003 (enough to install Sage and a SP2). Anyone have any ideas what would cause the whole system to pack up?

I am installing Mint Gnome 64 right now, and it's copying files as I type. I wonder if you got a bad download? I can try KDE later if you like.

[kbyrd]

Originally Posted by stanger89

The LiveCD boots just fine into KDE and it all appears to be running fine. Problem is, whenever I try to "install" Mint, my ESXi machine (the whole machine) just powers off. Somewhere between it creating the file system and copying the files.

Like I said I'm baffled, I haven't run it a lot, but I've put a few hours on another VM with Windows Server 2003 (enough to install Sage and a SP2). Anyone have any ideas what would cause the whole system to pack up?

Uhh, wow crashing the host is pretty bad. The way you phrased it, it sounds like you can consistently reproduce this. If that's the case, can you do it again and grab the vmware.log file from the VM directory on the ESXi host?

[mikejaner]

How much ram did you assign to the VM?
I just went through the install of Mint 64 KDE, but only after increasing the ram from 512MB to 2048MB. At 512 MB, the VM kept crashing the installation program, and it was very slow. I bumped up the memory, and now the VM is installing smoothly.

Originally Posted by stanger89

OK guys, I put ESXI 4 u1 on a spare PC so I could play with it, but I've run into a snag that has me utterly baffled.

I'm attempting to install Linux Mint 8 (KDE) in a VM. I downloaded the ISO, uploaded it to an ESXi datastore, setup a VM (picked ubuntu 64 as the linux version).

The LiveCD boots just fine into KDE and it all appears to be running fine. Problem is, whenever I try to "install" Mint, my ESXi machine (the whole machine) just powers off. Somewhere between it creating the file system and copying the files.

Like I said I'm baffled, I haven't run it a lot, but I've put a few hours on another VM with Windows Server 2003 (enough to install Sage and a SP2). Anyone have any ideas what would cause the whole system to pack up?

[stanger89]

Originally Posted by mikejaner

I am installing Mint Gnome 64 right now, and it's copying files as I type. I wonder if you got a bad download? I can try KDE later if you like.

If it is a bad download, it's both my 32 and 64bit downloads.

Originally Posted by kbyrd

Uhh, wow crashing the host is pretty bad. The way you phrased it, it sounds like you can consistently reproduce this. If that's the case, can you do it again and grab the vmware.log file from the VM directory on the ESXi host?

I could do that again, I did it once but didn't see anything that stuck out at me.

Originally Posted by mikejaner

How much ram did you assign to the VM?
I just went through the install of Mint 64 KDE, but only after increasing the ram from 512MB to 2048MB. At 512 MB, the VM kept crashing the installation program, and it was very slow. I bumped up the memory, and now the VM is installing smoothly.

I gave it 1GB.

[aflat]

Does your processor have virtualization enabled in the bios?

[stanger89]

I'm pretty sure I did. Like I said, it's weird, it will run Mint (off the live CD) just fine, and it runs Windows Server 2003 and XP just fine, but for some reason the Mint installer kills it dead every time.

[aflat]

I just ran into an ESXi issue now where I installed it, but I forgot to turn on virtualization. I flipped it after I installed ESXi, but I couldn't use VMWare converter to put a 64bit OS into the ESXi server, it kept saying it doesn't support 64 bit. I had to reinstall to get it to support 64bit. Granted it was a new server, so it was easy to reinstall, there's probably some console setting to do it as well.

[kbyrd]

If you're running 64-bit guests, enable VT or whatever AMD calls it in the BIOS first.

[harrijay]

I got everything switched over and it is running pretty good. I ran into a couple problems along the way but mainly due to the way ESXi works. I decided to pass my NIC directly to the VM since the motherboard I am running has 2 NIC's on it but ESXi would only recognize one of them. I passed the disks in physical mode and was able to mount them back into ubuntu without any issue or data loss. I am able to access my PCI HD tuner (A180) without any issue and I do not see any kind of performance problems yet. as of now so far so good but I will let you know if anything changes. Thanks for everyones input.

Jayson

[kbyrd]

Are you RAIDing the disks? If so, are you doing this in the Ubuntu guest or in hardware or something else?

[stanger89]

I've got a question... I see the new i5's have VT-d, but I'm having a heck of a time finding a motherboard that supports it to go with it. Does anyone know if any of the intel H55 or H57 board support VT-d? Or any i5 boards with confirmed VT-d?

-er scratch that maybe. Looks like the DQ57TM officially includes VT-d.