Monthly Archives: July 2013

XenServer 6.1 Live Migration via CLI

You’ll need to know the SR UUID for local storage that you’re migrating to, You can find this by running:
xe sr-list host={name of the host platform} name-label=Local\ storage params=uuid --minimal
(this is expecting a default of SR named Local storage).

You’ll also need to know the vm uuid which can be found running:
xe vm-list name-label={name of vm} params=uuid --minimal

Then you can run:
xe vm-migrate --live vm={vm uuid or name} remote-master={pool master name} remote-address={platform in pool you want to migrate to, name or uuid} remote-username=root remote-network={name of interface you want to use to transfer, I used xenbr1 because we didn't want to migrate over the management network} remote-password={root pw} destination-sr-uuid={SR uuid on platform you're migrating to}

XenServer Windows 2008 Phantom hard drive usage

We got an alert that smtp went down on a customers Windows Server 2008 R2 VM. After logging in it looked like the hard drive was full. We went ahead and loaded windirstat (http://windirstat.info/) to try to track down where the usage was. Unfortunately it didn’t register. We were able to track it down however to an older version of SmarterMail (http://www.smartertools.com/) that had a 90GB clam/tmp folder with all of it’s virus definitions. There was literally hundreds of folders that administrator couldn’t delete. After futsing a bit with it I could take ownership and then give myself full privileges and delete them, but there were literally hundreds of these things. Over night a co-worker came up with the following command to run in command prompt:
takeown /f “c:\Program Files (x86)\SmarterTools\SmarterMail\Service\Clam\tmp\*” /R

Then you can delete them at your leisure and free up all that space. It’s likely because the server was running an ancient version of SmarterMail.

Clearing Postfix queue after a compromised account

Recently we had someone upload a malicious script onto a customer wordpress site. After tracking it down and removing it, there was still a ton of random username@customer_domain.tld mail kicking around in the queue. To remedy I stumbled upon a command that would clear out all of the email associated to that user. In my example I will use customer_domain.tld, but you’d use whatever the fqdn is.
mailq | grep customer_domain.tld | awk ‘{print $1}’ | sed -e ‘s/\*//g’ | postsuper -d –

Likely there will be a lot of returned mail as well, so to clean that out you can use:
mailq | grep MAILER-DAEMON | awk ‘{print $1}’ | sed -e ‘s/\*//g’ | postsuper -d –

The spamming had gone on for quite a while unnoticed by the customer, and it ended up running all night and removing over 1.5 million articles of spam.

*I piped it through sed to remove the asterisk because some of the mailq id’s had an asterisk in them, but if you tried to pipe that to postsuper -d it came back with a ID not found.

Upgrading XenServer 5.6sp2 to Xenserver 6.1 with a 3ware Raid Card

Originally trying to upgrade our platform servers running XenServer 5.6SP2 on a LSI 9750 3ware Raid card wasn’t working correctly, as they weren’t seeing the old data to upgrade. So what I ended up doing was loading the raid drivers before the installer ever started. Here is a brief checklist of the steps I took to successfully upgrade the servers:

Required Items:
USB CD Rom
XenServer 6.1 9750 Driver Disk (created using the DDK)
XenServer 6.1 Installation ISO (Downloaded off the XenServer Website)

9750 driver disk in usb
XenServer 6.1 Installation ISO in ipmi virtual storage
boot: shell
-(When prompted to press enter for an installation or upgrade, instead type in shell and hit enter.)
mkdir driver
mount -o loop /dev/cdrom driver
rpm -i driver/3ware-modules-*
cp /lib/modules/2.6.32*xen/extra/3w-sas.ko /lib/modules/2.6.32.*xen/kernel/drivers/scsi
insmod /lib/modules/2.6.32.*xen/kernel/drivers/scsi/3w-sas.ko
lsmod|grep 3w
-(double check the driver is actually loaded)
exit
proceed with upgrade.

Unplug the USB cdrom before the server boots back up.

Making 3ware Raid Drivers for XenServer

I went and got the latest DDK from Citrix and imported it into the XenServer I wanted to make a driver for. On boot up you give it a password, but I also gave it an ip address so I could ssh into it (working in the console is not fun).

I went to the lsi-3ware site and got a copy of the latest code set and scp’d it over to the ddk VM. Since the DDK comes with the latest kernel, I went and checked which kernel it currently was using (2.6.32 in my case), unzipped the code set, and found the 2.6.32 .tgz file. I made a new folder in root to work in, and copied over the helloworld.spec and Makefile from /root/examples/driver/. Then I altered them to reflect the 3ware driver I wanted to make, i.e. wherever there was a reference to helloworld, I’d change it to 3w-sas. Since the driver expects the .tgz to be under /usr/src/redhat/SOURCES/ I went ahead and put it there. Running the make build-srctarballs command was supposed to unzip the .tgz file into the current directory, but because the contents of that .tgz is in a subfolder I manually unzipped it into place. Running make build-iso finished the process, and I was able to copy over the .iso to the dom0 and install the kernel module so under the new kernel we could use the 3ware card.