Thoughts…

Mixed results with the new Ubuntu so far.

At work, my desktop machine refused to install the latest kernel which left me with a broken system for a while, awesome!

At home, the upgrade when smoothly however I had issues with ZFS-Fuse. The filesystem refused to mount correctly. I’ve eventually isolated this to a problem with the drive order however I had specified ‘by-path’ to eliminate that problem? Further investigation reveals that drives on a SATA card appear to be swapping since the upgrade to Lucid, very odd…. sorted this now by mounting via id instead.

sudo zpool export tank
sudo zpool import -d /dev/disk/by-id/ tank

Another issue seen is the lack of sleeping disk drives. The drives of the ZFS raid are supposed to pop into standby when idle for 15 mins… this then allows the machine to switch itself off at night when not in use. However for some reason that I have not yet identified they are not going sleepy-bye-byes.
I am able to manually put the western digital drive into slumber, but the samsung disks refuse and to make life interesting they give no reason why.

hdparm doesn’t appear to have changed? I am perplexed.

Haven’t dared to upgrade my vmware server box just yet, will give it another week or so

WIN! I only have about 500MB left on my internet usage this month so grabbing some entertainment was going to be a no no… until I remembered my phone!

Take one phone, an unlimited internet contract, a USB cable and *BING*, one laptop connected to the internet via my phone! Win win!

Sure it’s pretty slow, but slow is better than not at all!

Will be interesting to see at what point I get traffic shaped

Starting to make some headway on the new box now. Spent 2 days trying to figure out iptables to get port forwarding to work, so that getting to the web server would be possible… and lo and behold it’s a VMware setting I should have been looking at not iptables. Grrr

Still on the way now. Web server in place and running. Nothing left to do now but finish off the website for North Ridge and reap my rewards! a busy weekend for me then

After just a few hours of my server being up and running, I spotted the appalling performance that zfs-fuse appeared to be giving me.

Massive pauses during write operations, not very good at all…

So I’ve been thinking all afternoon of possible alternatives. I’d been attracted to zfs due to it’s awesome ease of use and the fact I’ve been using it on my home server for months now. Yet I realise now that my home server is a file server, simply serving up files and any performance hit I may be experiencing is missable as it doesn’t impact on streaming media files to a media player and I don’t notice any problem when writing files to it either.

But.. my home server is not booting from a USB pen drive and not forming the basis for a virtual environment. My new server of course, is.

So again the question, problem with the USB or problem with ZFS-fuse?

I could do a full reinstall and install to disk this time… or I could alter my raid. Afterall the reason I put the OS on the USB in the first place was to leave the disks untouched and fully available for storage. A perfect opportunity therefore to simply wipe the zfs zpool and start the RAIDing again.

So… first problem. I want to be able to test vaguely accuratley if the change in RAID helps, so to keep the VM currently stored on it…. Nicely enough the VM gzips down to a managable 650MB so I could easily download it. However I had a minor brainwave and remembered that /tmp was a tmpfs drive and not actually on the USB key, so threw it in there, win!

Bonza! Next issue, I don’t know any other raid type stuff other than zfs, you know because it’s easy and I try to keep away from complicated. I’ve heard ‘md’ spoken around the office and more importantly by people I trust to know a thing or two about this sort of thing, so deep into google I went and started to learn about mdadm, more win.

After sifting through rather a large amount of twoddle, I’ve picked up that which I think I need to know… the basics appear clear enough.

Create a suitable partition on each drive in question, run the relevant mdadm command, make a filesystem and mount it. Simples.

Actually, it was simple I even went as far as to make the filesystem ext4 cos I’m just like that.

A couple of commands later and we have our VM restored and running. A few commands more and the VM is working away at writing data to the RAID, raid 5 I should probably mention.

And so far, for the last 20 mins at time of writing, all is very well. There have been no ‘obvious’ signs of crappy performance. I is pleased.

Based on this I’m forced to take another look at my original design for this server. Do I still want this to run from USB? I see now that it’s a pretty serious single point of failure. Sure, my redundancy plan of a 2nd identical USB stick, would still work. But rejigging my disks and partitions a touch would leave me with redundancy that would have no downtime on single disk failure. Definately more thought needed, but soon since I want to start working on this server now.

Soooo… the new server is in and installed. The OS (Ubuntu 9.04-server) is on a USB key and VMWare Server 2 is installed also.

Great! After getting the relevant ports opened up, I hit the first snag. I can’t seem to browse to the web interface, slight issue.

After much playing around and chcking of logs, I simply reboot and F5 and lo and behold it starts working… better but not promising. Still it’s been working since, so I won’t tempt fate and leave it at that.

Next, time to create a virtual machine. I already have my 4x 500GB drives mounted and ready via zfs-fuse, so we use that as out ‘datastore’ and create our new machine… Creation is easy, getting it to turn on appears to be less so. A wonderful error that means very little “Failed to initialize monitor device”… ok. Google points to issues with the vmmon module.. but all ariticles I find relate to failure of the vmmon module to be built correctly…. which it wasn’t. The module is there and is up and running nicely. Still I faithfully re-ran the vmware config command again and again, rebuilding the modules again and, indeed, again. No luck.

This morning a colleague advises that having virtualisation enabled in the bios could cause this issue… interesting. I had indeed enabled that thinking it might ‘help’! Ok disable and reboot, all appears well and whoopla(!) the box powers up!

Excellent, now installation. What follows is the longest MINIMAL debian install I have ever witnessed. At least an hour and change. Now the question: Is this an issue with the OS running on a USB key, or performance related to running a VM from a zfs-fuse filesystem? Unknown at this time.

I finished the install and have added some applications and I’ll keep an eye on it. If I continue to see strange things I may well switch from zfs-fuse to md/lvm to manage my disks. Although this means significantly more complicated / time consuming configurations.

New server arrived on Wednesday. Exciting stuff!

4x 500GB drives installed, after a mad search about for small enough screws!

Internal USB ports make sticking the OS USB key in nice and neat.

Installed the server into the RAQ and cabled it up… and promptly forgot my IP details doh! So no network connection for the moment! Sort that out today.

Apart from that all systems appear a go!

Another excellent test of my new server ideas.

After booting my home file server from my Ubuntu USB stick I installed zfs-fuse… it automatically found and mounted my zfs drives… awesome!

I’ll have to speed up a little bit as my new server is due to arrive on Wednesday!

Next will be a VMWare server install. Exciting stuff I know

The plans for my new server are moving along quite nicely. A test today of an operating system installed to a USB pen drive went swimmingly.

Ubuntu Server runs excellently from the pen drive and is quick to boot also, win!

New Dell server should arrive by the end of the month, then it’s plan implementation time.

Dell r410 running Ubuntu Server (from USB drive) with 4 500GB disks zfs’d up, oh yeah and VMWare Server as well… we shall see if it’s up to the task, but so far it’s looking very promising.