Thoughts…

Starting to make some headway on the new box now. Spent 2 days trying to figure out iptables to get port forwarding to work, so that getting to the web server would be possible… and lo and behold it’s a VMware setting I should have been looking at not iptables. Grrr

Still on the way now. Web server in place and running. Nothing left to do now but finish off the website for North Ridge and reap my rewards! a busy weekend for me then

After just a few hours of my server being up and running, I spotted the appalling performance that zfs-fuse appeared to be giving me.

Massive pauses during write operations, not very good at all…

So I’ve been thinking all afternoon of possible alternatives. I’d been attracted to zfs due to it’s awesome ease of use and the fact I’ve been using it on my home server for months now. Yet I realise now that my home server is a file server, simply serving up files and any performance hit I may be experiencing is missable as it doesn’t impact on streaming media files to a media player and I don’t notice any problem when writing files to it either.

But.. my home server is not booting from a USB pen drive and not forming the basis for a virtual environment. My new server of course, is.

So again the question, problem with the USB or problem with ZFS-fuse?

I could do a full reinstall and install to disk this time… or I could alter my raid. Afterall the reason I put the OS on the USB in the first place was to leave the disks untouched and fully available for storage. A perfect opportunity therefore to simply wipe the zfs zpool and start the RAIDing again.

So… first problem. I want to be able to test vaguely accuratley if the change in RAID helps, so to keep the VM currently stored on it…. Nicely enough the VM gzips down to a managable 650MB so I could easily download it. However I had a minor brainwave and remembered that /tmp was a tmpfs drive and not actually on the USB key, so threw it in there, win!

Bonza! Next issue, I don’t know any other raid type stuff other than zfs, you know because it’s easy and I try to keep away from complicated. I’ve heard ‘md’ spoken around the office and more importantly by people I trust to know a thing or two about this sort of thing, so deep into google I went and started to learn about mdadm, more win.

After sifting through rather a large amount of twoddle, I’ve picked up that which I think I need to know… the basics appear clear enough.

Create a suitable partition on each drive in question, run the relevant mdadm command, make a filesystem and mount it. Simples.

Actually, it was simple I even went as far as to make the filesystem ext4 cos I’m just like that.

A couple of commands later and we have our VM restored and running. A few commands more and the VM is working away at writing data to the RAID, raid 5 I should probably mention.

And so far, for the last 20 mins at time of writing, all is very well. There have been no ‘obvious’ signs of crappy performance. I is pleased.

Based on this I’m forced to take another look at my original design for this server. Do I still want this to run from USB? I see now that it’s a pretty serious single point of failure. Sure, my redundancy plan of a 2nd identical USB stick, would still work. But rejigging my disks and partitions a touch would leave me with redundancy that would have no downtime on single disk failure. Definately more thought needed, but soon since I want to start working on this server now.

Soooo… the new server is in and installed. The OS (Ubuntu 9.04-server) is on a USB key and VMWare Server 2 is installed also.

Great! After getting the relevant ports opened up, I hit the first snag. I can’t seem to browse to the web interface, slight issue.

After much playing around and chcking of logs, I simply reboot and F5 and lo and behold it starts working… better but not promising. Still it’s been working since, so I won’t tempt fate and leave it at that.

Next, time to create a virtual machine. I already have my 4x 500GB drives mounted and ready via zfs-fuse, so we use that as out ‘datastore’ and create our new machine… Creation is easy, getting it to turn on appears to be less so. A wonderful error that means very little “Failed to initialize monitor device”… ok. Google points to issues with the vmmon module.. but all ariticles I find relate to failure of the vmmon module to be built correctly…. which it wasn’t. The module is there and is up and running nicely. Still I faithfully re-ran the vmware config command again and again, rebuilding the modules again and, indeed, again. No luck.

This morning a colleague advises that having virtualisation enabled in the bios could cause this issue… interesting. I had indeed enabled that thinking it might ‘help’! Ok disable and reboot, all appears well and whoopla(!) the box powers up!

Excellent, now installation. What follows is the longest MINIMAL debian install I have ever witnessed. At least an hour and change. Now the question: Is this an issue with the OS running on a USB key, or performance related to running a VM from a zfs-fuse filesystem? Unknown at this time.

I finished the install and have added some applications and I’ll keep an eye on it. If I continue to see strange things I may well switch from zfs-fuse to md/lvm to manage my disks. Although this means significantly more complicated / time consuming configurations.

New server arrived on Wednesday. Exciting stuff!

4x 500GB drives installed, after a mad search about for small enough screws!

Internal USB ports make sticking the OS USB key in nice and neat.

Installed the server into the RAQ and cabled it up… and promptly forgot my IP details doh! So no network connection for the moment! Sort that out today.

Apart from that all systems appear a go!