Archive for the 'OpenSource' Category
Project Kemari
Kemari is a virtual machine synchronization mechanism for Xen. It allows you to run a hot standby copy of a virtual machine on a different host. If the active host becomes unavailable for some reason, the standby host takes over without downtime. (Note that this is different from live migration, where both nodes need to be up and running.) This allows for higher availability of your servers. Of course VMware had this feature for some time, but I’m glad someone is working on an open source alternative.
Unfortunately most of the Linux distributions are focusing on KVM instead. For RedHat this makes sense, because they acquired Qumranet (who developed KVM). But Debian and Ubuntu also pretty much gave up on Xen. The only mainstream Linux distribution still actively working on Xen support is SuSE. Actually, the only UNIX based platform that really has good Xen integration is not even Linux based, it’s OpenSolaris, go figure.
I agree that in the long run a Linux kernel based approach to virtualization makes sense, but I also believe today Xen has an edge over KVM and will keep that edge, probably for years to come. Why not bet on both horses and let end users decide which virtualization solution to use?
No commentsvirt-manager on Ubuntu Intrepid Ibex
Today I’ve been playing with virt-manager/libvirt (a Redhat emerging technologies project) on the new upcoming Ubuntu release Intrepid Ibex (8.10) and.. well.. it just works! I’m using it with KVM, for which you need hardware virtualization support.
It’s quite easy to create a new virtual machine, just clicky-clicky in the GUI, select an ISO to boot from and you’re ready to go. Even networking is automatically configured for you (using NAT by default). I’m impressed! Thank you, Redhat and Ubuntu developers, wherever you are! 
Native iPhone/iPod touch support in Linux? There is hope after all!
Matt Colyer and friends have been busy reverse engineering the USB protocol used by Apple to sync iPhone and iPod touch devices and is now working on iFuse, an impressive project that will provide native USB access to those devices in Linux without the need for jailbreaking! As the name suggests, it is implemented as a FUSE filesystem.
By the way, you still need an app using libgpod to actually update the music library database on the iPod. iFuse only handles the mounting of the filesystem. I’m not sure if this works with Rhythmbox (since it uses libgpod for iPod support), but I guess I’ll just have to try. At least it works with gtkpod for the time being, which is better than the slow and inconvenient alternatives: jailbreaking and syncing over WiFi or running iTunes in a virtual machine (I use vmware player for this, because it is the only solution to properly virtualize USB 2.0).
1 commentiPod touch syncing
This week VMware released updated versions of their Workstation and Player products. This fixes a long standing issue with USB support for the iPod touch and iPhone. I’m now finally able to sync my iPod touch from iTunes in a virtual machine! Just don’t try to update your firmware from a virtual machine.. It fails and renders the device unusable until you flash it from a native (non-virtualized) iTunes :S
Not being able to sync my iPod touch was one of the major regressions I had since I dumped Windows. It still is not the best possible solution, because I still need Windows in a virtual machine for this to work. Unfortunately there hasn’t been much progress with regard to native Linux support for these things without jailbreaking them..
If I’m going to buy a new portable music player some day it sure as hell isn’t going to be an Apple product, unless they open up and allow other people to interface with them. If you are a Linux user: Don’t buy one of these things. I know they’re sexy and all, but really .. don’t!
Weblog maintenance
I installed a reCaptcha plugin for Wordpress, which allows people to directly comment on my posts without having to register for an account first. Should be effective against spammers, who (despite the fact that they had to register for an account) were spamming my moderation queue.
If you also happen to run Wordpress I highly recommend it! Simply sign up for an API key, drop the files in your plugin directory and enable the plugin from the admin interface. As an added bonus you help to digitize books (Did I mention it is free?)
OpenLDAP improvements
I just discovered that OpenLDAP now supports multimaster replication! This means there finally is a serious enterprise-ready(?) and open alternative for the Netscape directory server family (on which both Fedora DS and Sun JES are based iirc). Without multimaster replication it is impossible to create a real redundant directory setup, which is what you want when your whole data center depends on LDAP for authentication/authorization or other types of lookup maps. (Well, you could always just replicate the directory, but you would not be able to change anything in it while the master server was down.)
From the OpenLDAP roadmap:
- OpenLDAP 2.4 (released October 2007)
- Functional enhancements and improved scalability:
- Updated slapd dispatcher
- MirrorMode and MultiMaster replication
- Proxy Sync replication
- Expanded monitoring
- Multiple new Overlays
- Expanded documentation
- New socket backend (experimental)
- LDAPv3 extensions:
- LDAP Chaining Operation support
- LDAP Don’t Use Copy Control support
- LDAP Dynamic Directory Services (RFC2589)
- LDAP Transaction support (work in progress)
Kudos to the OpenLDAP development team!
1 commentToday is the day I got rid of my Windows partition
Woohoo! The developers of the music program I work with (Renoise) have released a beta of a Linux build! That means I can finally make music in Linux and get rid of my dualboot Windows partition!
Unfortunately I still need a Windows virtual machine to be able to manage my Nokia N80 (backups, firmware updates) and my iPod touch (a very cool christmast gift from my employer). But at least I don’t have to reboot to make some music anymore! 
Server migration
We have finally migrated to our new server! This hardware replacement was long due. The old one (a dual Intel PIII-700 with 1Gb of memory) started to show some hardware and performance problems. Everything should be fast and smooth again
Because of this migration this weblog and my pet project TheMirror have been offline for about a day.
On the old server we used ispworks to manage virtual hosts, but the project didn’t have an update since december 2003. We now use ispconfig to manage apache virtual hosts, ftp accounts, email accounts, mysql databases, dns stuff, bandwith limits, quota’s, you name it. It features a nice self-service panel on which users can reset their password and configure some basic spamassassin settings. So far I’m very satisfied.
I would like to express my gratitude towards Thijs, Johannes and Jacob for preparing and doing the actual migration. Unfortunately I didn’t have much spare time to help out. Thanks guys!
I would also like to thank our end users for the generous donations which made it possible to actually buy the new hardware.
No commentsStop spamattacks with postfix in a high volume environment
Lately I’ve been doing some tweaking of the incoming mailservers at a customer site. They have about 500.000 active mailboxes and have been flooded with spambots over the last weeks. It probably has something to do with the storm worm. Such an attack puts a heavy strain on the infrastructure and some measures were needed to make sure the whole thing didn’t fall apart.
On my own mailserver I simply use postgrey to greylist unknown sources. But in such an environment any delays on mail delivery are unacceptable. Besides, I doubt postgrey scales well to such a high volume environment.
These are the measures I took to slow down spambots and making sure the mailservers can still accept mail from legitimate senders even when under a spamattack.
Use rate limiting on your inbound connections.
Allow only 10 connections/minute from the same sender with a maximum of 10 open connections.
It will slow down spammers that are not blacklisted in any of the RBL’s you configured and are using a dictionary attack in an attempt to guess valid email addresses in order to spam them.
You will have to enable the anvil service in your master.cf and add the following settings to your main.cf :
anvil_rate_time_unit = 60s
smtpd_client_connection_count_limit = 10
smtpd_client_connection_rate_limit = 10
There might be some parties that you receive a lot of mail from and don’t want to be throttled. For example this particular customer uses the nlwhitelist to make sure dutch ISP’s are never throttled.
smtpd_client_event_limit_exceptions = hash:/etc/postfix/whitelist, $mynetworks
Increase available resources
Disconnect inactive smtp sessions after 30 seconds (instead of the default of 300 seconds). Some spambots open up a lot of connections and sit there taking up all your resources. Legitimate connections will then be refused if the maximum of connections is reached.
(You can also consider increasing the maximum number of connections postfix will accept.)
smtpd_timeout = 30s
This should result in better availability of your SMTP service and stop at least some spambots even before they reach your RBL checks or your spamfilter.
You’ll soon notice these kind of lines in your postfix logfiles:
Sep 5 13:14:45 **** postfix/smtpd[27510]: [ID 947731 mail.warning] warning: Connection rate limit exceeded: 217 from *.sdsl.bell.ca[69.159.*.*] for service smtp
217 blocked connections in one minute from just one IP. Not bad
Fixing a degraded Linux software RAID-1 mirror
A quick HOWTO on how to fix a degraded Linux software RAID-1 mirror, mostly a reminder for myself in case I run into this problem again.
First replace the faulty disk with a new drive of at least the same size.
Copy the MBR of the active disk (sda) to the new disk (sdb):
# dd if=/dev/sda of=/dev/sdb bs=512 count=1
Copy the partition table of the active disk to the new disk:
# sfdisk -d /dev/sda | sed s/sda/sdb/g > /tmp/ptable
# sfdisk /dev/sdb < /tmp/ptable
Inform MD about the new partitions:
# for MD in /dev/md?
> do
> NEWPART=$(mdadm --detail $MD | awk '/active sync/{ print $7}' | sed s/sda/sdb/)
> mdadm -a $MD $NEWPART
> done
(Disclaimer: Do not just blindly copy and paste these lines. Your setup might be different and you could break stuff if you don’t use mdadm wisely.)
I noticed this particular system wasn’t able to boot from the second disk. It seems that the GRUB bootloader isn’t very much RAID aware and didn’t install the bootloader on to the second disk. So I installed good ol’ LILO instead, which seems to be able to handle such a setup better. I added the following lines to /etc/lilo.conf:
boot=/dev/md0
root=/dev/md1
raid-extra-boot=/dev/sda,/dev/sdb
If you run the lilo command it writes the bootloader to both disks, so you should be able to boot from both, even in case of a failing disk.
4 comments