Home

Search Posts:

Archives

Login

May 2007

S M T W H F S
1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30 31

zil_disable

by: Jeremy
at: May 19 2007 18:08
3 Comments (post new)

UPDATE 12/12/2010:

I noticed that this post is the first hit when you google for 'zil_disable' so I thought it was worth mentioning that 'zil_disable' has been replaced with a per-filesystem option in recent versions of ZFS. On snv 140 or later (which includes OI and the new Solaris Express), disabling the Zil is now done with the zfs 'sync' property! The original post remains below, but that mechanism is thankfully now obsolete.

I've been fighting a major performance issue with NFS over ZFS for a while now. It hasn't been a deal breaker or anything - everything I want to do over NFS has been working adequately - but I've noticed that my performance is subpar.

I didn't realize just *how* subpar until I started dealing with small files. Large files were generally OK - writing/reading a single large file via NFS would be about 80% as fast as with local storage. Since most of my mythtv operations are, obviously, with large files, this wasn't a big deal.

I noticed the problem with small files when I started migrating home directories to ZFS. Copying over the 85 MB ~mythtv directory took, according to time, 3m37.114s! What the hell is that about?

I then performed the same operation with rsync over ssh, which gave me a more reasonable result of 0m17.847s. That's almost 12 times faster!

I googled around a bit - well, more like a lot - and finally found out about zil_disable. Turns out that by adding 'set zfs:zil_disable=1' to /etc/system you can *dramatically* improve NFS performance on ZFS.

This is vaguely similar to turning on 'async' for Linux's NFSv3 server, but it impacts *all* filesystem operations, not only NFS. Effectively, NFS clients frequently request commit operations which inform the server to immediately write changes to stable storage. This happens at least on every file close over NFS, and the server itself forces a write to storage whenever metadata is modified. You can imagine then why there's such a huge impact when dealing with thousands of files - each single operation requires a flush of the cache and a write to disk.

After disabling the zil, I re-ran the same copy operation and achieved dramatically improved results:

mingus ~ # time cp -r ~mythtv /nfs/pvr/test/

real 0m18.265s
user 0m0.076s
sys 0m1.740s

Now that's more like it!

I've also noticed a moderate improvement with large files - both over NFS *and* locally.

I guess I'm actually a little surprised that the base configuration of the system would result in such horrid performance. I'm willing to make some trade-offs of performance for data consistency, but reading the blog mentioned above the risks seem tiny compared to the benefits of an order of magnitude performance increase.

Working on the MythTV box

by: Jeremy
at: May 19 2007 16:20
0 Comments (post new)

I've been working for a while to migrate all of my data to my raidz box off of my mythtv box itself - which I finally completed this week.  I've been planning to rebuild the system using software RAID, so getting all of the video data off made things much simpler (instead of the ultracrappy LVM setup I had previously - basically, I had appended 2 400 GB drives together as a workaround for my limited storage).

Once everything had been rsync'd over to my Solaris box, the process of actually rebuilding and moving to software RAID was pretty painless. I'd made the rsync backup using an LVM snapshot when the system was live, which worked really well. Rebuilding the system using a gentoo live CD only took about an hour, and everything is working fine.

Wiviz on OpenWRT White Russian

by: Jeremy
at: May 10 2007 23:31
1 Comments (post new)

I found a post in some other language describing what you need to do to enable wiviz on OpenWRT .9.

Basically:

- Add backports to /etc/ipkg.conf
- ipkg install wiviz wl
- wl monitor 1
- Hit your router's web server at http://192.168.0.1/wiviz/wiviz.html

Monitoring OpenWRT with NTOP

by: Jeremy
at: May 05 2007 12:29
4 Comments (post new)

I've been running OpenWRT on a Linksys WRTSL54GS for a few months now, and I've been incredibly happy with the results. One of the main attractions was the ability to run OpenVPN on one of these little embedded devices, which pretty much eliminated the need for a full-fledged Linux or Solaris box as a router.

Now, the only real disappointment I had with the setup was a lack of fine-grained monitoring that I'd been used to with NTOP. After poking around a bit, I learned that ntop could support NetFlows sent from other device via a plugin, and that the OpenWRT device could send NetFlows with fprobe. This means I can use my MythTV box (for example) to run ntop on, and still receive all the data from the linksys.

On the OpenWRT side of things, setup is really easy:

root@openwrt:~# ipkg install fprobe
root@openwrt:~# fprobe -ibr0 192.168.1.6:2055

Where '192.168.1.6' is replaced with the IP of your ntop box. If you want to automatically start sending Flows every time the router boots up, you can add this to an init script.

Assuming you already have ntop up and running, adding a NetFlow is pretty simple there too. Navigate to the configuration options for the NetFlow plugin and fill in the relevant information. Activate the NetFlow plugin, and then you can view the statistics by selecting "Switch NIC" from the "Admin" menu in ntop. Once you do so, you'll start seeing what the router sees, which depending on your network may be, well, everything that's going on. At the very least you'll be able to track all connections from your internal hosts to hosts outside of your network, as well as all wireless traffic using the device.