LizardFS linux CookBook

Setting DirectIO for your setup

In some cases we have experienced that the caching mechanism in some systems may slow down performance significantly. What has helped is switching the caching off and moving to Direct IO which omits the OS cache and writes directly to the block device underneath.

To enable DirectIO on your installation, you need to update the .lizardfs_tweaks file in the root of your mounted LizardFS. This is done by issuing the following on a mounted file system:

echo "DirectIO=true" > .lizardfs_tweaks

You can verify if the setting has changed to true by issuing the following command:

cat .lizardfs_tweaks | grep DirectIO

If you find that this does not improve your performance or in fact, slows it down, you can always change it back by running:

echo "DirectIO=false" > .lizardfs_tweaks

The changes are effective immediately.

URAFT Cookbook

The HA system utilized by LizardFS to keep your master servers always alive is called uraft. It has been developed by Sky Technologies Sp. z o.o. and is based on the raft algorithm developed by Diego Ongaro and John Ousterhout.

HA with only 2 masters

Warning

This is unsupported and only recommended if setup by certified engineering personnel.

Since it is a quorum based algorithm we usually recommend to users to have 1 master and 2 shadow nodes. But there is a way to run your HA with one master, one shadow and a raft only add-on on one of your chunkservers. This stub will still run a master server daemon but it will never switch it to active so it can be running anything.

All that is required to switch a node to “non master” mode is setting:

URAFT_ELECTOR_MODE = 1

in the lizardfs-uraft.cfg file. Everything else must be setup like it would be a normal lizardfs-master with uraft node except that the master will never be put into a real master role.

ZFS on Linux

ZFS is a high performance 128 bit file system developed by SUN Microsystems. We will show you here the basics how to install it on Linux. For specifics how to fine tune, optimize and manage zfs, please consult the links in the “see also” part at the end of the ZFS articles. On Linux we use the Open-ZFS way and do not use FUSE to get maximum performance.

Installing ZFS on RHEL/Centos 7

To avoid all the licensing discussions (we do not get into that but you can read up on it <here https://www.softwarefreedom.org/resources/2016/ linux-kernel-cddl.html>_ if you like) the Open-ZFS project has a way where you while installing the driver compile it yourself and that way get around all the license discussions for binary modules it seems. So here we go:

You will require to add the epel repository to your system:

$ yum install epel-release
$ yum update

And than the open-zfs project repository:

$ yum localinstall -y --nogpgcheck http://archive.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm

after which you can install the sources required and automatically build the required modules on your system:

yum install -y kernel-devel zfs

Test if your installation worked:

modprobe zfs
lsmod | zfs

Test if you can use the zfs commands:

zfs list
zpool list

Now you can install zpools and file systems with ZFS.