GlusterFS
GlusterFS
GlusterFS is a free software parallel distributed file system, capable of scaling to several petabytes.
GlusterFS is a network/cluster filesystem. GlusterFS package comes with two components, a server and a client. The storage server (or each in a cluster) runs glusterfsd and the clients use mount command or glusterfs client to mount the exported filesystem. Storage can be kept scaling beyond petabytes as demand increases.
To mount GlusterFS file systems, the client computers need FUSE support in the kernel.
For details, see http://en.wikipedia.org/wiki/GlusterFS
Installing
At the time of this writing, there is no FreeBSD port for gluster, you will first need to install FUSE manually.
cd /usr/ports/sysutils/fusefs-kmod make install clean echo "fusefs_enable="YES"">>/etc/rc.conf sh /usr/local/etc/rc.d/fusefs start
Next you will need to complile and install the glusterFS manually.
cd /tmp fetch http://ftp.zresearch.com/pub/gluster/glusterfs/1.4-pre/glusterfs-1.4.0pre5.tar.gz tar -zxf glusterfs-1.4.0pre5.tar.gz cd glusterfs-1.4.0pre5 export LDFLAGS="-L/usr/local/lib -liconv" export CFLAGS="-O0 -g -DDEBUG -I/usr/local/include/" ./configure -enable-fuse-client make install clean
Freebsd 7: Both client and server were installed successfully
Known Issues
ISSUE #1 (Freebsd 7 client) - cannot use fstab to mount the fuse partition:
glusterfs -f /usr/local/etc/glusterfs/client.vol /usr/local/www/mantis/files => works FSTAB: /usr/local/etc/glusterfs/client.vol /usr/local/www/mantis/files glusterfs rw 0 0 mount: /usr/local/etc/glusterfs/client.vol : Operation not supported by device => the error
ISSUE #2 (Freebsd 6.2 client) – Client segfault
[root@martini] $ glusterfs --no-daemon --log-file=/dev/stdout --log-level=DEBUG -f /usr/local/etc/glusterfs/client.vol /usr/local/www/mantis/files 2008-10-03 16:33:40 D [glusterfs.c:271:_get_specfp] glusterfs: loading volume specfile /usr/local/etc/glusterfs/client.vol Version : glusterfs 1.4.0pre5 built on Oct 2 2008 21:47:51 TLA Revision : glusterfs--mainline--3.0--patch-359 Starting Time: 2008-10-03 16:33:40 Command line : glusterfs --no-daemon --log-file=/dev/stdout --log-level=DEBUG -f /usr/local/etc/glusterfs/client.vol /usr/local/www/mantis/files given volume specfile +----- 1: ############################################## 2: ### GlusterFS Client Volume Specification ## 3: ############################################## 4: 5: volume gohabs 6: type protocol/client 7: option transport-type tcp/client 8: option remote-host gohabs 9: option remote-subvolume mantis 10: end-volume 11: 12: volume bondeau 13: type protocol/client 14: option transport-type tcp/client 15: option remote-host bondeau 16: option remote-subvolume mantis 17: end-volume 18: 19: volume afr 20: type cluster/afr 21: subvolumes bondeau gohabs 22: end-volume +----- 2008-10-03 16:33:40 D [spec.y:194:new_section] parser: New node for 'gohabs' 2008-10-03 16:33:40 D [xlator.c:289:xlator_set_type] xlator: attempt to load file /usr/local/lib/glusterfs/1.4.0pre5/xlator/protocol/client.so 2008-10-03 16:33:40 D [spec.y:219:section_type] parser: Type:gohabs:protocol/client 2008-10-03 16:33:40 D [spec.y:285:section_option] parser: Option:gohabs:transport-type:tcp/client 2008-10-03 16:33:40 D [spec.y:285:section_option] parser: Option:gohabs:remote-host:gohabs 2008-10-03 16:33:40 D [spec.y:285:section_option] parser: Option:gohabs:remote-subvolume:mantis 2008-10-03 16:33:40 D [spec.y:367:section_end] parser: end:gohabs 2008-10-03 16:33:40 D [spec.y:194:new_section] parser: New node for 'bondeau' 2008-10-03 16:33:40 D [xlator.c:289:xlator_set_type] xlator: attempt to load file /usr/local/lib/glusterfs/1.4.0pre5/xlator/protocol/client.so 2008-10-03 16:33:40 D [spec.y:219:section_type] parser: Type:bondeau:protocol/client 2008-10-03 16:33:40 D [spec.y:285:section_option] parser: Option:bondeau:transport-type:tcp/client 2008-10-03 16:33:40 D [spec.y:285:section_option] parser: Option:bondeau:remote-host:bondeau 2008-10-03 16:33:40 D [spec.y:285:section_option] parser: Option:bondeau:remote-subvolume:mantis 2008-10-03 16:33:40 D [spec.y:367:section_end] parser: end:bondeau 2008-10-03 16:33:40 D [spec.y:194:new_section] parser: New node for 'afr' 2008-10-03 16:33:40 D [xlator.c:289:xlator_set_type] xlator: attempt to load file /usr/local/lib/glusterfs/1.4.0pre5/xlator/cluster/afr.so 2008-10-03 16:33:40 D [xlator.c:324:xlator_set_type] afr: strict option validation is not enforced -- neglecting 2008-10-03 16:33:40 D [spec.y:219:section_type] parser: Type:afr:cluster/afr 2008-10-03 16:33:40 D [spec.y:352:section_sub] parser: child:afr->bondeau 2008-10-03 16:33:40 D [spec.y:352:section_sub] parser: child:afr->gohabs 2008-10-03 16:33:40 D [spec.y:367:section_end] parser: end:afr 2008-10-03 16:33:40 D [xlator.c:289:xlator_set_type] xlator: attempt to load file /usr/local/lib/glusterfs/1.4.0pre5/xlator/mount/fuse.so 2008-10-03 16:33:40 D [xlator.c:324:xlator_set_type] fuse: strict option validation is not enforced -- neglecting 2008-10-03 16:33:40 D [glusterfs.c:771:main] glusterfs: running in pid 99336 2008-10-03 16:33:40 D [fuse-options.c:140:fuse_options_validate] fuse-options: using mount-point = /usr/local/www/mantis/files 2008-10-03 16:33:40 D [fuse-options.c:147:fuse_options_validate] fuse-options: using attr-timeout = 1 2008-10-03 16:33:40 D [fuse-options.c:159:fuse_options_validate] fuse-options: using entry-timeout = 1 2008-10-03 16:33:40 D [fuse-options.c:171:fuse_options_validate] fuse-options: using direct-io-mode = 1 2008-10-03 16:33:40 W [common-utils.c:158:gf_print_bytes] glusterfs: Total data (in bytes): transfered (0), received (0) pending frames: Signal received: 11 configuration details:db.h 1 dlfcn 1 libpthread 1 spinlock 1 extattr.h 1 package-string: glusterfs 1.4.0pre5 Segmentation fault: 11 (core dumped) [root@martini] $ gdb glusterfs glusterfs.core GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "i386-marcel-freebsd"... Core was generated by `glusterfs'. Program terminated with signal 11, Segmentation fault. Reading symbols from /usr/local/lib/libglusterfs.so.0...done. Loaded symbols for /usr/local/lib/libglusterfs.so.0 Reading symbols from /usr/local/lib/libiconv.so.3...done. Loaded symbols for /usr/local/lib/libiconv.so.3 Reading symbols from /lib/libpthread.so.2...done. Loaded symbols for /lib/libpthread.so.2 Reading symbols from /lib/libc.so.6...done. Loaded symbols for /lib/libc.so.6 Reading symbols from /usr/local/lib/glusterfs/1.4.0pre5/xlator/protocol/client.so...done. Loaded symbols for /usr/local/lib/glusterfs/1.4.0pre5/xlator/protocol/client.so Reading symbols from /usr/local/lib/glusterfs/1.4.0pre5/xlator/cluster/afr.so...done. Loaded symbols for /usr/local/lib/glusterfs/1.4.0pre5/xlator/cluster/afr.so Reading symbols from /usr/local/lib/glusterfs/1.4.0pre5/xlator/mount/fuse.so...done. Loaded symbols for /usr/local/lib/glusterfs/1.4.0pre5/xlator/mount/fuse.so Reading symbols from /usr/local/lib/libfuse.so.2...done. Loaded symbols for /usr/local/lib/libfuse.so.2 Reading symbols from /libexec/ld-elf.so.1...done. Loaded symbols for /libexec/ld-elf.so.1 #0 0x281cb537 in pthread_testcancel () from /lib/libpthread.so.2 [New Thread 0x805e400 (sleeping)] [New Thread 0x805e200 (LWP 100102)] [New Thread 0x805e000 (LWP 100143)] (gdb) bt #0 0x281cb537 in pthread_testcancel () from /lib/libpthread.so.2 #1 0x281b9ee5 in sigaction () from /lib/libpthread.so.2 #2 0x281c2d31 in pthread_mutexattr_init () from /lib/libpthread.so.2 #3 0x281c2d9f in pthread_mutexattr_init () from /lib/libpthread.so.2 #4 0x2827a6bf in _ctx_start () from /lib/libc.so.6 #5 0x00000000 in ?? () #6 0xbfbfe460 in ?? () #7 0xbfbfe1a0 in ?? () #8 0x00000000 in ?? () #9 0x281c2d5c in pthread_mutexattr_init () from /lib/libpthread.so.2 #10 0x282f035b in init (this_xl=0x8056880) at fuse-bridge.c:2694 #11 0x0804ba34 in main (argc=7, argv=0xbfbfebb0) at glusterfs.c:775 (gdb) quit [root@martini] $ uname -imsr FreeBSD 6.2-STABLE i386 GENERIC