UDTFS: A remote file system based on UDT4 and FUSE
(C) 11/2009 Jan Wagner
Introduction
This is a read-only remote file system. It is based on FUSE (file system in userspace) and the UDT4 library (UDP-based Data Tranfer protocol).
The server program exports a root directory to multiple clients. Subdirectories and timestamps are supported. When clients attach to the server (mount command), shared files and directories are visible as in a normal file system.
There are two restrictions: the exported file system will be read-only and a single client can have only one file open at any time (in the current version). However, the server supports connections from multiple clients and each of them may access or copy files and directories concurrently.
For additional details see readme.txt
Source code
The source code is here udtfs-1.0.tar.gz and a SourceForge site will be up shortly.
Compiling should be as simple as make ; sudo make install. On the clients the FUSE packages have to be installed, see the readme.txt. On servers the FUSE packages are not required and you can ignore the compile errors relating to udtfs.c as long as udtfs_server.c compiles fine.
How to use the tools
After installing udtfs on two machines, here is how to share a directory from a server:
oper@tetricus:~$ udtfs_server /raid UDTFS V1.0 - A file system based on FUSE and UDTv4 (C) 2009 Jan Wagner, Metsahovi Radio Observatory, Aalto Licensed under GNU GPL v3 Config: 1500 MTU, UDT base port 9000, UDT ratelimit 1000 Mbps Sharing directory /raid Listening on port 1420
Note that at the moment the server does not yet support running in daemon mode. I hope to add this feature in the next release.
Now with a directory exported on the server, we mount it on two machines:
oper@pertinax:~$ udtfs tetricus.metsahovi.fi /mnt/fuse -f & UDTFS V1.0 - A file system based on FUSE and UDTv4 (C) 2009 Jan Wagner, Metsahovi Radio Observatory, Aalto Licensed under GNU GPL v3 TCP and UDT connected to server. oper@pertinax:~$ ll /mnt/fuse/ total 4 drwxr-xr-x 14 root root 153 2009-11-26 18:54 . drwxr-xr-x 5 root root 4096 2009-11-26 13:25 .. drwxr-xr-x 2 root root 6 2009-11-26 18:54 bt drwxr-xr-x 3 root root 4096 2009-06-26 12:49 onsala drwxr-xr-x 2 root root 35 2009-11-17 17:44 r1399on drwxr-xr-x 2 root root 91 2009-11-18 13:14 sa2410 drwxr-xr-x 2 root root 8192 2009-10-25 12:34 sa2510 drwxr-xr-x 3 root root 15 2009-11-18 17:39 v1310 drwxr-xr-x 2 root root 4096 2009-11-17 10:29 v1711 drwxr-xr-x 4 root root 24 2009-11-18 17:33 v1811 drwxr-xr-x 4 root root 24 2009-11-20 16:04 v2011 drwxr-xr-x 3 root root 15 2009-11-26 11:31 v2611 drwxr-xr-x 2 root root 798720 2009-11-25 13:52 wrtest drwxr-xr-x 2 root root 109 2009-09-02 12:20 yebes
Mounting on the second machine and listing another directory:
oper@abidal:/mnt$ udtfs tetricus.metsahovi.fi /mnt/udtfs/ -f & oper@abidal:/mnt$ ll /mnt/udtfs/ total 0 drwxr-xr-x 14 root root 153 2009-11-26 18:54 . drwxr-xr-x 9 root root 232 2009-12-01 10:02 .. drwxr-xr-x 2 root root 6 2009-11-26 18:54 bt drwxr-xr-x 3 root root 4.0K 2009-06-26 12:49 onsala drwxr-xr-x 2 root root 35 2009-11-17 17:44 r1399on drwxr-xr-x 2 root root 91 2009-11-18 13:14 sa2410 drwxr-xr-x 2 root root 8.0K 2009-10-25 12:34 sa2510 drwxr-xr-x 3 root root 15 2009-11-18 17:39 v1310 drwxr-xr-x 2 root root 4.0K 2009-11-17 10:29 v1711 drwxr-xr-x 4 root root 24 2009-11-18 17:33 v1811 drwxr-xr-x 4 root root 24 2009-11-20 16:04 v2011 drwxr-xr-x 3 root root 15 2009-11-26 11:31 v2611 drwxr-xr-x 2 root root 780K 2009-11-25 13:52 wrtest drwxr-xr-x 2 root root 109 2009-09-02 12:20 yebes oper@abidal:/mnt$ ll /mnt/udtfs/v2611/ total 0 drwxr-xr-x 3 root root 15 2009-11-26 11:31 . drwxr-xr-x 14 root root 153 2009-11-26 18:54 .. drwxr-xr-x 2 root root 4.0K 2009-11-26 14:28 Wz
Finally we can copy a few files:
oper@pertinax:/mnt/fuse/v2611/Wz$ ls -alh total 0 drwxr-xr-x 2 root root 4.0K 2009-11-26 14:28 . drwxr-xr-x 3 root root 15 2009-11-26 11:31 .. -rw-r--r-- 1 root root 256M 2009-11-26 11:58 smallfile -rw-r--r-- 1 root root 15G 2009-11-26 12:06 v2611_wz_No0001.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 12:11 v2611_wz_No0002.evn -rw-r--r-- 1 root root 34G 2009-11-26 12:32 v2611_wz_No0003.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 12:36 v2611_wz_No0004.evn -rw-r--r-- 1 root root 34G 2009-11-26 12:56 v2611_wz_No0005.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 13:00 v2611_wz_No0006.evn -rw-r--r-- 1 root root 34G 2009-11-26 13:18 v2611_wz_No0007.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 13:22 v2611_wz_No0008.evn -rw-r--r-- 1 root root 34G 2009-11-26 13:40 v2611_wz_No0009.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 13:44 v2611_wz_No0010.evn -rw-r--r-- 1 root root 34G 2009-11-26 14:02 v2611_wz_No0011.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 14:06 v2611_wz_No0012.evn -rw-r--r-- 1 root root 34G 2009-11-26 14:25 v2611_wz_No0013.evn -rw-r--r-- 1 root root 7.2G 2009-11-26 14:28 v2611_wz_No0014.evn -rw-r--r-- 1 root root 15G 2009-11-26 14:36 v2611_wz_No0015.evn oper@pertinax:/mnt/fuse/v2611/Wz$ cp smallfile /dev/null oper@pertinax:/mnt/fuse/v2611/Wz$ dd if=smallfile of=/dev/null 524288+0 records in 524288+0 records out 268435456 bytes (268 MB) copied, 5.5194 s, 48.6 MB/s oper@pertinax:/mnt/fuse/v2611/Wz$ dd if=v2611_wz_No0002.evn of=/dev/null 15000000+0 records in 15000000+0 records out 7680000000 bytes (7.7 GB) copied, 170.775 s, 45.0 MB/s
If you watch what happens on the server side:
server_sendsegment: sent 32.00MB at 864.811150 Mbps server_sendsegment: sent 32.00MB at 891.461920 Mbps server_sendsegment: sent 32.00MB at 854.644038 Mbps server_sendsegment: sent 32.00MB at 927.938321 Mbps server_sendsegment: sent 32.00MB at 877.520797 Mbps server_sendsegment: sent 32.00MB at 880.248464 Mbps server_sendsegment: sent 32.00MB at 887.084911 Mbps server_sendsegment: sent 32.00MB at 899.089610 Mbps server_sendsegment: sent 32.00MB at 890.207508 Mbps
The reason why the transfer rate is high but the actual average rate is about half lower is that the client at the moment does not use read-ahead. It requests a block of 32MB of data (cache) and once the data has arrived, returns smaller data segmets from it to the file system or user application. During this time, one could request and transfer the next 32MB block already, in the background. This doubles the sequential I/O throughput but requires some smart mutexed buffers. These may be implemented in the next client release version.
Finally, running two md5sum's in parallel on the same 256MB file but from different clients and also on the server:
oper@abidal:/mnt/udtfs/v2611/Wz$ md5sum smallfile 75d421bff4ab6025d516b4e12504cb13 smallfile oper@pertinax:/mnt/fuse/v2611/Wz$ md5sum smallfile 75d421bff4ab6025d516b4e12504cb13 smallfile oper@tetricus:~$ md5sum /raid/v2611/Wz/smallfile 75d421bff4ab6025d516b4e12504cb13 /raid/v2611/Wz/smallfile
To unmount:
oper@pertinax:~$ fusermount -u /mnt/fuse oper@abidal:~$ fusermount -u /mnt/udtfs/