USB HDD tests
Guifré Molera - 23/04/2007
USB HDD testsGuifré Molera - 23/04/2007 |
|
|
Tests days take a couple of days of work, alternating between different modes of USB and Firewire recording and reading operations. The material used is composed by and HDD 500 GB enclosure with USB2 and/or Firewire connection. A cheap memory sticks of 1 GB and 500 MB capacity. All of them were used in jfs format, mainly for its efecteviness of formatting and erasing. Probably using other file system might varied a bit the result but not significantly. To evaluate the basic performance of each disk it has been used a couple of tools which gives a hint of the disk's performance. Tests were not using enough amount of data and probably either times were too short, but it is still accurate enough. Main results of the disks operations are:
USB2 HDD 500 GB gofrito@juliano:/usr/src/vsib$ sudo hdparm -t /dev/sde1 /dev/sde1: Timing buffered disk reads: 100 MB in 3.04 seconds = 32.87 MB/sec gofrito@juliano:~$ sudo cat /dev/sde1 | pv > /dev/null 2070MB 0:01:19 [32.15MB/s] [ <=> ] USB2 MEMORY STICK 1GB /dev/sdg1: Timing buffered disk reads: 48 MB in 3.04 seconds = 15.81 MB/sec gofrito@juliano:~$ sudo cat /dev/sdf1 | pv > /dev/null 970MB 0:01:19 [13.1MB/s] [ <=> ] FIREWIRE HDD 500 GB gofrito@juliano:/usr/src/vsib$ sudo hdparm -t /dev/sde1 /dev/sde1: Timing buffered disk reads: 100 MB in 3.04 seconds = 27.18 MB/sec gofrito@juliano:~$ sudo cat /dev/sdf1 | pv > /dev/null 3900MB 0:02:19 [27.20MB/s] [ <=> ]
The target was to produce real recording throught the vsib board to the usb disks, so it was tested configuring properly the formatter and the vsic VSIC was set up at mode D, and the formatter was commanded as: form=m,8,1:2 (at 128 Mbps) and form=m,16,1:2 (for 256 Mbps).
The measearuments let's us clear it's impossible to work with cheap memory sticks, also the follow link confirms the hypotesis. USB flash memory tests, the maximum recording rate was limited to 62 Mbps. On the other hand, the HDD enclosure using the USB2 port gives excellent results for recording at low rate (128 Mbps), as used until now in the geodetics experiments. The recording was done perfectly without errors and the ring buffer was filled only few Megabytes. But at 256 Mbps, the rate is too close to the limit predicted by hdparm -> 32.87 MB/sec == 262.96 Mbps, and the total output rate was close to 240 Mbps with ring buffer overflowed during the recording.
Even the performance of the Firewire vs. USB2 is assumed to be a bit better, it has not showed up during the experiments. There was not a problem to do the recording at 128 Mbps, similar than USB2. But when we move to 256 Mbps the resulting rate drops to 178. Main reason as we can see in the lspci tree it is due to PCI VSIB board and Firewire controller share the same PCI bus address, so the communications bus is overflowed by the data out/input, with no clever solution for it.
USB2 MEMORY STICK 1GB gofrito@juliano:/usr/src/vsib$ sudo ./wr 8000 16000 1 0 1 0 1 /media/usbdisk/t%06d 4000 < /dev/vsib Took 16.370638 seconds, 62.551013 Mbits(dec)/s (5.9% of PCI33), usleeps = 2738. USB2 HDD 500 GB gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 160000 0 0 0 0 1 /media/usbdisk/t%03d 40000 /dev/vsib Took 160.426336 seconds, 127.659838 Mbits(dec)/s (12.1% of PCI33), usleeps = 23103. vsib: big secondary ring buffer filled to 6144000 bytes gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 3200000 1 2 0 0 1 /media/usbdisk/t%06d 320000 /dev/vsib Took 1794.831861 seconds, 228.210792 Mbits(dec)/s (21.6% of PCI33), usleeps = 15460. vsib: big secondary ring buffer filled to 143200000 bytes FIREWIRE HDD 500 GB gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 160000 0 0 0 0 1 /media/usbdisk/t%03d 40000 /dev/vsib Took 160.521463 seconds, 127.342521 Mbits(dec)/s (12.1% of PCI33), usleeps = 23103. vsib: big secondary ring buffer filled to 7234000 bytes gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 3200000 1 2 0 0 1 /media/usbdisk/t%06d 320000 /dev/vsib Took 2000.831861 seconds, 178.152364 Mbits(dec)/s (21.6% of PCI33), usleeps = 15460. vsib: big secondary ring buffer filled to 143251000 bytes
lspci tree shows clear that IEEE-1394 controller and the VSIB board (Unknown device) shares the same bus address, it is also the case for the rest of computers in the VLBI rack: Didius, Tetricus and Pertinax. So even the latest NForce 5 mobo has not independent Firewire controller. So no chances to test the power of both techs independently using the VSI board.
gofrito@juliano:~$ lspci -tv
+-1d.7 Intel Corporation 82801G (ICH7 Family) USB2 EHCI Controller
+-1e.0-[0000:01]--+-01.0 PLX Technology, Inc. Unknown device 5406
| \-03.0 Texas Instruments TSB43AB22/A IEEE-1394a-2000 Controller (PHY/Link)
The idea is to connect two USB memories in order to create an array to increase the effectivity of the High Speed connection. Must remember that the specifications of USB2 guarantees a 480 Mbps bus of data. As to set up a raid of disk or also called md (multiple disk) the disks used might have similar specifications, the tests done has been only using small memory flash. It is expected to receive a second HDD enclosure soon and repeat the tests with higher volums. Meanwhile the test used 2 1GB and 1 500 MB flash memory drives.
2 USB2 MEMORY STICKS gofrito@juliano:~$ sudo hdparm -t /dev/md1 /dev/md1: Timing buffered disk reads: 92 MB in 3.06 seconds = 30.03 MB/sec gofrito@juliano:~$ sudo cat /dev/md1 | pv > /dev/null 1.9GB 0:01:01 [31.5MB/s] [ <=> ] 3 USB2 MEMORY STICKS gofrito@juliano:~$ sudo hdparm -t /dev/md1 /dev/md1: Timing buffered disk reads: 110 MB in 3.06 seconds = 38.03 MB/sec
And it seems to work, since looking to the vsib recording of two memory sticks allows to run small vsib recording at 128 Mbps (while did not before). But the buffer is easiliest filled than the singld HDD, in this case its around 20 MBytes. ,
gofrito@juliano:~$ sudo ./wr 20000 120000 0 0 0 0 1 /media/usbdisk2/t%06d 40000 < /dev/vsib Took 150.766210 seconds, 127.349490 Mbits(dec)/s (12.1% of PCI33), usleeps = 19149. /media/usbdisk2/t%06d max. 1.267820 seconds.
To enable the use of Firewire: First needs to be enabled from the bios (it was not), second might be loaded the modules automatically (on the /etc/modules), with the follow order: ieee1394 ohci1394 sbp2 Once done, it should be automatically find on the lspci devices and also the kernel should load the scsi disk when it is plug in. In our case will an extra sd disk-> /dev/sde1 Last tests using the firewire on Juliano: lspci shows clear the dependence of the PCI bus of VSIB board and Firewire which limits the performing of both when is meant to achieve data and record it to the Firewire HDD. Discarding the idea of possible 512 Mbps using 2 USB2 + 1 Firewire...
To extend the tests it has been purchased a second enclosure for HDD. This includes compatibility with PATA and SATA, and also USB2, Firewire and RJ-45 at 10/100 Mbps. It was also thought to get a 1 Gbps HDD enclosure (D-Link or Netgear) they had already in the market similar structures, which offers 2 S-ATA disks mounted as RAID, and interface based on Firewire, USB2 or ethernet at 1 Gbps. But proofreading the manuals showed up the max performance achievable is 120/180 Mbps (reading/writing). The results are fare away from the Gigabit connectivity or the USB2 interface.
gofrito@juliano:~$ sudo hdparm -t /dev/sdf1 /dev/sdf1: Timing buffered disk reads: 100 MB in 3.02 seconds = 33.09 MB/sec gofrito@juliano:~$ sudo cat /dev/sdf | pv > /dev/null 14.1GB 0:07:15 [33.2MB/s] [ <=> ] gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 480000 0 0 0 0 1 /media/usbdisk/t%03d 80000 /dev/vsib Took 262.339687 seconds, 234.200173 Mbits(dec)/s (22.2% of PCI33), usleeps = 10418. gofrito@juliano:/dev$ sudo hdparm -t /dev/md1 /dev/md1: Timing buffered disk reads: 136 MB in 3.02 seconds = 45.02 MB/sec gofrito@juliano:/dev$ sudo cat /dev/md1 | pv > /dev/null 9.05GB 0:03:23 [45.8MB/s] [ <=> ] gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 480000 0 0 0 0 1 /media/usbdisk/t%03d 80000 /dev/vsib Took 240.906931 seconds, 255.036249 Mbits(dec)/s (24.2% of PCI33), usleeps = 31166. gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 480000 0 0 0 0 1 /media/usbdisk/t%03d 80000 /dev/vsib Took 166.334955 seconds, 369.375156 Mbits(dec)/s (35.0% of PCI33), usleeps = 3739
Create the file block md1: mknod md1 b 9 1; chown root:disk md1; chmod 660 md1
Notes about the VSIB functionality:
modeThe table is clearly explained on the documentation pre-sw-use.pdf. Summary for a fast use:
mode=0 32 bits used rate 1600 Mbps
mode=1 16 bits used rate 800 Mbps
mode=2 8 bits used rate 400 Mbps
Skip mode It means how many bits we skip for each counted, so if skip=1 the rate will be decreased 1/2. skip=2 rate will be decreased 1/3 and this serie for the rest of cases.