« www.tkk.fi

Suomeksi | På svenska | In English | GER | CAT | SPA | EPO | IT | Old

Siirry sivun sisällön alkuun





How To Record VLBI data on USB disks

Guifré Molera - Fri Jun 29 11:38:10 EEST 2007

TKK logo MRO logo

Introduction

The purpose of the web page is to show the ability of recording VLBI observations using the MRO-VSIB-PC, developed by Metsähovi, and external USB disks. The USB disks are compoused by USB HDD enclosure which allows connect S-ATA and P-ATA to a USB2.0 ports. The models used in the laboratory (Agestar NCB3AST and CBOX) also let configuring the disks using Firewire interface or 10/100 Mbps conectivity. Both of them are not strongly recommended for different reasons. The operational log is stored on USBdisk-log and it can be followed to get a correct configuration.


How to set up a correct HDD configuration

128 Mbps observations

For single observations at 128 Mbps, as used in the geodetics experiments carried on in Metsähovi, a single unit of 1 USB2.0 hard disk is needed. The real speed of an USB2.0 port is aproximetely up to 260 Mbps (see USBdisk-log), even the specifications claim to rates close to 480 Mbps. Even in this case there is enough margin to record at 128 Mbps without a problem. Basic configuration is set up by the HDD enclosure connected to USB2.0 port (nowdays all COTS have included USB2.0 controller chip), create a partition of the whole disk and give format to the unit (our experience advises to use jfs format) and mount it to the desired directory

gofrito@juliano:sudo fdisk /dev/sde	
gofrito@juliano:sudo mkfs.jfs /dev/sde1
gofrito@juliano:sudo mount -t jfs /dev/sde1 /media/usbdisk/

256 Mbps observations

For observations at 256 Mbps, it is compulsory to use two hard disks connected to two different USB2.0 ports. Even the tests claims that the USB port speed is higher than the recording rate, during the recording it is easily to overflow the limit and fail the recording. Both HDD disks will be set up as a RAID of 2 disks so both are accessed simultaneously as they were just a single unit. The real speed then of the device is 3/2 times faster than the previous, around 360 Mbps. It becomes easy to achieve the target of 256 Mbps. Basic instructions

gofrito@juliano:sudo fdisk /dev/sde ; sudo fdisk /dev/sdf
gofrito@juliano:sudo mdadm sudo mdadm --create /dev/md0 --chunk=256 --raid-disks=2 --level=0 /dev/sde1 /dev/sdf1
gofrito@juliano:sudo mkfs.jfs /dev/md0
gofrito@juliano:sudo mount -t jfs /dev/md0 /media/usbdisk/

Observation results

128 Mbps observations

The Tests resume recording at 128 Mbps, using Mark IV formatter direct to our vsib pc and recording to a mounted USB disk, gives a good track of the 128 Mbps rate target and with a significant low value of the ring buffer pointer. Mainly the ring buffer indicates how much "fast" memory is required to store the data before storing it to the HDD, low value means the transfer from the vsib board to the HDD is fast and it is not using a high power of the CPU and RAM.

USB2 HDD 500 GB
gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 160000 0 0 0 0 1 /media/usbdisk/t%03d 40000 /dev/vsib
Took 160.426336 seconds, 127.659838 Mbits(dec)/s (12.1% of PCI33), usleeps = 23103.
vsib: big secondary ring buffer filled to 6144000 bytes

256 Mbps observations

Using two hard disks to record at 256 Mbps seems to work pretty smoothly, keeping perfectly track of the recording rate. Also the ring buffer is filled up to lower levels, so it does not seem to be a risk to do long observations. The tests have been done using two Hard Disks PATA with 8 MB of internal buffer and capacity of 500 GB, so the total space available is 1 TB. So around 9 hours of continuos recording.

gofrito@juliano:/usr/src/vsib$ sudo ./wr 16000 480000 0 0 0 0 1 /media/usbdisk/t%03d 80000 /dev/vsib
Took 240.906931 seconds, 255.036249 Mbits(dec)/s (24.2% of PCI33), usleeps = 31166.

Special considerations could be the use of extended number of RAID. As example, our recording machine has 8 USB2 ports available, allowing up to 4 TB of memory to use for storage. It has not been yet tested, but the configuration could it be done, by using 4 different RAID's md0 to md4 and swapping to the next one in order once it is full. Or the second configuration claims to configure a big RAID using the eight disks as a single device of capacity 4 TB.


Conclusions

...