next up previous contents
Next: Data Monitoring Up: Radio Astronomical Instrumentation Previous: Computing Infrastructure   Contents

Data Collecting and Archival

Project Team: Mujunen, Engelberg

Metsähovi main NFS server ``/home'' continues to be the same 32 GB four-disk RAID0 disk array which was installed in 1998. The hard disk array enclosure has two sets of four 9GB hard disks, ``/home'' and ``/backup''. All user files can thus be backed up ``on-line'' using the Linux ``mirrordir'' command in less than an hour. This process was automated by using a ``cron'' job to execute mirroring daily at 18:30 local time.

This division of available hard disks into ``backup'' and ``hot'' disks proved its usefulness in 2000 when one of the 9GB disks failed. The failure was in ``backup'' set, so it was enough to shut down the automatic backup procedure while waiting for the replacement disk. If the failure had been in the ``hot'' set, ``/backup'' would have been re-mounted as ``/home'' for immediate backup data availability.

Backing up ``/backup'' (instead of ``/home'' directly) on DAT makes tape backups more consistent since ``/backup'' does not change during the DAT-``tar'' procedure. It also provides a snapshot of yesterday's files which allows us to recover from, for instance, accidental user file deletions and similar mishaps.

The necessary infrastructure for multiple concurrent software developers in the form of ``/home/cvs'', a common CVS repository, has been found extremely useful. CVS (Concurrent Versions System) keeps track of the evolution of source code and documentation files and allows multiple developers have private ``sandbox'' development copies of the same set of files. Later changes can be merged back to the common repository version of each file. CVS conventions at Metsähovi are documented in a laboratory report ``Using CVS at Metsähovi''.

The additional ``data warehouse'' NFS server ``/data'' created on ``data.kurp.hut.fi'' continues its full operation. The server uses a similar Linux RAID0 setup as what was used for ``/home''. Two 9GB Ultra-Wide SCSI hard disks were combined for 18GB of total ``/data'' disk space and another two disks make up on-line ``/backup''.

``/data'' is reserved for continuous log and data file storage for different components of Metsähovi observation system (see section 2.2). Environmental and auxiliary measurement information is collected by background daemons and archived in ASCII files, one file per each UTC day (``yyyymmdd.log'').

Hydrogen maser conditions were monitored with a Linux software daemon ``efos9d''. The latest version of the software measures 34 monitoring channels and the maser frequency value and saves them locally and afterwards these daily files are being moved to the main data archive located in ``data.kurp.hut.fi''.

Weather station software and RFI spectral monitoring software continue their autonomous operations as is explained in their respective sections.


next up previous contents
Next: Data Monitoring Up: Radio Astronomical Instrumentation Previous: Computing Infrastructure   Contents
Ari Mujunen 2001-07-30