Software spectrometer setup instructions and test data
Background
Swspectrometer calculates precise integrated auto and cross-correlation spectra for several types of data, and optionally detects and extracts a phase cal signal from the data. It supports an arbitrary number of DFT/FFT points (1.280.000 points or more), allows arbitrary integration time and windowed-overlap with several window functions. The software is heavily SIMD vectorized, threaded and I/O-buffered for fast performance. It is intended to be run on a single multi-core computer. Cluster support may be added later. The software is "free" as per GNU GPL and is distributed as part of the cellspe-tasklib but yes it works on Intel as well. An advanced non-free version 'sctracker' for spacecraft tracking exists with similar but more functionality.
Platforms
The software runs in PlayStation3 (FFTs with 4K points or less) and on Intel/AMD systems (DFT, FFT of arbitrary size). To compile on a PS3 with Linux you need IBM Cell SDK2.0/2.1. To compile on Intel with Linux or Windows you need the Intel IPP library. For Linux you can get a free IPP license from Intel Non-Commercial Software Download.
Compiling
You need these Ubuntu or equivalent packages:
pgplot5 libpng3-dev libx11-dev cl-plplot
The Intel IPP setup requires a few extra steps. After installing, set an environment variable IPP_PATH (~/.bashrc or /etc/bash.bashrc) to point to the Intel IPP SDK root directory, for example /opt/intel/ipp/5.3.1.062/em64t/. Next you have to update ldconfig so it can find the IPP libraries:
oper@abidal:/etc/ld.so.conf.d$ sudo pico intel_ipp.conf /opt/intel/ipp/5.3.1.062/em64t/sharedlib/ /opt/intel/ipp/5.3.1.062/em64t/lib/ oper@abidal:/etc/ld.so.conf.d$ sudo ldconfig
You can get swspectrometer as source code from the CVS:
$ cvs -z3 \ -d:pserver:anonymous@cellspe-tasklib.cvs.sourceforge.net:/cvsroot/cellspe-tasklib \ co -P cellspe-tasklib $ cd cellspe-tasklib/spectrometer/src/ $ make
Test files
Data from W75 maser: W75n (180 MB)
Democonfig file: democonfig.ini
Short scans from VEX 1107: v1107 dir
Example Octave/Matlab for importing result data: v1107Plot.m
Data format
The data input format is a bit tricky. Currently TaskCore*.cpp contains unpacking code for certain data formats (iBob 8-bit, Maxim 12-bit, 2-bit 4-channel VLBA) and the formats are selected based only on bits-per-sample and number-of-channels. Unpacking is very fast. Some specific Mark4/VLBA formats are supported. An interface to a Mark5/VLBA decoder library will be added later -- unfortunately these decoder libraries tend to be very slow and inefficient, so it may be better to implement your own vectorized or unrolled unpack routine for the specific data format that you use.
The data output format depends on how you compiled swspectrometer. The default compile results in binary, single-precision 32-bit/4-byte float output data. The cross-spectrum is complex with N (re,im) value pairs i.e. the file contains [Re0 Im0 Re1 Im1 ... ReN-1 ImN-1]. The power spectra are real-valued with N/2+1 points, the first point is DC and the last point is Nyquist i.e. [Re0 Re1 Re2 Re3 .. ReN/2] where Re0 is point 1 (DC) and ReN/2 is point N/2+1 Nyquist.
The output can be a file (typical use) or PGPlot for plotting (without saving). There are provisions to support more inputs such as Metsahovi VSIB and real-time iBob.
Spectrum data can be read into Matlab or Octave with for example realdata = fread(fd, [(Nfft/2+1) 1], 'float32'); for real-valued data. Similarly for complex-valued cross-spectrum data, except that now but you have to combine complexdata = data(even) + i*data(odd); since Matlab/Octave do not appear to support a native 'complex32' type.
Phase cal results, when enabled, are currently output to the screen as text.
Running v1.02
Once that the spectrometer software, two binary files appear in the src directory. You can run it by typing ./swspectrometer or ./intel_swspectrometer. Both commands require to add a few extra settings. See example below.
oper@juliano:/usr/src/cellspe-tasklib/spectrometer/src$ ./intel_swspectrometer IBM Cell/Intel Software Spectrometer v1.02 Usage: ./swpsectrometer [--option=x] in_uri1 [in_uri2] out_uri1 [out_uri2] [out_xpol] --cores=x - how many cores to use --fftpts=x - how many points per FFT (a 2^N value) --fftint=x - how many FFTs to integrate --bits=x - how many bits per sample --channels=x - how many channels from each resource --usechan=x - which of the channels to process in_uri1 - input data resource for source 1 in_uri2 - input data resource for optional source 2 out_uri1 - output data resource for source 1 out_uri2 - output data resource for optional source 2 out_xpol - output data resource for optional source 1-vs-2 cross-polarization
The follow example summarize how to use it:
$ ./swspectrometer --usechan=1 --cores=1 --fftpts=65536 --fftint=2560 --bits=2 --channels=4 \ W75n_Mh_N0000.evn w75n_Mh_N0000.asc
Running v1.03
The newest version of spectrometer includes iniFile.ini (see example democonfig where all the parameters of the FFT and data type are declared. Following is the usage help for the newer version.
oper@juliano:/usr/src/cellspe-tasklib/spectrometer/src$ ./intel_swspectrometer IBM Cell/Intel Software Spectrometer v1.03 Build date 20081017 number 2 Usage: ./swpsectrometer inifile.ini ch1inUri ch1outUri [in_uri2 out_uri2 [out_xpol]] inifile.ini : file and path to an INI file with run settings ch1inUri : input data resource for channel 1 data ch1outUri : output for the channel 1 intergrated spectrum ch2inUri : optional input data resource for channel 2 data ch2outUri : output for the channel 2 intergrated spectrum out_xpol : output for channel 1x2 cross-polarization spectrum
And the usage with say democonfig.ini is
$ ./swpsectrometer democonfig.ini W75n_Mh_N0000.evn W75n_Mh_N000.bin $ ./swpsectrometer democonfig.ini v1107_Mh_No0003_short.evn v1107_no3.bin
The expected result from the demo data sets is below (TODO...)