NetCDF toolbox for Octave
From OCG Test Wiki
NetCDF is a machine-independent file format for scientific data sets. Octcdf is a NetCDF toolbox for Octave which aims to be compatible with the matlab toolbox. The matlab NetCDF toolbox is a collection of matlab scripts written in an object-oriented approach which is not (yet) supported by octave. This octave toolbox is written in C calling directly the NetCDF library. With the octcdf toolbox, it is therefore possible to write m-files using NetCDF which can run in matlab and octave.
With octcdf, it is also possible to access a dataset from an OPeNDAP server (Open-source Project for a Network Data Access Protocol, formerly known as DODS). OPeNDAP allows you to download efficiently a subset of large datasets such as e.g. numerical model results. The data is imported directly in octave, regardless in which format the data is stored on the server.
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
Octcdf is also included in the octave-forge repository since release 2006.01.28. Some Linux distributions (e.g. Debian, Ubuntu, Fedora) already include octave-forge and you only need to install the octave-forge package in order to use octcdf.
Contents |
[edit] Requirements
- NetCDF libraries and headers (version 3.6.0 or later). Detailed installing instructions are available from the NetCDF site. I wrote a [get_netcdf.html really quick guide] for Linux and Windows/Cygwin.
- Octave and headers (version 2.9.7 or higher)
- gcc and g++ (the same version used to build octave)
Most Linux distributions provide pre-compiled packages of these dependencies. Note that if you use octave 2.9.9, you have to make sure that octave was compiled with perl-compatible regular expressions (pcre). You can check this by issuing the command octave_config_info('DEFS') in octave. You should see "-DHAVE_PCRE=1". This is not an issue for future versions of octave.
For optional OPeNDAP support you will need:
- DAP SDK (version 3.5.2 or higher)
- OPeNDAP NetCDF Client Library (version 3.5.2 or higher)
[edit] Installation
[edit] Installation with octave's package manager (recommended)
- Download the toolbox from http://ocgmod1.marine.usf.edu/octcdf/octcdf-1.0.7.tar.gz. Do not decompress the tar file.
- In octave, issue the following command:
pkg install /absolute/path/to/octcdf-1.0.7.tar.gz
- Add "pkg load all" in $HOME/.octaverc (create this file if necessary).
Per default, the NetCDF libraries are installed in /usr/local/lib and include files in /usr/local/include where octcdf will find them. If you have the NetCDF libraries (libnetcdf.a) and include files (netcdf.h) in a non-standard location, then you need to set the environment variables CPPFLAGS and LDFLAGS. For example, if you have installed NetCDF in /usr/local/netcdf (using ./configure --prefix /usr/local/netcdf), the following variables have to be defined before starting octave:
export CPPFLAGS=-I/usr/local/netcdf/include export LDFLAGS=-L/usr/local/netcdf/lib
The directory /usr/local/netcdf/lib should contain at least the file libnetcdf.a (or libnetcdf.so) and the directory /usr/local/netcdf/include should contain at least the file netcdf.h
[edit] Manual installation
If the previous approach does not work for you, you can install octcdf without the package manager:
tar zxvf octcdf-1.0.7.tar.gz cd octcdf-1.0.7/src ./configure make mkdir $HOME/octcdf-1.0.7 cp *.oct ../inst/*.m ../PKG_ADD $HOME/octcdf-1.0.7
You can choose any directory instead of "$HOME/octcdf-1.0.7". You need to tell octave about octcdf by using addpath:
octave:1> addpath ~/octcdf-1.0.7
You can put the addpath line in $HOME/.octaverc if you want to load octcdf when octave starts.
[edit] Testing the toolbox
If you have octave-forge installed, you can test the toolbox by typing "nctest" in octave. All tests should succeed.
octave:1> nctest writing test output to nctest.log >>>>> /home/johndoe/octave/octcdf-1.0.7/nctest.m PASSES 20 out of 20 tests
If some (or all) test fail, please send me (abarth at marine dot usf dot) the log file "nctest.log".
[edit] Using the toolbox
The toolbox uses an operator syntax to manipulate dimensions, variables and attributes. The toolbox provides the following functions: ncbyte, ncchar, ncclose, ncdouble, ncfloat, ncint, ncshort and netcdf. To learn more about these function type "help function_name" in octave.
The file example_netcdf.m illustrates a typical usage of the NetCDF toolbox.
% This is an example for creating a netcdf file with
% octcdf, a netcdf toolbox for octave
% create some variables to store them in a netcdf file
latitude = -90:1:90;
longitude = -179:1:180;
[y,x] = meshgrid(pi/180 * latitude,pi/180 * longitude);
temp = cos(2*x) .* cos(y);
%---------------------------------------%
% %
% write data to a netcdf file %
% %
%---------------------------------------%
% create netcdf file called example.nc
nc = netcdf('example.nc','c');
% define the dimension longitude and latitude of size
% 360 and 181 respectively.
nc('longitude') = 360;
nc('latitude') = 181;
% coordinate variable longitude
nc{'longitude'} = ncdouble('longitude'); % create a variable longitude of type double with
% 360 elements (dimension longitude).
nc{'longitude'}(:) = longitude; % store the octave variable longitude in the netcdf file
nc{'longitude'}.units = 'degree West'; % define a string attribute of the variable longitude
% coordinate variable latitude
nc{'latitude'} = ncdouble('latitude');; % create a variable latitude of type double with
% 181 elements (dimension latitude).
nc{'latitude'}(:) = latitude; % store the octave variable latitude in the netcdf file
nc{'latitude'}.units = 'degree North'; % define a string attribute of the variable latitude
% variable temp
nc{'temp'} = ncdouble('longitude','latitude'); % create a variable temp of type double of the size 360x181
% (dimension longitude and latitude).
nc{'temp'}(:) = temp; % store the octave variable temp in the netcdf file
nc{'temp'}.units = 'degree Celsius'; % define a string attribute of the variable
nc{'temp'}.valid_range = [-10 40]; % define a vector of doubles attribute of the variable
nc.history = 'netcdf file created by example_netcdf.m in octave';
% define a global string attribute
ncclose(nc) % close netcdf file and all changes are written to disk
disp(['example.nc file created. You might now inspect this file with the shell command "ncdump -h example.nc"']);
%---------------------------------------%
% %
% read data from a netcdf file %
% %
%---------------------------------------%
nc = netcdf('example.nc','r'); % open netcdf file example.nc in read-only
n = nc('longitude'); % get the length of the dimension longitude
temp = nc{'temp'}(:); % retrieve the netcdf variable temp
temp_units = nc{'temp'}.units; % retrieve the attribute units of variable temp
temp_valid_range = nc{'temp'}.valid_range; % retrieve the attribute valid_range of variable temp
global_history = nc.history; % retrieve the global attribute history
[edit] OPeNDAP
The file example_opendap.m shows how to use octcdf with a OPeNDAP server. Instead of using a NetCDF file name, you use a OPeNDAP URL. The following code is a minimal example showing how to download a subset of a variable from a OPeNDAP server.
% open a opendap URL
nc = netcdf('http://asterix.rsmas.miami.edu/thredds/dodsC/atl-ops-forecast/temp','r');
% download a subset of the variable temp
temp = nc{'temp'}(end,1,661:996,77:588);
% download the attribute "missing_value" and replace every
% occurrences by NaN in our local copy
temp(temp == nc{'temp'}.missing_value) = NaN;
% close the connection
ncclose(nc);
[edit] Tested platforms
On the following platforms, octcdf is known to work:
- Ubuntu 6.06, Intel x86
- Ubuntu 7.04 and Fedora Core 3-5 on AMD 64
- Fedora Core 4, Intel x86
- SGI Altix, Itanium 2
- Cygwin, Windows XP SP2. You will need to install the following cygwin packages:
- NetCDF 3.6.1 or later
- gcc-core version 3.3.3
- gcc-c version 3.3.3
- gcc-g77 version 3.3.3
- octave-headers
Note: you have to use the cygwin's old 3.3.3 gcc version and not the default 3.4.4 (see below). You can download the NetCDF library form http://www.unidata.ucar.edu/software/netcdf/. Other packages can be install with cygwin's setup.exe.
[edit] Known issues
- Due to a bug in gcc 3.4.4, octcdf need to be compiled with gcc 3.3.3. Binaries of this version can be downloaded using the cygwin's setup program. (Status: December 2, 2005).
- The package manager returns the following error:
error: dotexceptnewline: regexp not implemented in this version
This is a known problem in octave's package manager. The package manager requires perl compatible regular expressions (pcre), but octave was compiled without this library. This library is available on most Linux/UNIX systems (called libpcre3-dev in Ubuntu and pcre-devel in Fedora). Ideally you install this library and recompile octave.
- On Mac OS X, you see the following error:
tar: octcdf-1.0.7.tar.gz: Cannot open: No such file or directory
However, octcdf-1.0.7.tar.gz is in the working directory. This is a known problem of octave 2.9.9 under Mac OS X. As a work-around, you can install octcdf by specifying the absolute path.
octave:1> pkg install /absolute/path/to/octcdf-1.0.7.tar.gz
- on Linux with AMD 64 bit you are seeing:
`a local symbol' can not be used when making a shared object; recompile with -fPIC /usr/lib/gcc/x86_64-linux-gnu/3.4.6/../../../../lib/libnetcdf.a: could not read symbols: Bad value
well, this means that your have to recompile NetCDF with the option -fPIC:
./configure --with-pic
[edit] Report Bugs
If you want to write me a bug report, please include the following information:
- Name and version of your operating system. For Linux, please specify the distribution (for example Windows XP or Linux Fedora Core 6). If you use cygwin, please sent me also the list of installed packages by running the command:
cygcheck -c > all_packages.txt
and sent me the file "all_packages.txt".
- Name of your architecture (for example Intel Celeron or AMD Opteron)
- Version of octave and from where you got it (for example octave 2.9.13 from http://www.gnu.org/software/octave/download.html or octave 2.1.73 from the cygwin repository)
- Version of octcdf and from where you got it (for example octcdf 1.0.7 from http://ocgmod1.marine.usf.edu/octcdf/ or octcdf 1.0.7 from Fedora Core's octave-forge package)
- Version of your NetCDF library
- The file nctest.log created by nctest in octave
[edit] Comments and feedback
Any suggestions, comments, bug fixes, ... are of course very appreciated. Send them to: abarth at marine dot usf dot edu.
Alexander Barth