I’ll be at the AGU Fall Meeting this year presenting work about SatelliteDL:

SatelliteDL is an IDL toolkit for the analysis of satellite Earth observations from a diverse set of platforms and sensors. The core function of the toolkit is the spatial and temporal alignment of satellite swath and geostationary data.

It’s in the Informatics session on Tuesday afternoon, IN23D-3748. Check out the poster!

The biggest change introduced by IDL 8.4 is the treatment of all variables as objects. Every variable now has attributes that you would normally get returned from the SIZE function: length, ndim, dim, tname, typecode, and typename. For example:

IDL> a = bindgen(2, 3)
IDL> print, a.length
           6
IDL> print, a.ndim
           2
IDL> print, a.dim
           2           3
IDL> print, a.tname
BYTE

There are also static methods available for all variables:

IDL> n = 50
IDL> print, n->toString()
50

Strings, numbers, integers, and pointers have their own set of special methods appropriate for their respective types. For example integers have some handle base conversion methods:

IDL> print, n->toHex()
32
IDL> print, n->toBinary()
110010

Strings have some methods that are not available through other IDL library routines, including the very useful replace method:

IDL> s = 'ITT VIS'
IDL> print, s.replace('ITT', 'Exelis')
Exelis VIS

IDL 8.4 adds a new class FolderWatch to watch a directory for changes to its files:

IDL> fw = folderwatch('.', lambdap(fw, info: print, info.file), /added)

Then if you drop a file into the current directory, for example test.txt, you should see:

IDL>
test.txt

The callback procedure, which I passed a lambda procedure1 to in my example, takes two arguments: the FolderWatch object and an info structure with definition:

{ IDLFolderWatchInfo, file: '', added: 0B, modified: 0B, removed: 0B }

The object also has a USER_DATA property which allows you to store information that you might need in the callback routine.


  1. I think I’m going to like lambda routines. 

The accepted test for determining if the user passed in a value for an argument, say arg1, has been:

if (n_elements(arg1) eq 0) then ...

But this can be thwarted since IDL 8.0, which introduced operating overloading, by an object that overloads ::_overloadSize and returns 0.

Brian Griglak in the current IDL Data Point article concludes:

So putting it all together, the best way to validate undefined vs null vs non-null is:

if (Size(val, /TYPE) eq 0) then begin
  ...
endif else if (ISA(val, /NULL)) then begin
  ...
endif begin  ; I added this line and the next to Brian's code to
  ...        ; distinguish between his three cases
endelse

This is a tricky issue to get right for your particular purposes because N_ELEMENTS being 0 has multiple meanings. In my case, I had two cases that I needed to handle:

  1. the number of elements in an array of objects
  2. determine if an argument was present (return nonzero value if it was an object with _overloadSize returning 0)

These issues have always been muddled in IDL (ARG_PRESENT doesn’t tell you if an argument is present, it indicates that it was passed as a named variable).

My manner of testing was to use OBJ_VALID if the argument was an object and you didn’t want to use the normal operator overloading returned value:

function mg_n_elements, var, no_operatoroverload=noOperatoroverload
  compile_opt strictarr

  if (~keyword_set(noOperatoroverload) || size(var, /type) ne 11) then begin
    return, n_elements(var)
  endif

  return, n_elements(obj_valid(var))
end

Download src.

IDL 8.4 introduces arbitrarily large integers with the BigInteger class:

IDL> n = BigInteger(2)^400
IDL> help, n
N               BIGINTEGER <ID=3 LENGTH=401 bits> = 2.582249878086...x10^120
IDL> print, n
2582249878086908589655919172003011874329705792829223512830659356540647622016
841194629645353280137831435903171972747493376

Standard arithmetic with operators such as +, *, etc. works fine and there are a few methods to perform some other common operations:

IDL> print, n->nextPrime()
2582249878086908589655919172003011874329705792829223512830659356540647622016
841194629645353280137831435903171972747493557
IDL> print, n->log2()
                   400

Wow, I wish I had decided to stop giving free updates before IDL 8.4 instead of before IDL 9.0 — there is a lot of new material in Modern IDL 1.4! There are new sections about treating all IDL variables as objects and functional programming, as well as the many minor updates of new routines. If you have purchased Modern IDL by PDF in the past, but didn’t receive an email this morning with a link to your new version of Modern IDL, please let me know.

Note that I am still planning to continue the free updates until IDL 9.0. At that point, I will have to decide based on what’s new in IDL 9.0 how it will continue (but it won’t be a free update).

mglib 1.1.1, my library of IDL routines, was released last month. There are binary packages for OS X and 64-bit Linux on the download page.

My library contains routines in the following areas:

  • visualizations
  • file formats (netCDF, HDF 4 and 5, GRIB, save files, XML)
  • collections (list, hash, etc. classes)
  • routines for distributing IDL applications
  • networking
  • string handling
  • calendar routines
  • many other miscellaneous routines

IDL 8.4 also introduces a new boolean “type”, which is actually not a real type, but just a metadata flag on byte variables1. This boolean flag allows for better understanding of the purpose of the variable. For example, JSON_SERIALIZE can convert boolean IDL variables to boolean JSON declarations.

Create boolean variables by converting existing variables with the BOOLEAN function or by creating arrays with BOOLARR. The rules used to convert variables to boolean are not the normal truth values in IDL2; they are the rules used with the logical_predicate compile option is set, i.e., null, empty, or 0 values are false, everything else is true:

IDL> print, boolean(indgen(5))
   0   1   1   1   1
IDL> print, boolean(['', 'yes', 'no'])
   0   1   1

There are also convenient new !true and !false boolean system variables:

IDL> help, !true
<Expression>    BOOLEAN   = true (1)
IDL> help, !false
<Expression>    BOOLEAN   = false (0)

  1. For those familiar with IDL’s internal API, IDL 8.4 introduces a IDL_V_BOOLEAN flag that is used to mark the flags field of an IDL_VARIABLE structure. There is also an IDL_BOOLEAN macro to determine if a variable is a boolean. The macro requires that the IDL_V_BOOLEAN flag is set and that the variable is of type byte. 

  2. Thank goodness. The “normal” IDL rules for the truth value of integers depend on the lowest order bit of the integer, i.e., whether the integer is even or odd:

    IDL> if 1 then print, 'true' else print, 'false'
    true
    IDL> if 2 then print, 'true' else print, 'false'
    false
    IDL> if 3 then print, 'true' else print, 'false'
    true
    

Beau Legeer gives a great screencast of some of the new features in IDL 8.4. This is a quick way to see some of the new features in action.

Basics

IDL adds some functional programming syntax in 8.4:

  • lambda functions/procedures (inline routines)
  • filter, map, and reduce methods

Why is functional programming so good? With side-effect free functions, the order of computation is not important, making it easier to distribute computations over multiple cores. MapReduce is an increasingly popular programming model for dealing with large data in a distributed environment which uses this idea. While these additions to IDL don’t make any of this directly possible yet, they do provide some of the necessary background. And, of source, some just are happier in a functional programming environment.

Continue reading “IDL 8.4: functional programming.”

older posts »