The biggest change introduced by IDL 8.4 is the treatment of all variables as objects. Every variable now has attributes that you would normally get returned from the SIZE function: length, ndim, dim, tname, typecode, and typename. For example:

IDL> a = bindgen(2, 3)
IDL> print, a.length
           6
IDL> print, a.ndim
           2
IDL> print, a.dim
           2           3
IDL> print, a.tname
BYTE

There are also static methods available for all variables:

IDL> n = 50
IDL> print, n->toString()
50

Strings, numbers, integers, and pointers have their own set of special methods appropriate for their respective types. For example integers have some handle base conversion methods:

IDL> print, n->toHex()
32
IDL> print, n->toBinary()
110010

Strings have some methods that are not available through other IDL library routines, including the very useful replace method:

IDL> s = 'ITT VIS'
IDL> print, s.replace('ITT', 'Exelis')
Exelis VIS

http://www.exelisvis.com/Company/PressRoom/Blogs/IDLDataPointDetail/TabId/902/ArtMID/2926/ArticleID/14135/IDL-84—Sliced-bread-step-aside.aspx

IDL 8.4 adds a new class FolderWatch to watch a directory for changes to its files:

IDL> fw = folderwatch('.', lambdap(fw, info: print, info.file), /added)

Then if you drop a file into the current directory, for example test.txt, you should see:

IDL>
test.txt

The callback procedure, which I passed a lambda procedure1 to in my example, takes two arguments: the FolderWatch object and an info structure with definition:

{ IDLFolderWatchInfo, file: '', added: 0B, modified: 0B, removed: 0B }

The object also has a USER_DATA property which allows you to store information that you might need in the callback routine.


  1. I think I’m going to like lambda routines. 

The accepted test for determining if the user passed in a value for an argument, say arg1, has been:

if (n_elements(arg1) eq 0) then ...

But this can be thwarted since IDL 8.0, which introduced operating overloading, by an object that overloads ::_overloadSize and returns 0.

Brian Griglak in the current IDL Data Point article concludes:

So putting it all together, the best way to validate undefined vs null vs non-null is:

if (Size(val, /TYPE) eq 0) then begin
  ...
endif else if (ISA(val, /NULL)) then begin
  ...
endif begin  ; I added this line and the next to Brian's code to
  ...        ; distinguish between his three cases
endelse

This is a tricky issue to get right for your particular purposes because N_ELEMENTS being 0 has multiple meanings. In my case, I had two cases that I needed to handle:

  1. the number of elements in an array of objects
  2. determine if an argument was present (return nonzero value if it was an object with _overloadSize returning 0)

These issues have always been muddled in IDL (ARG_PRESENT doesn’t tell you if an argument is present, it indicates that it was passed as a named variable).

My manner of testing was to use OBJ_VALID if the argument was an object and you didn’t want to use the normal operator overloading returned value:

function mg_n_elements, var, no_operatoroverload=noOperatoroverload
  compile_opt strictarr

  if (~keyword_set(noOperatoroverload) || size(var, /type) ne 11) then begin
    return, n_elements(var)
  endif

  return, n_elements(obj_valid(var))
end

Download src.

IDL 8.4 introduces arbitrarily large integers with the BigInteger class:

IDL> n = BigInteger(2)^400
IDL> help, n
N               BIGINTEGER <ID=3 LENGTH=401 bits> = 2.582249878086...x10^120
IDL> print, n
2582249878086908589655919172003011874329705792829223512830659356540647622016
841194629645353280137831435903171972747493376

Standard arithmetic with operators such as +, *, etc. works fine and there are a few methods to perform some other common operations:

IDL> print, n->nextPrime()
2582249878086908589655919172003011874329705792829223512830659356540647622016
841194629645353280137831435903171972747493557
IDL> print, n->log2()
                   400

Wow, I wish I had decided to stop giving free updates before IDL 8.4 instead of before IDL 9.0 — there is a lot of new material in Modern IDL 1.4! There are new sections about treating all IDL variables as objects and functional programming, as well as the many minor updates of new routines. If you have purchased Modern IDL by PDF in the past, but didn’t receive an email this morning with a link to your new version of Modern IDL, please let me know.

Note that I am still planning to continue the free updates until IDL 9.0. At that point, I will have to decide based on what’s new in IDL 9.0 how it will continue (but it won’t be a free update).

mglib 1.1.1, my library of IDL routines, was released last month. There are binary packages for OS X and 64-bit Linux on the download page.

My library contains routines in the following areas:

  • visualizations
  • file formats (netCDF, HDF 4 and 5, GRIB, save files, XML)
  • collections (list, hash, etc. classes)
  • routines for distributing IDL applications
  • networking
  • string handling
  • calendar routines
  • many other miscellaneous routines

IDL 8.4 also introduces a new boolean “type”, which is actually not a real type, but just a metadata flag on byte variables1. This boolean flag allows for better understanding of the purpose of the variable. For example, JSON_SERIALIZE can convert boolean IDL variables to boolean JSON declarations.

Create boolean variables by converting existing variables with the BOOLEAN function or by creating arrays with BOOLARR. The rules used to convert variables to boolean are not the normal truth values in IDL2; they are the rules used with the logical_predicate compile option is set, i.e., null, empty, or 0 values are false, everything else is true:

IDL> print, boolean(indgen(5))
   0   1   1   1   1
IDL> print, boolean(['', 'yes', 'no'])
   0   1   1

There are also convenient new !true and !false boolean system variables:

IDL> help, !true
<Expression>    BOOLEAN   = true (1)
IDL> help, !false
<Expression>    BOOLEAN   = false (0)

  1. For those familiar with IDL’s internal API, IDL 8.4 introduces a IDL_V_BOOLEAN flag that is used to mark the flags field of an IDL_VARIABLE structure. There is also an IDL_BOOLEAN macro to determine if a variable is a boolean. The macro requires that the IDL_V_BOOLEAN flag is set and that the variable is of type byte. 

  2. Thank goodness. The “normal” IDL rules for the truth value of integers depend on the lowest order bit of the integer, i.e., whether the integer is even or odd:

    IDL> if 1 then print, 'true' else print, 'false'
    true
    IDL> if 2 then print, 'true' else print, 'false'
    false
    IDL> if 3 then print, 'true' else print, 'false'
    true
    

Beau Legeer gives a great screencast of some of the new features in IDL 8.4. This is a quick way to see some of the new features in action.

Basics

IDL adds some functional programming syntax in 8.4:

  • lambda functions/procedures (inline routines)
  • filter, map, and reduce methods

Why is functional programming so good? With side-effect free functions, the order of computation is not important, making it easier to distribute computations over multiple cores. MapReduce is an increasingly popular programming model for dealing with large data in a distributed environment which uses this idea. While these additions to IDL don’t make any of this directly possible yet, they do provide some of the necessary background. And, of source, some just are happier in a functional programming environment.

Continue reading “IDL 8.4: functional programming.”

IDL 8.4 adds a new routine, CODE_COVERAGE (docs), which returns information about the lines of a routine that have been executed. Using CODE_COVERAGE is fairly straight-forward — you do not need to enable code coverage. Just call CODE_COVERAGE at any time to find the lines of a routine that have been executed. Note that the routine must have been at least compiled before you call CODE_COVERAGE (even if you are clearing the status of the routine). Also, pay particular definition of the definition of a “line of code” in the docs, e.g., empty lines, comments, and END statements do not count. Between the return value and the output from the EXECUTED keyword, you should get all the lines of code in a routine.

CODE_COVERAGE adds another useful developer tool to the timing routines like PROFILER1, TIC and TOC. I think CODE_COVERAGE has a range of uses, but most interesting for me is the ability to determine the coverage of your unit test suite, i.e., how much of my code base is executed by my test suite?

I have already implemented some basic test coverage information in my unit testing framework, mgunit. For example, mgunit can now tell me that I’m missing coverage of a few lines in the helper routines for MG_SUBS:

"mg_subs_ut" test case starting (5 tests)
  test_basic: passed (0.000223 seconds)
  test_derived: passed (0.000354 seconds)
  test_derived2: passed (0.000369 seconds)
  test_derived_perverse: passed (0.000477 seconds)
  test_not_found: passed (0.000222 seconds)
Test coverage: 90.5%
  Untested lines
    mg_subs_iter: lines 135
    mg_subs_getvalue: lines 72-73, 79
  Completely covered routines
    mg_subs
Results: 5 / 5 tests passed, 0 skipped

This means that after the unit tests have been run, line 135 from MG_SUBS_ITER and lines 72-73, 79 from MG_SUBS_GETVALUE have not been executed. This is useful (though not complete) information for determining if you have enough unit tests. Grab mgunit from the master branch on GitHub to give it a try (see mglib for an example of unit tests that take advantage of it). I’m not sure of the exact format for displaying the results, but I am fairly certain of the mechanism for telling the unit tests which routines it is testing (an ::addTestingRoutine method). I intend to start using this for the unit tests of my products GPULib and FastDL soon!


  1. There is also a CODE_COVERAGE keyword to PROFILER now that displays the number of lines of a routine that were executed. 

older posts »