I wrote an overview almost eight years ago about IDL’s basic routines to visualize a vector field. Not much has changed since then. This paper by Laidlaw, et al, mentioned at the end of my overview, outlines six common visualization techniques and evaluates them on the speed and accuracy of doing common tasks with them.

I think it would be great to have two of the better performing techniques: OSTR (image-guided streamlines, integral curves) and LIC (line-integral convolution) techniques. Between the two of them, they have the best or nearly the best performance in every category. An example OSTR visualization is shown below:

OSTR example

I have tackled LIC and have code to produce images from vector fields in my library.

Public voting is open for the 2015 NSF Vizzies, the visualization competition with categories for photos, illustrations, posters/graphics, interactive visualizations, and videos. Check them out and vote for your favorites — I find them very motivating!

Antarctica: a chromatic paradox

While interactive graphics can be created in IDL, you are currently limited to distributing an IDL application to provide them to others. It would be useful to be able to create interactive web graphics from IDL — particularly if it was just a click or ->save call from function graphics.

Bokeh is a library for Python capable of generating Javascript to display interactive visualizations. Check out its gallery for examples. Bokeh has bindings for other languages; all that is required for the language is to output the correct JSON to the Javascript part of Bokeh. Bokeh is BSD licensed so it could be used in a commercial product like IDL.

Of course, Bokeh can be used via the Python bridge right now, but that adds an additional burden to its use.

NASA’s Solar Dynamics Observatory (SDO) is capturing spectacular images of the sun in ten different wavelengths. NASA Goddard has produced this amazing HD video of the images. There is a lot going on:

It’s always shining, always ablaze with light and energy that drive weather, biology and more. In addition to keeping life alive on Earth, the sun also sends out a constant flow of particles called the solar wind, and it occasionally erupts with giant clouds of solar material, called coronal mass ejections, or explosions of X-rays called solar flares. These events can rattle our space environment out to the very edges of our solar system.

I could just sit and watch this — it would make a fantastic screen saver.

via FlowingData

IDL already contains syntax for passing arbitrary (and perhaps unknown beforehand) keywords to a routine using the _EXTRA and _REF_EXTRA keywords, depending on whether you are passing a variable back to the caller through the keyword. For example, you can do something like this:

pro my_wrapper_routine, _extra=e
  my_routine, _extra=e

Now, MY_WRAPPER_ROUTINE passes all the keywords that MY_ROUTINE accepts along to it (ignoring any others, use _STRICT_EXTRA on the MY_ROUTINE call to generate an error if an unknown keyword is passed in).

It would be useful to have a similar mechanism for positional parameters, making it possible to write routines in IDL which accept an arbitrary number of keywords or pass along parameters to another routine. For example, passing all parameters after the first two to another routine and printing the first “extra” parameter with the new library routine GET_PARAM:

pro my_wrapper_routine, a, b, _extra
  my_routine, _extra
  print, get_param(_extra, 0)

What exactly would the _extra variable be? I’m not sure. Maybe it isn’t even needed in the GET_PARAM call, but it certainly is needed in the MY_ROUTINE call. Python handles this through tuples and the special * notation, maybe IDL can use structures and a bit of new notation?

But I would certainly like to get rid of code like the following:

case n_params() of
  0: _sql_query = ''
  1: _sql_query = sql_query
  2: _sql_query = string(arg1, format='(%"' + sql_query + '")')
  3: _sql_query = string(arg1, arg2, format='(%"' + sql_query + '")')
  4: _sql_query = string(arg1, arg2, arg3, format='(%"' + sql_query + '")')

IDL has several places where callback functions are used to provide a mechanism for IDL library routines to call user-defined functions. For example, in curve fitting routines such as CURVEFIT the callback function allows the user to define custom functions that are evaluated by the fitting routine.

For many callbacks, this interface is perfectly adequate. But for some cases, the callback routine requires some data that is not passed to it in the interface defined by the calling routine. In this case, it is typical for the callback routine to use a technique like common blocks to access the data. But even Fortran recognizes the problems with common blocks; surely there must be a better way to do this in 2015.

IDL 8.5 provides just such a mechanism — function pointers. Objects which inherit from IDL_Object and define an _overloadFunction method can be called like a function. This would allow passing a function pointer with the appropriate user-define data as a callback function. A reference to object could be kept and the data could be modified as needed.

This would provide a much nicer interface in a wide variety of applications which use a callback routine: numeric routines such as for curve fitting; several areas in widget programming such as XMANAGER itself, creating a compound widget, and WIDGET_TREE drag and drop; and some networking classes such as IDLnetURL and IDLnetOGCW[CM]S.

The change should not be difficult, just a check for a string versus and object passed as the callback. Something like:

function mg_call_callback, callback, arg1, arg2, _extra=e compile_opt stricter

if (size(callback, /type) eq 11) then begin
  return, callback(arg1, arg2, _extra=e)
endif else begin
  return, call_function(callback, arg1, arg2, _extra=e)


Of course, this is complicated by the fact that the above routine assumes that there are two positional parameters (though it can handle any keywords), but I will save that for another wish.

Fun site for map nerds or anyone trying to get a grasp on the differences between projections.

via Visualizing Data

Hans Rosling discusses the news media view of the world versus one based on data.

I agree. The media’s need for interesting and entertaining news means that stories such as “It’s a Pretty Safe Time for Most” or “Things are Slightly Better Than Last Year” are not written or at least not emphasized. We should build context through data.

Link via FlowingData.

The ternary operator is a handy syntax for creating a result based on a (scalar) condition that is evaluated for truth. For example, I frequently use it to set a default value for an input variable, like the following that uses a default value of 5.0 if x is not present:

_x = n_elements(x) eq 0L ? 5.0 : x

Without the syntax, you would need an if statement:

if (n_elements(x) eq 0L) then _x = 5.0 else _x = x

(Or, in this case, a special purpose routine like default.) But, besides being a bit longer, the if statement hides the possibility that you might not assign to _x in both cases of the if statement, whereas it is clear in the ternary statement that _x is going to receive a value in any case.

The ternary statement is even more useful because it can be chained with other statements. For example,

results[f] = day + (year eq currentYear $
                      ? string(time, format='(A6)') $
                      : string(year, format='(I6)'))

Without the ternary operator, this would require a new local variable and would be twice as long:

if (year eq currentYear) then begin
  yeartime = string(time, format='(A6)')
endif else begin
  yeartime = string(year, format='(I6)')
result[f] = day + yeartime

One of the main limitations to the ternary operator is that, like an if statement, the condition must be evaluated to a scalar truth value. I would like to extend the ternary operator to arrays. Let’s see how it would work.

First, let’s create some arrays to use in the examples:

IDL> x = findgen(5)
IDL> y = 2 * findgen(5)
IDL> z = lindgen(5) mod 2
IDL> print, x
      0.00000      1.00000      2.00000      3.00000      4.00000
IDL> print, y
      0.00000      2.00000      4.00000      6.00000      8.00000
IDL> print, z
           0           1           0           1           0

Then, my proposal would be for the following lines to work:

IDL> print, z ? x : y
      0.00000      1.00000      4.00000      3.00000      8.00000

One, or both, of the operators could be a scalar:

IDL> print, z ? 1.0 : y
      0.00000      1.00000      4.00000      1.00000      8.00000
IDL> print, z ? 1.0 : 2.0
      2.00000      1.00000      2.00000      1.00000      2.00000

Now, of course, this can be done already, but it takes a lot more code. For example, the first case expands from one line to:

IDL> result = fltarr(5)
IDL> ind = where(z, count, complement=not_ind, ncomplement=not_count)
IDL> if (count gt 0L) then result[ind] = x[ind]
IDL> if (not_count gt 0L) then result[not_ind] = y[not_ind]
IDL> print, result
      0.00000      1.00000      4.00000      3.00000      8.00000

IDL already has rules for type promotion if an operation has operands of two differing types; those could be used here to determine the type of the result since the elements of both value operands would be intermixed in a single array.

I linked to a great post on FlowingData a few weeks ago about some basic rules for visualizations. Nathan then gave a more detailed treatment of his first rule:

I already covered the small handful of rules that pertain mostly to traditional statistical graphics. The first one–to always start your bar charts with a zero baseline–unexpectedly drew some disagreement, and I am unexpectedly compelled to go into more depth.

Great alternatives to a non-zero baseline.

older posts »