A nice list of resources for doing remote sensing in Python, especially if you already know IDL.

The recent article about how to investigate object code got me thinking about the various methods I use to find out about an object/class.

The code in the article, for the example of an object of class IDLgrColorBar, prints:

IDL> ab_obj_info, 'idlgrcolorbar'
Class: IDLGRCOLORBAR
Superclass: IDLGRMODEL
Superclass: IDLGRCONTAINER
Superclass: IDL_CONTAINER
Superclass: IDLGRCOMPONENT
Superclass: IDLITCOMPONENT</p>

as well as some HTML information listing the methods of the objects.

One of the most useful techniques for me is one of my library routines to find the class hierarchy of an object:

IDL> mg_class_hierarchy, idlgrcolorbar()
IDLGRCOLORBAR
IDLGRMODEL
IDLGRCONTAINER
IDL_CONTAINER
IDLGRCOMPONENT
IDLITCOMPONENT

This gives the same top-level information with a bit more detail (IDLgrContainer inherits from both IDL_Container and IDLitComponent), but does not provide any listing of the methods. If you know the name of the method, you can use another of my library routines to find out about it’s arguments:

IDL> man, 'idlgrcolorbar::computedimensions'
Filename: /Applications/exelis/idl85/lib/idlgrcolorbar__define.pro
result = idlgrcolorbar::computedimensions(self, osrcdest, PATH=path)

So I added a METHODS keyword to print out the methods of each class:

IDL> mg_class_hierarchy, 'idlgrcolorbar', /methods
IDLGRCOLORBAR
result = idlgrcolorbar::computedimensions()
result = idlgrcolorbar::init()
idlgrcolorbar::calcsize
idlgrcolorbar::cleanup
idlgrcolorbar::getproperty
idlgrcolorbar::setproperty
IDLGRMODEL
result = idlgrmodel::getctm()
result = idlgrmodel::getxyzrange()
result = idlgrmodel::init()
idlgrmodel::add
...

IDLdoc produces these docs, which list the methods of IDLgrColorBar and the hierarchy of superclasses along with a lot of other information including comments that might be in the code headers, but not the methods inherited from those superclasses.

A security update for IDL and ENVI license servers was released today by Harris Geospatial Solutions:

The security vulnerability is limited to computers running a license manager server and should not be an issue when the license server components are only exposed on a trusted network.

If you run an outside facing IDL or ENVI license server, it sounds like you need to update immediately.

Python has the Zen of Python as a guiding philosophy. I think IDL would have something a bit more practical. This is my take on IDL’s philosophy:

The Tao of IDL

Interactive is better than compiled.
Fast to write is better than fast to execute.
But vectorized is better than loops;
`WHERE` is better than `FOR`/`IF`.
There are more uses of histograms than first meet the eye.

A picture is worth at least ten thousand bytes of data,
A million if its 3D and you can rotate it interactively.
Whether direct or function, graphics are easy to create
But the possibilities are endless.

Backwards compatibility is great!
But it doesn't mean you should index arrays with parentheses forever.
IDL might not have started with objects,
But it has them now, so use them!

There are many file formats and each is the most important to someone.
If you can't read the data, you can't analyze it.

Keywords are a great idea — especially
If your parameter has a useful default
Or is an optional output.

By the way, the Bad habits posting is very funny and relevant to Fortran/IDL users.

Noah Lorang argues that data scientists mostly just do arithmetic:

The dirty little secret of the ongoing “data science” boom is that most of what people talk about as being data science isn’t what businesses actually need. Businesses need accurate and actionable information to help them make decisions about how they spend their time and resources. There is a very small subset of business problems that are best solved by machine learning; most of them just need good data and an understanding of what it means that is best gained using simple methods.

This rings true for me in terms of the amount of time I spend doing various tasks during a typical day. I would estimate that 90-95% of my time is spent doing basic shell scripting, writing/modifying IDL code to do basic file and array manipulations, writing IDL GUI applications, etc. But the 5-10% of the other time is still important! The mundane tasks would be pointless without choosing the correct optimization technique and verifying it works. It might be that improving the performance of some section of code makes the difference between keeping up with incoming data or not and that might mean using some “hardcore” technique such as writing the section in C, using a GPU, or making use of some multicore technology.

via FlowingData

I’ve found this translation guide for writing and understanding Python code quite useful. I think it should work if you are familiar with Python and wanting to read/write IDL code also.

Here are ten little programs of ten lines or less to introduce new programmers to IDL. This post^[1] is motivated by this comp.lang.python thread which became this page on the Python wiki.

1 line: output, calling a procedure:

print, 'Hello, world!'

2 lines: assignment, calling a function, system variables, array operations, keywords:

x = findgen(360) * !dtor
plot, x, sin(x), xstyle=9, ystyle=8

3 lines: input, output format codes:

name = ''
read, 'What is your name? ', name
print, name, format='("Hello, ", A, "!")'

5 lines: filename handling, reading images, variable metadata, boolean keywords, displaying image:

filename = filepath('people.jpg', subdir=['examples', 'data'])
im = read_image(filename)
dim = size(im, /dimensions)
window, xsize=dim[1], ysize=dim[2], /free, title=file_basename(filename)
tv, im, true=1

6 lines: logical unit numbers, read binary data, contour plots, line continuation:

convec = bytarr(248, 248)
openr, lun, file_which('convec.dat'), /get_lun
readu, lun, convec
free_lun, lun
window, xsize=500, ysize=500, /free
contour, convec, xstyle=1, ystyle=1, $
         levels=(max(convec) - min(convec)) * findgen(10) / 9. + min(convec)

7 lines (contributed by Mark Piper): query image, image processing, automatic positioning of images:

file = file_which('moon_landing.png')
!null = query_image(file, info)
astronaut = read_image(file)
window, /free, xsize=3 * info.dimensions[0], ysize=info.dimensions[1]
tv, astronaut, 0
tvscl, sobel(astronaut), 1
tvscl, canny(astronaut), 2

8 lines: writing a function, compile_opt statement, if statements, for loops:

.compile
function mg_fibonacci, x
  compile_opt strictarr
  if (x eq 0) then return, 0
  if (x eq 1) then return, 1
  return, mg_fibonacci(x - 1) + mg_fibonacci(x - 2)
end
for i = 0L, 20L do print, i, mg_fibonacci(i)

9 lines (contributed by Mark Piper): array generation, FFTs, line plots, multiple plots/window, query for screen size:

x = (2.0 * !pi) * findgen(100) / 100
y = sin(3.0 * x) + cos(12.0 * x) + cos(25.2 * x)
magspec_y = abs(fft(y))
ss = get_screen_size()
window, /free, xsize=0.4 * ss[0], ysize=0.25 * ss[1]
!p.multi = [0,2,1]
plot, x, y, xtitle='sample number', ytitle='value', title='Series', xstyle=1
plot, magspec_y, xtitle='mode', ytitle='spectral density', $
      title='Magnitude Spectrum', xrange=[0, n_elements(x) / 2], xstyle=1<br />!p.multi = 0

10 lines: maps, read ASCII file, indexed color, structures:

header = strarr(5
data = replicate({loc:fltarr(2), elev:0, temp:0, dew:0, wspd:0, wdir:0}, 15)
openr, lun, file_which('ascii.txt'), /get_lun
readf, lun, header<br />readf, lun, data
free_lun, lun
device, decomposed=0
loadct, 5
map_set, limit=[min(data.loc[1, *], max=maxlat), $
          min(data.loc[0, *], max=maxlon), maxlat, maxlon], /mercator, /usa
          plots, data.loc[0, *], data.loc[1, *], psym=4, color=bytscl(data.temp), $
          symsize=2., thick=2

Finally, for the my last (for now) IDL wish list item: a new widget toolkit. This wish list item is for a native widget toolkit, not the ability to create interactive web pages, though that would be good too.

This widget toolkit would:

1. be supported on all platforms supported by IDL
2. have a clean, modern look
3. have all the capabilities of the current IDL widget toolkit
4. have an embeddable web browser window
5. have a richer set of features for existing widgets (tables, in particular)
6. be accessible through a consistent, object-oriented API

I think the main candidates currently are wxWidgets, Qt, and GTK. My experience with these toolkits has been with Qt. Potentially, this could be done by piggy backing on the PySide project which created several generic tools for generating bindings that could be used for IDL instead of Python.

All three of these toolkits are license under something close to LGPL. I think^[1] this should work for a commercial product like IDL since only the source code for the widget toolkit and its bindings would have to be provided since the library would be provided as a DLM and not part of the main IDL executable.

When learning IDL many years ago, the first thing that caused me to do a double take was the comma between a procedure name and its first argument when calling it and between either a function or procedure name and its first argument when declaring it. While removing this comma would not provide any noteworthy capability to my code, it would:

1. be one less keystroke per procedure call
2. eliminate approximately 25% of my syntax errors when writing in other languages
3. look a lot prettier
4. eliminate most of the shame I feel when showing non-IDL programmers my IDL code

If . can be used for the -> operator, the extra comma can be removed from IDL!

IDL has had keyword inheritance for a long time. The special keywords _EXTRA, _REF_EXTRA, and _STRICT_EXTRA are used to have a wrapper routine pass its keywords to some routine that it calls. It would be convenient to have corresponding inherited positional parameters, perhaps using the analogous specially named parameter _extra:

pro mg_print, _extra, _extra=e
  compile_opt strictarr
  print, _extra, _extra=e
end

This routine would accept any number of positional parameters/keywords and pass those that PRINT accepts along to it.