### Category "IDL"

I plot a lot of data on daily cycles, where there is no data collected at night. Let’s mock up some sample data with the following simple code:

IDL> x = [findgen(10), findgen(10) + 25, findgen(10) + 50]
IDL> seed = 0L
IDL> y = randomu(seed, 30)
IDL> plot, x, y


Then I get a plot like this:

This plot doesn’t show the nightly breaks in data well. Connecting the last data point collected from a day to the first data point collected the next day emphasizes the trend between these points, which may not be appropriate.

I have been using a fairly simple routine to insert NaNs into the data to break the plot into disconnected sections. For example, modify the above data for plotting with:

IDL> new_y = mg_insert_nan(x, y, [10.0, 35.0], new_x=new_x)
IDL> plot, new_x, new_y


The new plot shows the gaps between the “days” in the data:

James Hague writes:

Though my fascination with Forth is long behind me, I still tend toward minimalist programming, but not in the same, extreme, way. I’ve adopted a more modern approach to minimalism:

Use the highest-level language that’s a viable option.

Lean on the built-in features that do the most work.

Write as little code as possible.

I think this is good advance, but I would add one more point about having as few third party dependencies as possible that tries to balance the last point to write as little code as possible.

It is often useful to display a progress bar showing the state of a task. MG_Progress can easily be used to display a progress bar, percent completion, and estimated time to completion. As a simple example, let’s pretend to load 100 items (while actually just waiting a bit):

foreach i, mg_progress(indgen(100), title='Loading') do wait, 0.1


The above line produces the following output:

Code for mg_progress__define is on GitHub (you will need mg_statusline also). See the code docs for the many other options that can be used with MG_Progress like dealing with a list of items that don’t all take equal time and customizing the display.

A nice list of resources for doing remote sensing in Python, especially if you already know IDL.

The recent article about how to investigate object code got me thinking about the various methods I use to find out about an object/class.

The code in the article, for the example of an object of class IDLgrColorBar, prints:

IDL> ab_obj_info, 'idlgrcolorbar'
Class: IDLGRCOLORBAR
Superclass: IDLGRMODEL
Superclass: IDLGRCONTAINER
Superclass: IDL_CONTAINER
Superclass: IDLGRCOMPONENT
Superclass: IDLITCOMPONENT


as well as some HTML information listing the methods of the objects.

One of the most useful techniques for me is one of my library routines to find the class hierarchy of an object:

IDL> mg_class_hierarchy, idlgrcolorbar()
IDLGRCOLORBAR
IDLGRMODEL
IDLGRCONTAINER
IDL_CONTAINER
IDLGRCOMPONENT
IDLITCOMPONENT


This gives the same top-level information with a bit more detail (IDLgrContainer inherits from both IDL_Container and IDLitComponent), but does not provide any listing of the methods. If you know the name of the method, you can use another of my library routines to find out about it’s arguments:

IDL> man, 'idlgrcolorbar::computedimensions'
Filename: /Applications/exelis/idl85/lib/idlgrcolorbar__define.pro
result = idlgrcolorbar::computedimensions(self, osrcdest, PATH=path)


So I added a METHODS keyword to print out the methods of each class:

IDL> mg_class_hierarchy, 'idlgrcolorbar', /methods
IDLGRCOLORBAR
result = idlgrcolorbar::computedimensions()
result = idlgrcolorbar::init()
idlgrcolorbar::calcsize
idlgrcolorbar::cleanup
idlgrcolorbar::getproperty
idlgrcolorbar::setproperty
IDLGRMODEL
result = idlgrmodel::getctm()
result = idlgrmodel::getxyzrange()
result = idlgrmodel::init()
...


IDLdoc produces these docs, which list the methods of IDLgrColorBar and the hierarchy of superclasses along with a lot of other information including comments that might be in the code headers, but not the methods inherited from those superclasses.

A security update for IDL and ENVI license servers was released today by Harris Geospatial Solutions:

The security vulnerability is limited to computers running a license manager server and should not be an issue when the license server components are only exposed on a trusted network.

If you run an outside facing IDL or ENVI license server, it sounds like you need to update immediately.

Python has the Zen of Python as a guiding philosophy. I think IDL would have something a bit more practical. This is my take on IDL’s philosophy:

The Tao of IDL

Interactive is better than compiled.
Fast to write is better than fast to execute.
But vectorized is better than loops;
WHERE is better than FOR/IF.
There are more uses of histograms than first meet the eye.

A picture is worth at least ten thousand bytes of data,
A million if its 3D and you can rotate it interactively.
Whether direct or function, graphics are easy to create
But the possibilities are endless.

Backwards compatibility is great!
But it doesn’t mean you should index arrays with parentheses forever.
IDL might not have started with objects,
But it has them now, so use them!

There are many file formats and each is the most important to someone.
If you can’t read the data, you can’t analyze it.

Keywords are a great idea — especially
If your parameter has a useful default
Or is an optional output.

By the way, the Bad habits posting is very funny and relevant to Fortran/IDL users.

Noah Lorang argues that data scientists mostly just do arithmetic:

The dirty little secret of the ongoing “data science” boom is that most of what people talk about as being data science isn’t what businesses actually need. Businesses need accurate and actionable information to help them make decisions about how they spend their time and resources. There is a very small subset of business problems that are best solved by machine learning; most of them just need good data and an understanding of what it means that is best gained using simple methods.

This rings true for me in terms of the amount of time I spend doing various tasks during a typical day. I would estimate that 90-95% of my time is spent doing basic shell scripting, writing/modifying IDL code to do basic file and array manipulations, writing IDL GUI applications, etc. But the 5-10% of the other time is still important! The mundane tasks would be pointless without choosing the correct optimization technique and verifying it works. It might be that improving the performance of some section of code makes the difference between keeping up with incoming data or not and that might mean using some “hardcore” technique such as writing the section in C, using a GPU, or making use of some multicore technology.

via FlowingData

I’ve found this translation guide for writing and understanding Python code quite useful. I think it should work if you are familiar with Python and wanting to read/write IDL code also.

Here are ten little programs of ten lines or less to introduce new programmers to IDL. This post1 is motivated by this comp.lang.python thread which became this page on the Python wiki.

Continue reading “Ten little IDL programs.”

1. I’m not sure why, but I’ve had a draft of this post around for almost seven years.

• #### GPULib

GPULib enables IDL developers to access the high-performance capabilities of modern NVIDIA graphics cards without knowledge of CUDA programming.

TaskDL is a task-farming solution for IDL designed for problems with loosely-coupled, parallel applications where no communication between nodes of a cluster is required.

#### mpiDL

mpiDL is a library of IDL bindings for Message Passing Interface (MPI) used for tightly-coupled parallel applications.

#### Remote Data Toolkit

The Remote Data Toolkit is a library of IDL routines allowing for easy access to various scientific data in formats such as OPeNDAP, HDF 5, and netCDF.

• #### Modern IDL

Modern IDL offers IDL programmers one place to look, for beginners and advanced users alike. This book also contains: a thorough tutorial on the core topics of IDL; a comprehensive introduction to the object graphics system; common problems and gotchas with many examples; advanced topics not normally found are discussed throughout the book: regular expressions, object graphics, advanced widget programming, performance, object-oriented programming, etc.

• #### IDLdoc

IDLdoc is an open source utility for generating documentation from IDL source code and specially formatted comments.

#### mgunit

mgunit is an open source unit testing framework for IDL.

#### rIDL

rIDL is an open source IDL command line replacement.

#### mglib

mglib is an open source library of IDL routines in areas of visualization, application development, command line utilities, analysis, data access, etc.