Category "IDL"


I plot a lot of data on daily cycles, where there is no data collected at night. Let's mock up some sample data with the following simple code:

IDL> x = [findgen(10), findgen(10) + 25, findgen(10) + 50]
IDL> seed = 0L
IDL> y = randomu(seed, 30)
IDL> plot, x, y

Then I get a plot like this:

This plot doesn't show the nightly breaks in data well. Connecting the last data point collected from a day to the first data point collected the next day emphasizes the trend between these points, which may not be appropriate.

I have been using [a fairly simple routine] to insert NaNs into the data to break the plot into disconnected sections. For example, modify the above data for plotting with:

IDL> new_y = mg_insert_nan(x, y, [10.0, 35.0], new_x=new_x)
IDL> plot, new_x, new_y

The new plot shows the gaps between the "days" in the data:

[a fairly simple routine]: https://github.com/mgalloy/mglib/blob/master/src/vis/util/mg_insert_nan.pro "mglib/mg_insert_nan.pro at master"

James Hague [writes]:

>Though my fascination with Forth is long behind me, I still tend toward minimalist programming, but not in the same, extreme, way. I've adopted a more modern approach to minimalism:

>Use the highest-level language that's a viable option.

>Lean on the built-in features that do the most work.

>Write as little code as possible.

I think this is good advance, but I would add one more point about having as few third party dependencies as possible that tries to balance the last point to write as little code as possible.

[writes]: http://prog21.dadgum.com/223.html "The New Minimalism"

It is often useful to display a progress bar showing the state of a task. `MG_Progress` can easily be used to display a progress bar, percent completion, and estimated time to completion. As a simple example, let's pretend to load 100 items (while actually just waiting a bit):

foreach i, mg_progress(indgen(100), title='Loading') do wait, 0.1

The above line produces the following output:

Code for [mg_progress__define] is on GitHub (you will need [mg_statusline] also). See the code docs for the many other options that can be used with `MG_Progress` like dealing with a list of items that don't all take equal time and customizing the display.

[mg_progress__define]: https://github.com/mgalloy/mglib/blob/master/src/cmdline_tools/mg_progress__define.pro "mglib/mg_progress__define.pro"

[mg_statusline]: https://github.com/mgalloy/mglib/blob/master/src/cmdline_tools/mg_statusline.pro "mglib/mg_statusline.pro"

A nice list of [resources] for doing remote sensing in Python, especially if you already know IDL.

[resources]: http://blog.rtwilson.com/resources-for-learning-python-for-remote-sensing-or-switching-from-idl/ "Resources for learning Python for Remote Sensing – or switching from IDL"

The recent article about how to [investigate object code] got me thinking about the various methods I use to find out about an object/class.

The code in the article, for the example of an object of class `IDLgrColorBar`, prints:

IDL> ab_obj_info, 'idlgrcolorbar'
Class: IDLGRCOLORBAR
Superclass: IDLGRMODEL
Superclass: IDLGRCONTAINER
Superclass: IDL_CONTAINER
Superclass: IDLGRCOMPONENT
Superclass: IDLITCOMPONENT

as well as some HTML [information] listing the methods of the objects.

One of the most useful techniques for me is one of my [library] routines to find the class hierarchy of an object:

IDL> mg_class_hierarchy, idlgrcolorbar()
IDLGRCOLORBAR
IDLGRMODEL
IDLGRCONTAINER
IDL_CONTAINER
IDLGRCOMPONENT
IDLITCOMPONENT

This gives the same top-level information with a bit more detail (`IDLgrContainer` inherits from both `IDL_Container` and `IDLitComponent`), but does not provide any listing of the methods. If you know the name of the method, you can use another of my library routines to find out about it's arguments:

IDL> man, 'idlgrcolorbar::computedimensions'
Filename: /Applications/exelis/idl85/lib/idlgrcolorbar__define.pro
result = idlgrcolorbar::computedimensions(self, osrcdest, PATH=path)

So I added a `METHODS` keyword to print out the methods of each class:

IDL> mg_class_hierarchy, 'idlgrcolorbar', /methods
IDLGRCOLORBAR
result = idlgrcolorbar::computedimensions()
result = idlgrcolorbar::init()
idlgrcolorbar::calcsize
idlgrcolorbar::cleanup
idlgrcolorbar::getproperty
idlgrcolorbar::setproperty
IDLGRMODEL
result = idlgrmodel::getctm()
result = idlgrmodel::getxyzrange()
result = idlgrmodel::init()
idlgrmodel::add
...

[IDLdoc] produces these [docs], which list the methods of `IDLgrColorBar` and the hierarchy of superclasses along with a lot of other information including comments that might be in the code headers, but not the methods inherited from those superclasses.

[investigate object code]: http://www.harrisgeospatial.com/Company/PressRoom/Blogs/IDLDataPointDetail/TabId/902/ArtMID/2926/ArticleID/14798/How-to-investigate-IDL-object-code.aspx "How to investigate IDL object code"
[library]: https://github.com/mgalloy/mglib "mglib GitHub repo"
[IDLdoc]: https://github.com/mgalloy/idldoc "IDLdoc GitHub repo"
[docs]: http://michaelgalloy.com/wp-content/uploads/2016/03/object_info-docs/idlgrcolorbar__define.html "IDLgrColorBar IDLdoc info"
[information]: http://michaelgalloy.com/wp-content/uploads/2016/03/idlgrcolorbar.html "IDLgrColorBar info"

A [security update] for IDL and ENVI license servers was released today by Harris Geospatial Solutions:

>The security vulnerability is limited to computers running a license manager server and should not be an issue when the license server components are only exposed on a trusted network.

If you run an outside facing IDL or ENVI license server, it sounds like you need to update immediately.

[security update]: http://www.harrisgeospatial.com/Company/PressRoom/Blogs/IDLDataPointDetail/TabId/902/ArtMID/2926/ArticleID/14795/IDL--ENVI-License-Server-Security-Patch.aspx "IDL & ENVI License Server Security Patch"

Python has the [Zen of Python] as a guiding philosophy. I think IDL would have something a bit more practical. This is my take on IDL's philosophy:

>The Tao of IDL

>Interactive is better than compiled.
>Fast to write is better than fast to execute.
>But vectorized is better than loops;
>`WHERE` is better than `FOR`/`IF`.
>There are more uses of histograms than first meet the eye.

>A picture is worth at least ten thousand bytes of data,
>A million if its 3D and you can rotate it interactively.
>Whether direct or function, graphics are easy to create
>But the possibilities are endless.

>Backwards compatibility is great!
>But it doesn't mean you should index arrays with parentheses forever.
>IDL might not have started with objects,
>But it has them now, so use them!

>There are many file formats and each is the most important to someone.
>If you can't read the data, you can't analyze it.

>Keywords are a great idea -- especially
>If your parameter has a useful default
>Or is an optional output.

By the way, the [Bad habits] posting is very funny and relevant to Fortran/IDL users.

[Zen of Python]: http://www.python.org/dev/peps/pep-0020/
[Bad habits]: http://www.python.org/doc/humor/#bad-habits

Noah Lorang argues that [data scientists mostly just do arithmetic][arithmetic]:

> The dirty little secret of the ongoing “data science” boom is that most of what people talk about as being data science isn’t what businesses actually need. Businesses need accurate and actionable information to help them make decisions about how they spend their time and resources. There is a very small subset of business problems that are best solved by machine learning; most of them just need good data and an understanding of what it means that is best gained using simple methods.

This rings true for me in terms of the amount of time I spend doing various tasks during a typical day. I would estimate that 90-95% of my time is spent doing basic shell scripting, writing/modifying IDL code to do basic file and array manipulations, writing IDL GUI applications, etc. But the 5-10% of the other time is still important! The mundane tasks would be pointless without choosing the correct optimization technique and verifying it works. It might be that improving the performance of some section of code makes the difference between keeping up with incoming data or not and that might mean using some "hardcore" technique such as writing the section in C, using a GPU, or making use of some multicore technology.

via [FlowingData]

[arithmetic]: https://m.signalvnoise.com/data-scientists-mostly-just-do-arithmetic-and-that-s-a-good-thing-c6371885f7f6#.h4o1x2j66 "Data scientists mostly just do arithmetic and that’s a good thing"
[FlowingData]: http://flowingdata.com/2016/02/18/data-scientists-mostly-just-do-arithmetic/ "Data scientists mostly just do arithmetic"

I've found [this translation guide] for writing and understanding Python code quite useful. I think it should work if you are familiar with Python and wanting to read/write IDL code also.

[this translation guide]: http://mathesaurus.sourceforge.net/idl-numpy.html "NumPy for IDL users"

Here are ten little programs of ten lines or less to introduce new programmers to IDL. This post[^1] is motivated by this [comp.lang.python thread] which became this [page][Simple Python programs] on the Python wiki.

[comp.lang.python thread]: http://groups.google.com/group/comp.lang.python/browse_frm/thread/3eb8837f6dc278c1 "ten small Python programs"
[Simple Python programs]: http://wiki.python.org/moin/SimplePrograms "Simple Python programs"

[^1]: I'm not sure why, but I've had a draft of this post around for almost seven years.

Continue reading "Ten little IDL programs."

« newer postsolder posts »