I have been doing some reading about machine learning recently, using Python as an implementation language. I lot of the routines used are fairly easy to implement in IDL, so I have started filling out my library with IDL versions.

I have written a scatter plot matrix routine that takes a collection of vectors and makes all the scatter plots between pairs of them. For example, here’s a scatter plot matrix produced by the routine for the classic iris dataset:

If you want to use the routine, it’s probably easiest to clone my entire library.

FlowingData rounds up his list of best visualization projects of 2016:

Visualization continues its merging into the everyday — less standalone and more of a medium that blends with words. I think this is partially because of a concentration on mobile. There’s simply less visual space on a phone than there is a giant computer screen, so the visualization is stripped or split up into smaller pieces that are more easily digested while scrolling.

For example, the Rhythm of Food shows the popular food by time of year:

And if that is not enough for you, here’s a roundup of 2016 visualization roundups.

A curated list of awesome Python frameworks, libraries, software and resources.

Worth checking out when looking for an existing solution.

National Geographic collects the best maps of 2016:

It’s been a good year for map lovers. Whether you’re into old maps, new maps, or new ways of interacting with old maps, there was much to cheer about in 2016.

via kottke.org

The NSF has opened up voting for the People’s Choice for visualizations in the Photo, Illustration, Poster/Graphic, Interactive, and Video categories.

Voting closes Sunday December 4 at 11:59 p.m. PST.

Upgrading from XQuartz from 2.7.9 to 2.7.11 on macOS Sierra broke IDL widgets for me:

~\$ idl
IDL Version 8.5.1, Mac OS X (darwin x86_64 m64).
(c) 2015, Exelis Visual Information Solutions, Inc., a subsidiary of Harris
Corporation.

Error: attempt to add non-widget child "dsm" to parent "idl" which supports
only widgets


The fix that worked for me were the following two commands:

sudo mv /opt/X11/lib/libXt.6.dylib{,.bak}
sudo cp /opt/X11/lib{/flat_namespace,}/libXt.6.dylib


Downgrading to 2.7.9 (but not 2.7.10) also worked for me.

D3 in Depth aims to bridge the gap between introductory tutorials/books and the official documentation.

I have found D3 extremely useful for creating dynamic plots on dashboard style websites for monitoring data pipelines. This looks an excellent resource for learning it.

via FlowingData

I plot a lot of data on daily cycles, where there is no data collected at night. Let’s mock up some sample data with the following simple code:

IDL> x = [findgen(10), findgen(10) + 25, findgen(10) + 50]
IDL> seed = 0L
IDL> y = randomu(seed, 30)
IDL> plot, x, y


Then I get a plot like this:

This plot doesn’t show the nightly breaks in data well. Connecting the last data point collected from a day to the first data point collected the next day emphasizes the trend between these points, which may not be appropriate.

I have been using a fairly simple routine to insert NaNs into the data to break the plot into disconnected sections. For example, modify the above data for plotting with:

IDL> new_y = mg_insert_nan(x, y, [10.0, 35.0], new_x=new_x)
IDL> plot, new_x, new_y


The new plot shows the gaps between the “days” in the data:

James Hague writes:

Though my fascination with Forth is long behind me, I still tend toward minimalist programming, but not in the same, extreme, way. I’ve adopted a more modern approach to minimalism:

Use the highest-level language that’s a viable option.

Lean on the built-in features that do the most work.

Write as little code as possible.

I think this is good advance, but I would add one more point about having as few third party dependencies as possible that tries to balance the last point to write as little code as possible.

Colorgorical is an alternative to ColorBrewer with a few different options for creating color tables. For example, you can add a couple specific colors that should be in the color table and let Colorgorical figure out the others which maximizes the perceptual difference between the colors. Colorgorical seems particularly well suited to generating qualitative color tables, e.g., to find sufficiently different colors for each line in a plot.

via FlowingData.

• #### GPULib

GPULib enables IDL developers to access the high-performance capabilities of modern NVIDIA graphics cards without knowledge of CUDA programming.

TaskDL is a task-farming solution for IDL designed for problems with loosely-coupled, parallel applications where no communication between nodes of a cluster is required.

#### mpiDL

mpiDL is a library of IDL bindings for Message Passing Interface (MPI) used for tightly-coupled parallel applications.

#### Remote Data Toolkit

The Remote Data Toolkit is a library of IDL routines allowing for easy access to various scientific data in formats such as OPeNDAP, HDF 5, and netCDF.

• #### Modern IDL

Modern IDL offers IDL programmers one place to look, for beginners and advanced users alike. This book also contains: a thorough tutorial on the core topics of IDL; a comprehensive introduction to the object graphics system; common problems and gotchas with many examples; advanced topics not normally found are discussed throughout the book: regular expressions, object graphics, advanced widget programming, performance, object-oriented programming, etc.

• #### IDLdoc

IDLdoc is an open source utility for generating documentation from IDL source code and specially formatted comments.

#### mgunit

mgunit is an open source unit testing framework for IDL.

#### rIDL

rIDL is an open source IDL command line replacement.

#### mglib

mglib is an open source library of IDL routines in areas of visualization, application development, command line utilities, analysis, data access, etc.