The Advent of Code for 2024 has started! Each day there are two coding problems — solve the first one to unlock the second one. I took the opportunity to try out the new IDL notebooks in VS Code that I just learned about yesterday. It was a nice environment to work through a problem.

IDL 9.0 was released recently^[1]. Some notable new features:

• Apple Silicon Mac support — most of my IDL use is from a terminal window on my Mac to a Linux, but occasionally I do some IDL development directly on my Mac. The improved speed is probably not a factor for me, but it is good to know NV5 is still going to support MacOS.
• HttpRequest class — I have some routines that I use to make requests, but this should be a nice addition to the standard library.
• IDL Extension for VS Code — I wrote about this already, but I missed the IDL native notebooks (not Jupyter notebooks). This looks very interesting, enough to install VSCode for a while to test them out.

There are a few more features, as well as updates and library updates. Details in What’s New in IDL 9.0.

APOD had an atypical image last January (I haven’t been writing a lot recently!) of a periodic table for the elements that showed where each element is normally created. The description lists some examples:

The hydrogen in your body, present in every molecule of water, came from the Big Bang. There are no other appreciable sources of hydrogen in the universe. The carbon in your body was made by nuclear fusion in the interior of stars, as was the oxygen. Much of the iron in your body was made during supernovas of stars that occurred long ago and far away. The gold in your jewelry was likely made from neutron stars during collisions that may have been visible as short-duration gamma-ray bursts or gravitational wave events. Elements like phosphorus and copper are present in our bodies in only small amounts but are essential to the functioning of all known life. The featured periodic table is color coded to indicate humanity’s best guess as to the nuclear origin of all known elements. The sites of nuclear creation of some elements, such as copper, are not really well known and are continuing topics of observational and computational research.

via kottke.org

Jason Kurth took his custom hydrogen alpha solar telescope to the desert in Utah to film the recent annular eclipse in 8k. The result is on YouTube. Amazing! There are also some other great images on his site and Instagram.

via kottke

I have not heard the term “infoposter” until I read Connie Malamed’s post describing them and contrasting them with infographics.

The infoposter, a category name that seems to fit this type of graphic, is not yet defined in Wikipedia. Yet it is probably safe to say that the infoposter is a graphic that conveys multiple segments of information typically using words and numbers to represent quantitative data. Infoposters generally use iconic-type graphic elements for visual design appeal and are typically vertical in orientation, similar to a wall poster.

By the way, this was a reference in Building Science Graphics by Jen Christiansen. I’ll have more to say about it when I finish reading it.

I often need to combine several vectors into an array of structures where the i-th element of the array of structures corresponds to the i-th elements of each of the vectors.

For example, if I have three vectors of differing data types:

IDL> x = indgen(3) + 1
IDL> names = ['Mike', 'George', 'Bill']
IDL> values = 100.0 * randomu(seed, 3)

And I want to combine them into a single array of structures for printing, I can do:

IDL> print, mg_zip(x, names, values), format='(%"%02d. %-15s: %6.2f")'
01. Mike : 3.22
02. George : 43.89
03. Bill : 97.14

This part of a growing collection of routines I have to manipulate tabular data, centered around an mg_table class. The code for mg_zip is in src/tables directory of the mglib repo.

More to come on tables…

There is an official VSCode extension for IDL! NV5 Geospatial mentions it in a blog post:

TLDR: We have an official extension for Visual Studio Code called IDL for VSCode! It is open source on GitHub, you can see some of the features we have planned, and there’s a place to ask questions and provide feedback. See below for a story about our journey to get here, our inspiration for the user experience, and our goals for community engagement.

The extension seems fairly useful. It’s more than just a syntax highlighter. It can give linting-type information: finding syntax errors, give recommendations (for example, you are missing a compile_opt statement), warnings like unused variables, etc. And, of course, everyone’s favorite editor feature — auto-completion.

There is a, so far, quiet, GitHub Discussion for discussing the extension.

The Datawrapper blog has been publishing some data visualization posts. This article gives a an overview of ways to make your legend colors more accessible to viewers:

You designed a useful (and fun!) color key for your data visualization to explain to your readers what your colors mean? Great! Now you’ll want to make sure people don’t forget those associations between colors and categories. This article explores a few ways to do so:

1. Keep your color key always close by
2. Reuse your colors in annotations
3. Make your color key interactive
4. Reuse your colors in tooltips
5. Reuse your colors in text around your visualization

IDL 8.9 was released several months ago, but I didn’t write about it at the time. Here are the release notes and there is a nice overview on YouTube as well. Here are my impressions on what I am personally excited to use.

New features

New licensing mechanism

Hopefully better than the last licensing scheme.

Advanced Scientific Data Format (ASDF) support

This could be useful for me. DKIST is using ASDF to provide a hierarchical listing of all the files in a dataset:

The ASDF file is provided alongside the Dataset to facilitate inspection and analysis of the metadata of a Dataset, without having to transfer all the data. This, in concert with the preview movie, is designed to help make decisions on if a given Dataset is of interest, or what parts of it are, without needing to transfer it.

See ADSF on GitHub.

Use i and j to create complex valued constants

Like:

IDL> help, 1 + 2i
<Expression>    COMPLEX   = (      1.00000,      2.00000)

I don’t need this, but it seems like a handy notation. I wonder how many odd bugs will be created with trying to increment a counter i and accidentally creating a complex variable?

Binary, octal, and hexadecimal constants

Creating binary, octal, and hexadecimal constants with 0b, 0o, and 0x prefixes, i.e., color = 0xff0000 also seems handy. I have a use for the binary constants with creating bitmasks. I create hex constants regularly, so maybe I will use that notation now.

Gabor transform filter

This looks interesting, but I don’t have definite plans for it now.

Template literal strings

I am probably most excited about template literal strings, string like Python f-strings that insert the values of variables into themselves. I tend to do things like the following a lot:

filename = string(date, wave_region, format='%s.ucomp.%s.intensity.mp4')

Now I can do it like this:

filename = `${date}.ucomp.${wave_region}.intensity.mp4`

I find this much easier to read and less error-prone. You can also embed C or Fortran-style format codes as well. Very nice!

New compile_opt options

There are new compile_opt options: float64 makes doubles the default floating point value, and idl3 which is shorthand for defint32, float64, logical_predicate, and strictarr. I tend to use just strictarr.

YAML Ain’t Markup Language (YAML) support

Here’s another one that I think I might end up using a lot. I don’t have a specific need for it right now, but it is another option now for configuration file data.

Updates

• JSON_SERIALIZE PRETTY keyword — I have to serialize JSON output quite a bit and a better printed output would be very useful. It can be difficult to look at strings like:

IDL> print, json
{"sep_forecast_submission":{"model":{"short_name":"MLSO K-Cor","spase_id":"spas
e://NSF/Catalog/MLSO/K-Cor/AutomatedEventList"},"issue_time":"2021-06-28T13:47:
00Z","mode":"realtime","triggers":[{"cme":{"start_time":"2021-06-28T19:28:12Z",
"pa":315,"speed":320.23999,"time_at_height":{"height":1.26,"time":"2021-06-28T1
9:28:12Z"},"catalog":"MLSO_KCOR"}}],"inputs":[{"coronagraph":{"observatory":"ML
SO","instrument":"K-Cor","products":[{"product":"White Light","last_data_time":
"2021-06-28T13:46Z"}]}}],"observations":[{"all_clear":{"all_clear_boolean":fals
e,"all_clear_type":"cme"},"alert":{"alert_type":"ALERT","start_time":"2021-06-2
8T13:47:00Z"}}]}}

This is much easier to read:

IDL> print, json_serialize(json_parse(json), /pretty)
{
    "sep_forecast_submission": {
        "model": {
            "short_name": "MLSO K-Cor",
            "spase_id": "spase://NSF/Catalog/MLSO/K-Cor/AutomatedEventList"
        },
        "issue_time": "2021-06-28T13:47:00Z",
        "mode": "realtime",
...

Libraries

• Keeping up with library updates is a good sign.
• Dropped support for 32-bit and Python Bridge support for Python 3.7. I don’t have old systems that need these, so I am fine with them leaving. You have to cut off old stuff eventually, otherwise new work becomes too difficult.

IDL 8.8.1 was recently released. Some new features and improvements:

• performance improvements by converting some .pro code to C including the List, Hash, and Dictionary classes
• improved JSON_PARSE speed
• many updated libraries, including support for Python 3.70.10, 3.8.10, and 3.9.5
• support for Mac M1 chip

For more details, see the IDL 8.8.1 Release Notes.