IDL 8.9 was released several months ago, but I didn’t write about it at the time. Here are the release notes and there is a nice overview on YouTube as well. Here are my impressions on what I am personally excited to use.

New features

New licensing mechanism

Hopefully better than the last licensing scheme.

Advanced Scientific Data Format (ASDF) support

This could be useful for me. DKIST is using ASDF to provide a hierarchical listing of all the files in a dataset:

The ASDF file is provided alongside the Dataset to facilitate inspection and analysis of the metadata of a Dataset, without having to transfer all the data. This, in concert with the preview movie, is designed to help make decisions on if a given Dataset is of interest, or what parts of it are, without needing to transfer it.

See ADSF on GitHub.

Use i and j to create complex valued constants

Like:

IDL> help, 1 + 2i
<Expression>    COMPLEX   = (      1.00000,      2.00000)

I don’t need this, but it seems like a handy notation. I wonder how many odd bugs will be created with trying to increment a counter i and accidentally creating a complex variable?

Binary, octal, and hexadecimal constants

Creating binary, octal, and hexadecimal constants with 0b, 0o, and 0x prefixes, i.e., color = 0xff0000 also seems handy. I have a use for the binary constants with creating bitmasks. I create hex constants regularly, so maybe I will use that notation now.

Gabor transform filter

This looks interesting, but I don’t have definite plans for it now.

Template literal strings

I am probably most excited about template literal strings, string like Python f-strings that insert the values of variables into themselves. I tend to do things like the following a lot:

filename = string(date, wave_region, format='%s.ucomp.%s.intensity.mp4')

Now I can do it like this:

filename = `${date}.ucomp.${wave_region}.intensity.mp4`

I find this much easier to read and less error-prone. You can also embed C or Fortran-style format codes as well. Very nice!

New compile_opt options

There are new compile_opt options: float64 makes doubles the default floating point value, and idl3 which is shorthand for defint32, float64, logical_predicate, and strictarr. I tend to use just strictarr.

YAML Ain’t Markup Language (YAML) support

Here’s another one that I think I might end up using a lot. I don’t have a specific need for it right now, but it is another option now for configuration file data.

Updates

  • JSON_SERIALIZE PRETTY keyword — I have to serialize JSON output quite a bit and a better printed output would be very useful. It can be difficult to look at strings like:
IDL> print, json
{"sep_forecast_submission":{"model":{"short_name":"MLSO K-Cor","spase_id":"spas
e://NSF/Catalog/MLSO/K-Cor/AutomatedEventList"},"issue_time":"2021-06-28T13:47:
00Z","mode":"realtime","triggers":[{"cme":{"start_time":"2021-06-28T19:28:12Z",
"pa":315,"speed":320.23999,"time_at_height":{"height":1.26,"time":"2021-06-28T1
9:28:12Z"},"catalog":"MLSO_KCOR"}}],"inputs":[{"coronagraph":{"observatory":"ML
SO","instrument":"K-Cor","products":[{"product":"White Light","last_data_time":
"2021-06-28T13:46Z"}]}}],"observations":[{"all_clear":{"all_clear_boolean":fals
e,"all_clear_type":"cme"},"alert":{"alert_type":"ALERT","start_time":"2021-06-2
8T13:47:00Z"}}]}}

This is much easier to read:

IDL> print, json_serialize(json_parse(json), /pretty)
{
    "sep_forecast_submission": {
        "model": {
            "short_name": "MLSO K-Cor",
            "spase_id": "spase://NSF/Catalog/MLSO/K-Cor/AutomatedEventList"
        },
        "issue_time": "2021-06-28T13:47:00Z",
        "mode": "realtime",
...

Libraries

  • Keeping up with library updates is a good sign.
  • Dropped support for 32-bit and Python Bridge support for Python 3.7. I don’t have old systems that need these, so I am fine with them leaving. You have to cut off old stuff eventually, otherwise new work becomes too difficult.

IDL 8.8.1 was recently released. Some new features and improvements:

  • performance improvements by converting some .pro code to C including the List, Hash, and Dictionary classes
  • improved JSON_PARSE speed
  • many updated libraries, including support for Python 3.70.10, 3.8.10, and 3.9.5
  • support for Mac M1 chip

For more details, see the IDL 8.8.1 Release Notes.

I am fascinated by this ASCII art drawing application:

Plain text has been around for decades and it’s here to stay. Monodraw allows you to easily create text-based art (like diagrams, layouts, flow charts) and visually represent algorithms, data structures, binary formats and more. Because it’s all just text, it can be easily embedded almost anywhere. Of course, exporting as images is also supported (PNG and SVG).

IDLdoc 3.6.4 was released today! This release contains another security patch for searching.

Update by downloading from the IDLdoc Releases page or using IPM (with package address "http://packages.idldev.com/idldoc.zip")

IDLdoc 3.6.3 was released today! This release contains a security patch for searching.

Update by downloading from the IDLdoc Releases page or using IPM (with package address "http://packages.idldev.com/idldoc.zip")

I have been using Nova as my main text editor for the last few months. It is a Mac-only, extendable editor that has a great "remote mode" which allows me to edit files on remote servers from my laptop. There are many extensions for handling various languages and data formats, but no IDL extension. So I created a simple one. It is definitely not a complete language specification that makes use of all of Nova’s extension capabilities, but it does basic syntax highlighting and some code completion.

You can download the extension through the app — just search for "IDL" in the extension library. Check out the source code and contribute to the project on the GitHub repo.

IDL 8.8 was released recently. The main changes are:

  • A new SOBOLSEQUENCE function to generate numbers from the Sobol sequence.
  • Support for more video formats: DV (Linux only), RAW, Bitmap (Windows only), HEVC, VC-1, and VP-8.
  • Many library updates.
  • Fixed various bugs.
  • IDL Workbench themes, e.g.., dark mode.

See the release notes for more details.

IDL 8.7.3 was released recently1. The new features are:

  1. Using the Intel Math Kernel Library (MKL) for matrix operations now. My experience has been that MKL is a highly optimized implementation of BLAS, a library of low-level matrix and vector operations. Matrix operations, such as the # and ##, should be faster now, but I don’t have 8.7.3 to test on, yet.
  2. New BLASS_GEMM matrix multiplication routine with many options for controlling the operation. This is a consequence of adding the MKL.
  3. New SOBOLSEQUENCE routine to generate numbers from the Sobel sequence.

UPDATE 2/25/2020: I have IDL 8.7.3 now and did some simple testing of the performance between IDL 8.7.3 and IDL 8.6 for # and ##. Mean time for multiplying two 50,000 by 50,000 element matrices went from 76.942 sec in IDL 8.6 down to 3.400 sec in IDL 8.7.3.


  1. I have no idea exactly when or how to find that out. ?

IDL 8.7.2 was released sometime last year1. Check out the What’s New for more details. The big feature is the WIDGET_BROWSER, which allows a browser window to be embedded in an IDL widget program.


  1. I only know because I saw a webinar for the IDL 8.7.3 release, so it must have been awhile ago. 

epochs is a Python package to handle configuration files specifying values which change over time.

For example, consider a configuration file epochs.cfg containing the following:

[2019-04-09 20:27:15]
value : 3

[2019-04-09 22:31:01]
value : 5

Then value can be retrieved for various times:

>>> import epochs
>>> ep = epochs.parse('epochs.cfg')
>>> value = ep.get('value', datetime='2019-04-09 21:55:45')
>>> print(value)
3
>>> value = ep.get('value', datetime='2019-04-09 23:15:40')
>>> print(value)
5

If you provide a specification for the values in a configuration file, epochs can also validate a config file and provide values as the correct type and default value. For example, for a specification file containing:

[city]
name : required=True, type=str
streets : required=True, type=List[str]
temp : required=False, type=float, default=0.0

Then an example configuration file following this specification:

[city]
name : Boulder streets : [Broadway, Baseline, Valmont]

Then to parse the configuration file with a specification:

>>> import epochs
>>> cf = epochs.parse('example.cfg', spec='spec.cfg')
>>> name = cf.get('name', section='city')
>>> print(name)
Boulder
>>> streets = cf.get('streets', section='city')
>>> print(steets)
['Broadway', 'Baseline', 'Valmont']
>>> temp = cf.get('temp', section='city') 
>>> print(temp, type(temp))
0.0

epochs is in PyPI. It can be installed with pip:

pip install epochs

The epochs source code is available on GitHub.

« newer postsolder posts »