Here’s a tutorial of how to make an animation of the moon’s shadow with GOES imagery during The Great American Eclipse of 2017:

Here is one of the coolest examples that I have created using IDL in a while. For this blog post, I’m going to walk through how I created an animation of the Moon’s shadow during the Great American Total Solar Eclipse using several different technologies for accessing, downloading, and visualizing the data.

The video is on Harris Geospatial Solutions’ Facebook page.

The dataviz.tools site is an annotated and categorized catalog of good visualization tools.

This site features a curated selection of data visualization tools meant to bridge the gap between programmers/statisticians and the general public by only highlighting free/freemium, responsive and relatively simple-to-learn technologies for displaying both basic and complex, multivariate datasets.

via FlowingData

Some great tips for spotting misleading visualizations:

By using dual axes, the magnitude can shrink or expand for each metric. This is typically done to imply correlation and causation. “Because of this, this other thing happened. See, it’s clear.”

There are some great links as examples of these problems, like the spurious correlations project by Tyler Vigen to automatically find correlations.

I think “machine learning” in this paper applies fairly well to any type of scientific pipeline code:

Using the framework of technical debt, we note that it is remarkably easy to incur massive ongoing maintenance costs at the system level when applying machine learning.

The authors argue that machine learning systems have the regular issues of a code, but also have other complexities that are not necessary addressed in the normal way of refactoring libraries, adding unit tests, etc.

IDL 8.6.1 was released today1. Some interesting new features:

  • Conditional breakpoints from the Workbench
  • Hexadecimal constants, e.g., a = 0xFF3A
  • Fix for strings that begin with numerals being confused with the octal notation: "123 is an octal value; "123" used to be a syntax error, but is now a valid string.

See the release notes for details.


  1. Really sometime in the last week or so. The announcement on the newsgroup was today, but the release notes was posted 7/27. 

Travis Oliphant, creator of NumPy the array package for Python, wrote a analog to the Zen of Python for NumPy:

Strided is better than scattered
Contiguous is better than strided
Descriptive is better than imperative (use data-types)
Array-oriented is often better than object-oriented
Broadcasting is a great idea — use where possible
Vectorized is better than an explicit loop
Unless it’s complicated — then use numexpr, weave, or Cython
Think in higher dimensions

I tried something for IDL last year.

I will be posting more about the Great American Eclipse of 2017 this summer. To start off with, NASA has made some great maps of where the total eclipse will be visible from:

On August 21, 2017, the moon will pass between Earth and the sun in a total solar eclipse that will be visible on a path from Oregon to South Carolina across the continental United States. This path of totality will occur in a little over 90 minutes, while observers on the ground will see the eclipse for about two and a half minutes. Standing at the edge of the moon’s shadow, or umbra, the difference between seeing a total eclipse and a partial eclipse comes down to elevation – mountains and valleys both on Earth and on the moon – which affect where the shadow lands. In this visualization, data from NASA’s Lunar Reconnaissance Orbiter account for the moon’s terrain that creates a jagged edge on its shadow. This data is then combined with elevation data on Earth as well as information on the sun angle to create the most accurate map of the eclipse path to date. Watch the video to learn more.

NASA also has downloadable, detailed state maps of the eclipse path.

via kottke

Here are some odd, but totally legal, IDL statements. I would suggest staying away from all of them and use more conventional syntax.

Here’s one that looks like a string, but is really specifying an octal value:

IDL> x = "12
IDL> help, x
X               INT       =       10

Back when octal values were much more important than now, I suppose it made some sense to have special syntax for entering them. In modern times, I would suggest x = '12'o.

This also means that if you are specifying a string that begins with digits 0-7 with double quotes, you can generate a bewildering syntax error:

IDL> y = "12 monkeys"
              ^
% Syntax error.

I recommend using single quotes for all strings, i.e., y = '12 monkeys'.

Next up is a convenience for the truly lazy:

IDL> s = 'some string

You don’t have to put the trailing single or double quote on a string if it is the last character on the line. This will probably make your text editor’s syntax highlighting confused. One character is not too much to type for some clarity.

Finally, I just saw this one last week:

IDL> for i = 0, 4 do begin y = i & print, i
       0
       1
       2
       3
       4

There are quite a few problems with this:

  • There is a begin with no matching end!
  • There is not a & after the begin even though you would normally have to start a new line there.
  • I would recommend against using &. It can be useful on the command line (it makes it easier to up arrow to a previous set of commands), but don’t do it in a file!

This is the standard syntax for that line (if you really need to put it all on a single line):

IDL> for i = 0, 4 do begin & y = i & print, i & endfor

I might count using parentheses for indexing arrays as a syntax oddity as well, but there are so many IDL programmers still doing it that it counts as commonplace. I still recommend against it.

Excellent rundown of all the horrible rules that organizations impose on your passwords:

  • They don’t work.
  • They heavily penalize your ideal audience, people that use real random password generators. Hey guess what, that password randomly didn’t have a number or symbol in it. I just double checked my math textbook, and yep, it’s possible. I’m pretty sure.
  • They frustrate average users, who then become uncooperative and use “creative” workarounds that make their passwords less secure.
  • They are often wrong, in the sense that the rules chosen are grossly incomplete and/or insane, per the many shaming links I’ve shared above.
  • Seriously, for the love of God, stop with this arbitrary password rule nonsense already. If you won’t take my word for it, read this 2016 NIST password rules recommendation. It’s right there, “no composition rules”. However, I do see one error, it should have said “no bullshit composition rules”.

My personal pet peeve is forced expiration for no reason. NIST is developing guidelines.

The Mathematics Genealogy Project is an amazing effort to record basic information about every mathematician in the world. We can create a family tree for any mathematician. Here is my tree:

For a description of how to create the graph of another mathematician’s genealogy, see Dana C. Ernst’s article.

older posts »