Category "Visualization"


iTerm2 is a macOS terminal emulator with a lot of extra features. In particular, it has a simple protocol for displaying images inline. It comes with a program imgcat that will display common image formats such as PNG, JPEG, GIF, etc. Most of the images I deal with are FITS, though. I wrote fitscat to be a handy utility to display FITS images, as well as to print basic information about the FITS file such as a listing of extensions or an extension header.

For example, fitscat can display an image in an extension, as seen below:

There are options to specify a minimum and maximum value for scaling, as well as to use a simple filter such square root.

Also, fitscat can also print basic information about a FITS file, such as a listing of extensions:

CoMP$ fitscat --list 20150624.170419.comp.1074.iqu.5.fts
Filename: 20150624.170419.comp.1074.iqu.5.fts
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      67   ()
  1  I, 1074.38    1 ImageHDU        33   (620, 620)   float32
  2  I, 1074.50    1 ImageHDU        33   (620, 620)   float32
  3  I, 1074.62    1 ImageHDU        33   (620, 620)   float32
  4  I, 1074.74    1 ImageHDU        33   (620, 620)   float32
  5  I, 1074.86    1 ImageHDU        33   (620, 620)   float32
  6  Q, 1074.38    1 ImageHDU        33   (620, 620)   float32
  7  Q, 1074.50    1 ImageHDU        33   (620, 620)   float32
  8  Q, 1074.62    1 ImageHDU        33   (620, 620)   float32
  9  Q, 1074.74    1 ImageHDU        33   (620, 620)   float32
 10  Q, 1074.86    1 ImageHDU        33   (620, 620)   float32
 11  U, 1074.38    1 ImageHDU        33   (620, 620)   float32
 12  U, 1074.50    1 ImageHDU        33   (620, 620)   float32
 13  U, 1074.62    1 ImageHDU        33   (620, 620)   float32
 14  U, 1074.74    1 ImageHDU        33   (620, 620)   float32
 15  U, 1074.86    1 ImageHDU        33   (620, 620)   float32

Or display a header:

CoMP$ fitscat --header -e 3 20150624.170419.comp.1074.iqu.5.fts
XTENSION= 'IMAGE   '           /extension type
BITPIX  =                  -32 /bits per data value
NAXIS   =                    2 /number of axes
NAXIS1  =                  620 /
NAXIS2  =                  620 /
PCOUNT  =                    0 /
GCOUNT  =                    1 /
EXTNAME = 'I, 1074.62'         /
WAVELENG=             1074.620 / WAVELENGTH OF OBS (NM)
POLSTATE= 'I       '           / POLARIZATION STATE
EXPOSURE=               250.00 / EXPOSURE TIME (MILLISEC)
NAVERAGE=                   16 / Number of images averaged together
FILTER  =                    1 / FILTER WHEEL POSITION (1-8)
DATATYPE=               'DATA' / DATA, DARK OR FLAT
LCVR1TMP=            29.639999 / DEGREES CELSIUS
LCVR2TMP=            33.429001 /
LCVR3TMP=            33.715000 /
LCVR4TMP=            33.738998 /
LCVR5TMP=            33.618999 /
LCVR6TMP=            28.847000 /
NDFILTER=                    8 / ND 1=.1, 2=.3, 3=.5, 4=1, 5=2, 6=3, 7=4, 8=cle
BACKGRND=               13.154 / Median of masked line center background
BODYTEMP=               34.023 / TEMPERATURE OF FILTER BODY (C)
BASETEMP=               33.599 / BASE PLATE TEMP (C)
RACKTEMP=               25.012 / COMPUTER RACK AMBIENT AIR TEMP (C)
OPTRTEMP=               33.306 / OPTICAL RAIL TEMP (C)
DEMULT  =                    1 / 1=DEMULTIPLEXED, 0=NOT DEMULTIPLEXED
FILTTEMP=               35.000 / ILX FILTER TEMPERATURE (C)
FLATFILE= '20150624.070023.FTS' / Name of flat field file
INHERIT =                    T /
DISPMIN =                 0.00 / Minimum data value
DISPMAX =                 5.00 / Maximum data value
DISPEXP =                 0.50 / Exponent value for scaling

The full interface of fitscat is shown below:

$ fitscat --help
usage: fitscat [-h] [--min MIN] [--max MAX] [--debug] [-d] [-l] [-r]
               [-e EXTEN_NO] [-f FILTER] [-s SLICE]
               filename

fitscat - a FITS query/display program

positional arguments:
  filename              FITS file to query

optional arguments:
  -h, --help            show this help message and exit
  --min MIN             min for scaling
  --max MAX             max for scaling
  --debug               set to debug
  -d, --display         set to display
  -l, --list            set to list HDUs
  -r, --header          set to display header
  -e EXTEN_NO, --exten_no EXTEN_NO
                        specify extension
  -f FILTER, --filter FILTER
                        specify filter (default: none)
  -s SLICE, --slice SLICE
                        specify slice of data array to display

Source code for the Python script is available on GitHub. The script is compatible with Python 2 and 3, but requires standard scientific Python packages AstroPy, NumPy, and PIL.

Here’s a tutorial of how to make an animation of the moon’s shadow with GOES imagery during The Great American Eclipse of 2017:

Here is one of the coolest examples that I have created using IDL in a while. For this blog post, I’m going to walk through how I created an animation of the Moon’s shadow during the Great American Total Solar Eclipse using several different technologies for accessing, downloading, and visualizing the data.

The video is on Harris Geospatial Solutions’ Facebook page.

The dataviz.tools site is an annotated and categorized catalog of good visualization tools.

This site features a curated selection of data visualization tools meant to bridge the gap between programmers/statisticians and the general public by only highlighting free/freemium, responsive and relatively simple-to-learn technologies for displaying both basic and complex, multivariate datasets.

via FlowingData

Some great tips for spotting misleading visualizations:

By using dual axes, the magnitude can shrink or expand for each metric. This is typically done to imply correlation and causation. “Because of this, this other thing happened. See, it’s clear.”

There are some great links as examples of these problems, like the spurious correlations project by Tyler Vigen to automatically find correlations.

The Mathematics Genealogy Project is an amazing effort to record basic information about every mathematician in the world. We can create a family tree for any mathematician. Here is my tree:

For a description of how to create the graph of another mathematician’s genealogy, see Dana C. Ernst’s article.

I have been doing some reading about machine learning recently, using Python as an implementation language. I lot of the routines used are fairly easy to implement in IDL, so I have started filling out my library with IDL versions.

I have written a scatter plot matrix routine that takes a collection of vectors and makes all the scatter plots between pairs of them. For example, here’s a scatter plot matrix produced by the routine for the classic iris dataset:

If you want to use the routine, it’s probably easiest to clone my entire library.

FlowingData rounds up his list of best visualization projects of 2016:

Visualization continues its merging into the everyday — less standalone and more of a medium that blends with words. I think this is partially because of a concentration on mobile. There’s simply less visual space on a phone than there is a giant computer screen, so the visualization is stripped or split up into smaller pieces that are more easily digested while scrolling.

For example, the Rhythm of Food shows the popular food by time of year:

And if that is not enough for you, here’s a roundup of 2016 visualization roundups.

National Geographic collects the best maps of 2016:

It’s been a good year for map lovers. Whether you’re into old maps, new maps, or new ways of interacting with old maps, there was much to cheer about in 2016.

via kottke.org

The NSF has opened up voting for the People’s Choice for visualizations in the Photo, Illustration, Poster/Graphic, Interactive, and Video categories.

Voting closes Sunday December 4 at 11:59 p.m. PST.

D3 in Depth:

D3 in Depth aims to bridge the gap between introductory tutorials/books and the official documentation.

I have found D3 extremely useful for creating dynamic plots on dashboard style websites for monitoring data pipelines. This looks an excellent resource for learning it.

via FlowingData

older posts »