Category "Python"


epochs is a Python package to handle configuration files specifying values which change over time.

For example, consider a configuration file epochs.cfg containing the following:

[2019-04-09 20:27:15]
value : 3

[2019-04-09 22:31:01]
value : 5

Then value can be retrieved for various times:

>>> import epochs
>>> ep = epochs.parse('epochs.cfg')
>>> value = ep.get('value', datetime='2019-04-09 21:55:45')
>>> print(value)
3
>>> value = ep.get('value', datetime='2019-04-09 23:15:40')
>>> print(value)
5

If you provide a specification for the values in a configuration file, epochs can also validate a config file and provide values as the correct type and default value. For example, for a specification file containing:

[city]
name : required=True, type=str
streets : required=True, type=List[str]
temp : required=False, type=float, default=0.0

Then an example configuration file following this specification:

[city]
name : Boulder streets : [Broadway, Baseline, Valmont]

Then to parse the configuration file with a specification:

>>> import epochs
>>> cf = epochs.parse('example.cfg', spec='spec.cfg')
>>> name = cf.get('name', section='city')
>>> print(name)
Boulder
>>> streets = cf.get('streets', section='city')
>>> print(steets)
['Broadway', 'Baseline', 'Valmont']
>>> temp = cf.get('temp', section='city') 
>>> print(temp, type(temp))
0.0

epochs is in PyPI. It can be installed with pip:

pip install epochs

The epochs source code is available on GitHub.

[HoloViews] is yet another Python visualization library, but it has a different approach:

> With HoloViews, instead of building a plot using direct calls to a plotting library, you first describe your data with a small amount of crucial semantic information required to make it visualizable, then you specify additional metadata as needed to determine more detailed aspects of your visualization. This approach provides immediate, automatic visualization that can be effortlessly requested at any time as your data evolves, rendered automatically by one of the supported plotting libraries (such as Bokeh or Matplotlib).

It can produce a variety of visualizations — check out its to see more examples.

[HoloViews]: http://holoviews.org "HoloViews"
: http://holoviews.org/gallery "Gallery - HoloViews"

[iTerm2] is a macOS terminal emulator with a lot of extra features. In particular, it has a simple protocol for displaying images inline. It comes with a program `imgcat` that will display common image formats such as PNG, JPEG, GIF, etc. Most of the images I deal with are FITS, though. I wrote `fitscat` to be a handy utility to display FITS images, as well as to print basic information about the FITS file such as a listing of extensions or an extension header.

For example, `fitscat` can display an image in an extension, as seen below:

There are options to specify a minimum and maximum value for scaling, as well as to use a simple filter such square root.

Also, `fitscat` can also print basic information about a FITS file, such as a listing of extensions:

CoMP$ fitscat --list 20150624.170419.comp.1074.iqu.5.fts
Filename: 20150624.170419.comp.1074.iqu.5.fts
No. Name Ver Type Cards Dimensions Format
0 PRIMARY 1 PrimaryHDU 67 ()
1 I, 1074.38 1 ImageHDU 33 (620, 620) float32
2 I, 1074.50 1 ImageHDU 33 (620, 620) float32
3 I, 1074.62 1 ImageHDU 33 (620, 620) float32
4 I, 1074.74 1 ImageHDU 33 (620, 620) float32
5 I, 1074.86 1 ImageHDU 33 (620, 620) float32
6 Q, 1074.38 1 ImageHDU 33 (620, 620) float32
7 Q, 1074.50 1 ImageHDU 33 (620, 620) float32
8 Q, 1074.62 1 ImageHDU 33 (620, 620) float32
9 Q, 1074.74 1 ImageHDU 33 (620, 620) float32
10 Q, 1074.86 1 ImageHDU 33 (620, 620) float32
11 U, 1074.38 1 ImageHDU 33 (620, 620) float32
12 U, 1074.50 1 ImageHDU 33 (620, 620) float32
13 U, 1074.62 1 ImageHDU 33 (620, 620) float32
14 U, 1074.74 1 ImageHDU 33 (620, 620) float32
15 U, 1074.86 1 ImageHDU 33 (620, 620) float32

Or display a header:

CoMP$ fitscat --header -e 3 20150624.170419.comp.1074.iqu.5.fts
XTENSION= 'IMAGE ' /extension type
BITPIX = -32 /bits per data value
NAXIS = 2 /number of axes
NAXIS1 = 620 /
NAXIS2 = 620 /
PCOUNT = 0 /
GCOUNT = 1 /
EXTNAME = 'I, 1074.62' /
WAVELENG= 1074.620 / WAVELENGTH OF OBS (NM)
POLSTATE= 'I ' / POLARIZATION STATE
EXPOSURE= 250.00 / EXPOSURE TIME (MILLISEC)
NAVERAGE= 16 / Number of images averaged together
FILTER = 1 / FILTER WHEEL POSITION (1-8)
DATATYPE= 'DATA' / DATA, DARK OR FLAT
LCVR1TMP= 29.639999 / DEGREES CELSIUS
LCVR2TMP= 33.429001 /
LCVR3TMP= 33.715000 /
LCVR4TMP= 33.738998 /
LCVR5TMP= 33.618999 /
LCVR6TMP= 28.847000 /
NDFILTER= 8 / ND 1=.1, 2=.3, 3=.5, 4=1, 5=2, 6=3, 7=4, 8=cle
BACKGRND= 13.154 / Median of masked line center background
BODYTEMP= 34.023 / TEMPERATURE OF FILTER BODY (C)
BASETEMP= 33.599 / BASE PLATE TEMP (C)
RACKTEMP= 25.012 / COMPUTER RACK AMBIENT AIR TEMP (C)
OPTRTEMP= 33.306 / OPTICAL RAIL TEMP (C)
DEMULT = 1 / 1=DEMULTIPLEXED, 0=NOT DEMULTIPLEXED
FILTTEMP= 35.000 / ILX FILTER TEMPERATURE (C)
FLATFILE= '20150624.070023.FTS' / Name of flat field file
INHERIT = T /
DISPMIN = 0.00 / Minimum data value
DISPMAX = 5.00 / Maximum data value
DISPEXP = 0.50 / Exponent value for scaling

The full interface of `fitscat` is shown below:

$ fitscat --help
usage: fitscat [-h] [--min MIN] [--max MAX] [--debug] [-d] [-l] [-r]
[-e EXTEN_NO] [-f FILTER] [-s SLICE]
filename

fitscat - a FITS query/display program

positional arguments:
filename FITS file to query

optional arguments:
-h, --help show this help message and exit
--min MIN min for scaling
--max MAX max for scaling
--debug set to debug
-d, --display set to display
-l, --list set to list HDUs
-r, --header set to display header
-e EXTEN_NO, --exten_no EXTEN_NO
specify extension
-f FILTER, --filter FILTER
specify filter (default: none)
-s SLICE, --slice SLICE
specify slice of data array to display

Source code for the Python script is available on [GitHub]. The script is compatible with Python 2 and 3, but requires standard scientific Python packages AstroPy, NumPy, and PIL.

[iTerm2]: https://iterm2.com "iTerm2"
[GitHub]: https://github.com/mgalloy/scripts/blob/master/fitscat "scripts/fitscat"

Travis Oliphant, creator of NumPy the array package for Python, wrote a [analog] to the [Zen of Python] for NumPy:

>Strided is better than scattered
>Contiguous is better than strided
>Descriptive is better than imperative (use data-types)
>Array-oriented is often better than object-oriented
>Broadcasting is a great idea -- use where possible
>Vectorized is better than an explicit loop
>Unless it’s complicated --- then use numexpr, weave, or Cython
>Think in higher dimensions

I tried [something for IDL] last year.

[analog]: http://technicaldiscovery.blogspot.com/2010/11/zen-of-numpy.html?m=1 "Zen of NumPy"
[Zen of Python]: https://www.python.org/dev/peps/pep-0020/ "PEP 20"
[something for IDL]: http://michaelgalloy.com/2016/03/22/philosophy-of-idl.html "Philosophy of IDL"

[Awesome Python]:

> A curated list of awesome Python frameworks, libraries, software and resources.

Worth checking out when looking for an existing solution.

[Awesome Python]: https://github.com/vinta/awesome-python "vinta/awesome-python"

Great post examining some of the reasons why the [FFT algorithm] is so fast compared to a naive implementation:

> The goal of this post is to dive into the Cooley-Tukey FFT algorithm, explaining the symmetries that lead to it, and to show some straightforward Python implementations putting the theory into practice. My hope is that this exploration will give data scientists like myself a more complete picture of what's going on in the background of the algorithms we use.

[FFT algorithm]: http://jakevdp.github.io/blog/2013/08/28/understanding-the-fft/ "Understanding the FFT Algorithm"

A nice list of [resources] for doing remote sensing in Python, especially if you already know IDL.

[resources]: http://blog.rtwilson.com/resources-for-learning-python-for-remote-sensing-or-switching-from-idl/ "Resources for learning Python for Remote Sensing – or switching from IDL"

I've found [this translation guide] for writing and understanding Python code quite useful. I think it should work if you are familiar with Python and wanting to read/write IDL code also.

[this translation guide]: http://mathesaurus.sourceforge.net/idl-numpy.html "NumPy for IDL users"

[Python Unlocked]:

> Python is a versatile programming language that can be used for a wide range of technical tasks — computation, statistics, data analysis, game development, and more. Though Python is easy to learn, its range of features means there are many aspects of it that even experienced Python developers don’t know about. Even if you’re confident with the basics, its logic and syntax, by digging deeper you can work much more effectively with Python – and get more from the language.

> Python Unlocked walks you through the most effective techniques and best practices for high performance Python programming - showing you how to make the most of the Python language.

I find myself reading more about best practices, especially in Python.

If you are interested, promo code PYTUNL30 will get you 30% off the ebook until Feb 20.

[Python Unlocked]: http://bit.ly/1PGLDHw "Python Unlocked"

One of the major features of IDL 8.5 is the two-way bridge between IDL and Python. This allows Python functionality to be accessed from IDL (Python has a lot of libraries for things that fall outside of the standard scientific routines found in IDL) as well as accessing IDL functionality from Python (call legacy IDL code).

The IDL-Python bridge works with either Python 2 or 3 (whew, I'm still on Python 2!).

Continue reading "IDL 8.5: IDL-Python bridge."

older posts »