The “Zen of Python” provides the basic philosophy of Python. From PEP 20:

There should be one — and preferably only one — obvious way to do it.

I doubt there is a single area in Python that violates this more than making a simple HTTP request. Python provides at least six builtin libraries to do this: httplib, httplib2, urllib, urllib2, urllib3, and pycurl. There are several reviews comparing the various libraries.

But there is a third party library, requests, that might be the “obvious way to do it” now:

import requests, json

url = ''
r = requests.get(url, auth=('mgalloy', 'my_password'))
print r.json()['updated_at']

requests is installed as part of Anaconda, which an easy way to get all the core scientific programming packages for Python.

After nearly six and a half years, today is my last day at Tech-X. Yesterday, I finished all my tasks for the GPULib 1.8 release and the transitioning to the new GPULib product manager.

GPULib will continue on at Tech-X. I will continue to use it and contribute to it. I will also help Tech-X support the product and do my best to make the transition seamless for GPULib users.

I am excited to start my next job at NCAR, but equally excited to have a couple weeks between jobs to explore other things.

GPULib 1.8 has been released with updates to the underlying libraries as well as many other features in many areas of the library. It has been updated to use the most recent versions of IDL and CUDA, IDL 8.4 and CUDA 6.5. The new features are:

  • Support for integer data types. I have been wanting to support integer types in GPULib for awhile and now GPULib supports all the numeric types provided by IDL! We can finally do:

    dx = gpuIndgen(10)
  • Added GPUREPMAT routine. This is a handy routine to create a new array by repeating a 2-dimensional array in a grid.

  • Added GPUCREATEKERNEL routine to create the source code of a simple kernel. This is a code generation routine that can be loaded with GPULOADMODULE/GPULOADFUNCTION and executed with GPUEXECUTEFUNCTION.

  • Added GPUFINITE routine similar to IDL’s library routine.

  • Added linear algebra routines GPULUDC, GPULUSOL, and GPULEAST_SQUARES. This fills out more of the GPU equivalent of the convenience routines provided by IDL so that the LAPACK interface of MAGMA is not required to perform linear algebra computations.

  • Added support for RHO and THETA keywords in GPURADON.

  • Added GPUMEAN routine. This routine has DIMENSION and NAN keywords with the same functionality as IDL’s library routine.

Full disclosure: I work for Tech-X and I am the product manager/developer for GPULib.

I’ll be at the AGU Fall Meeting this year presenting work about SatelliteDL:

SatelliteDL is an IDL toolkit for the analysis of satellite Earth observations from a diverse set of platforms and sensors. The core function of the toolkit is the spatial and temporal alignment of satellite swath and geostationary data.

It’s in the Informatics session on Tuesday afternoon, IN23D-3748. Check out the poster!

The biggest change introduced by IDL 8.4 is the treatment of all variables as objects. Every variable now has attributes that you would normally get returned from the SIZE function: length, ndim, dim, tname, typecode, and typename. For example:

IDL> a = bindgen(2, 3)
IDL> print, a.length
IDL> print, a.ndim
IDL> print, a.dim
           2           3
IDL> print, a.tname

There are also static methods available for all variables:

IDL> n = 50
IDL> print, n->toString()

Strings, numbers, integers, and pointers have their own set of special methods appropriate for their respective types. For example integers have some handle base conversion methods:

IDL> print, n->toHex()
IDL> print, n->toBinary()

Strings have some methods that are not available through other IDL library routines, including the very useful replace method:

IDL> s = 'ITT VIS'
IDL> print, s.replace('ITT', 'Exelis')
Exelis VIS

IDL 8.4 adds a new class FolderWatch to watch a directory for changes to its files:

IDL> fw = folderwatch('.', lambdap(fw, info: print, info.file), /added)

Then if you drop a file into the current directory, for example test.txt, you should see:


The callback procedure, which I passed a lambda procedure1 to in my example, takes two arguments: the FolderWatch object and an info structure with definition:

{ IDLFolderWatchInfo, file: '', added: 0B, modified: 0B, removed: 0B }

The object also has a USER_DATA property which allows you to store information that you might need in the callback routine.

  1. I think I’m going to like lambda routines. 

The accepted test for determining if the user passed in a value for an argument, say arg1, has been:

if (n_elements(arg1) eq 0) then ...

But this can be thwarted since IDL 8.0, which introduced operating overloading, by an object that overloads ::_overloadSize and returns 0.

Brian Griglak in the current IDL Data Point article concludes:

So putting it all together, the best way to validate undefined vs null vs non-null is:

if (Size(val, /TYPE) eq 0) then begin
endif else if (ISA(val, /NULL)) then begin
endif begin  ; I added this line and the next to Brian's code to
  ...        ; distinguish between his three cases

This is a tricky issue to get right for your particular purposes because N_ELEMENTS being 0 has multiple meanings. In my case, I had two cases that I needed to handle:

  1. the number of elements in an array of objects
  2. determine if an argument was present (return nonzero value if it was an object with _overloadSize returning 0)

These issues have always been muddled in IDL (ARG_PRESENT doesn’t tell you if an argument is present, it indicates that it was passed as a named variable).

My manner of testing was to use OBJ_VALID if the argument was an object and you didn’t want to use the normal operator overloading returned value:

function mg_n_elements, var, no_operatoroverload=noOperatoroverload
  compile_opt strictarr

  if (~keyword_set(noOperatoroverload) || size(var, /type) ne 11) then begin
    return, n_elements(var)

  return, n_elements(obj_valid(var))

Download src.

IDL 8.4 introduces arbitrarily large integers with the BigInteger class:

IDL> n = BigInteger(2)^400
IDL> help, n
N               BIGINTEGER <ID=3 LENGTH=401 bits> = 2.582249878086...x10^120
IDL> print, n

Standard arithmetic with operators such as +, *, etc. works fine and there are a few methods to perform some other common operations:

IDL> print, n->nextPrime()
IDL> print, n->log2()

Wow, I wish I had decided to stop giving free updates before IDL 8.4 instead of before IDL 9.0 — there is a lot of new material in Modern IDL 1.4! There are new sections about treating all IDL variables as objects and functional programming, as well as the many minor updates of new routines. If you have purchased Modern IDL by PDF in the past, but didn’t receive an email this morning with a link to your new version of Modern IDL, please let me know.

Note that I am still planning to continue the free updates until IDL 9.0. At that point, I will have to decide based on what’s new in IDL 9.0 how it will continue (but it won’t be a free update).

mglib 1.1.1, my library of IDL routines, was released last month. There are binary packages for OS X and 64-bit Linux on the download page.

My library contains routines in the following areas:

  • visualizations
  • file formats (netCDF, HDF 4 and 5, GRIB, save files, XML)
  • collections (list, hash, etc. classes)
  • routines for distributing IDL applications
  • networking
  • string handling
  • calendar routines
  • many other miscellaneous routines

older posts »