I have long implored others to abandon the use of common blocks for a myriad of reasons. They create a web of interconnections between the routines that use the common block (and which routines, exactly, are those? You can find out by grepping, but it is not immediately obvious). It is not clear at all which routine is responsible for changing the value of some variable stored in a common block. It is easy for the name of a variable intended to be local to clash with the name of a common block variable without realizing it.

But, sometimes, they provide a nice, quick solution for a problem that would otherwise require building a lot of infrastructure. I have been doing various types of profiling of a code recently. After finding the “hot” routines using PROFILER, I have been trying to narrow down to (and quantify the runtime of) the particular lines using up an appreciable amount of time. The problem is that I want the cumulative runtime for the lines over the course of executing many times during a run.

So I created a “profiling” common block with a time variable, initialized at the beginning of the program, which is incremented after the lines run each time. This can be reported at the end of the program. I have been moving the common block around to various routines to get times until I found the “hot” lines within the “hot” routines.

Full disclosure: this is not the only place I have used common blocks. MG_LOG, a routine that I use a lot, has a common block.

After spending awhile last Friday trying to vectorize a loop of a small matrix-vector multiplication for every pixel of an image, I gave up and decided to just write it as a DLM. For my image sizes of 1024 by 1024 pixels (actually two images of that size), the run time went from 3.15 seconds to 0.26 seconds on my MacBook Pro. That’s not a lot of time to save, but since we acquire imagery every 15 seconds, it was useful.

Check out analysis.c for source code. There are also unit tests showing how to use it.

The “Zen of Python” provides the basic philosophy of Python. From PEP 20:

There should be one — and preferably only one — obvious way to do it.

I doubt there is a single area in Python that violates this more than making a simple HTTP request. Python provides at least six builtin libraries to do this: httplib, httplib2, urllib, urllib2, urllib3, and pycurl. There are several reviews comparing the various libraries.

But there is a third party library, requests, that might be the “obvious way to do it” now:

import requests, json

url = 'https://api.github.com/repos/mgalloy/mglib'
r = requests.get(url, auth=('mgalloy', 'my_password'))
print r.json()['updated_at']

requests is installed as part of Anaconda, which an easy way to get all the core scientific programming packages for Python.

After nearly six and a half years, today is my last day at Tech-X. Yesterday, I finished all my tasks for the GPULib 1.8 release and the transitioning to the new GPULib product manager.

GPULib will continue on at Tech-X. I will continue to use it and contribute to it. I will also help Tech-X support the product and do my best to make the transition seamless for GPULib users.

I am excited to start my next job at NCAR, but equally excited to have a couple weeks between jobs to explore other things.

GPULib 1.8 has been released with updates to the underlying libraries as well as many other features in many areas of the library. It has been updated to use the most recent versions of IDL and CUDA, IDL 8.4 and CUDA 6.5. The new features are:

  • Support for integer data types. I have been wanting to support integer types in GPULib for awhile and now GPULib supports all the numeric types provided by IDL! We can finally do:

    dx = gpuIndgen(10)
    
  • Added GPUREPMAT routine. This is a handy routine to create a new array by repeating a 2-dimensional array in a grid.

  • Added GPUCREATEKERNEL routine to create the source code of a simple kernel. This is a code generation routine that can be loaded with GPULOADMODULE/GPULOADFUNCTION and executed with GPUEXECUTEFUNCTION.

  • Added GPUFINITE routine similar to IDL’s library routine.

  • Added linear algebra routines GPULUDC, GPULUSOL, and GPULEAST_SQUARES. This fills out more of the GPU equivalent of the convenience routines provided by IDL so that the LAPACK interface of MAGMA is not required to perform linear algebra computations.

  • Added support for RHO and THETA keywords in GPURADON.

  • Added GPUMEAN routine. This routine has DIMENSION and NAN keywords with the same functionality as IDL’s library routine.

Full disclosure: I work for Tech-X and I am the product manager/developer for GPULib.

I’ll be at the AGU Fall Meeting this year presenting work about SatelliteDL:

SatelliteDL is an IDL toolkit for the analysis of satellite Earth observations from a diverse set of platforms and sensors. The core function of the toolkit is the spatial and temporal alignment of satellite swath and geostationary data.

It’s in the Informatics session on Tuesday afternoon, IN23D-3748. Check out the poster!

The biggest change introduced by IDL 8.4 is the treatment of all variables as objects. Every variable now has attributes that you would normally get returned from the SIZE function: length, ndim, dim, tname, typecode, and typename. For example:

IDL> a = bindgen(2, 3)
IDL> print, a.length
           6
IDL> print, a.ndim
           2
IDL> print, a.dim
           2           3
IDL> print, a.tname
BYTE

There are also static methods available for all variables:

IDL> n = 50
IDL> print, n->toString()
50

Strings, numbers, integers, and pointers have their own set of special methods appropriate for their respective types. For example integers have some handle base conversion methods:

IDL> print, n->toHex()
32
IDL> print, n->toBinary()
110010

Strings have some methods that are not available through other IDL library routines, including the very useful replace method:

IDL> s = 'ITT VIS'
IDL> print, s.replace('ITT', 'Exelis')
Exelis VIS

IDL 8.4 adds a new class FolderWatch to watch a directory for changes to its files:

IDL> fw = folderwatch('.', lambdap(fw, info: print, info.file), /added)

Then if you drop a file into the current directory, for example test.txt, you should see:

IDL>
test.txt

The callback procedure, which I passed a lambda procedure1 to in my example, takes two arguments: the FolderWatch object and an info structure with definition:

{ IDLFolderWatchInfo, file: '', added: 0B, modified: 0B, removed: 0B }

The object also has a USER_DATA property which allows you to store information that you might need in the callback routine.


  1. I think I’m going to like lambda routines. 

The accepted test for determining if the user passed in a value for an argument, say arg1, has been:

if (n_elements(arg1) eq 0) then ...

But this can be thwarted since IDL 8.0, which introduced operating overloading, by an object that overloads ::_overloadSize and returns 0.

Brian Griglak in the current IDL Data Point article concludes:

So putting it all together, the best way to validate undefined vs null vs non-null is:

if (Size(val, /TYPE) eq 0) then begin
  ...
endif else if (ISA(val, /NULL)) then begin
  ...
endif begin  ; I added this line and the next to Brian's code to
  ...        ; distinguish between his three cases
endelse

This is a tricky issue to get right for your particular purposes because N_ELEMENTS being 0 has multiple meanings. In my case, I had two cases that I needed to handle:

  1. the number of elements in an array of objects
  2. determine if an argument was present (return nonzero value if it was an object with _overloadSize returning 0)

These issues have always been muddled in IDL (ARG_PRESENT doesn’t tell you if an argument is present, it indicates that it was passed as a named variable).

My manner of testing was to use OBJ_VALID if the argument was an object and you didn’t want to use the normal operator overloading returned value:

function mg_n_elements, var, no_operatoroverload=noOperatoroverload
  compile_opt strictarr

  if (~keyword_set(noOperatoroverload) || size(var, /type) ne 11) then begin
    return, n_elements(var)
  endif

  return, n_elements(obj_valid(var))
end

Download src.

IDL 8.4 introduces arbitrarily large integers with the BigInteger class:

IDL> n = BigInteger(2)^400
IDL> help, n
N               BIGINTEGER <ID=3 LENGTH=401 bits> = 2.582249878086...x10^120
IDL> print, n
2582249878086908589655919172003011874329705792829223512830659356540647622016
841194629645353280137831435903171972747493376

Standard arithmetic with operators such as +, *, etc. works fine and there are a few methods to perform some other common operations:

IDL> print, n->nextPrime()
2582249878086908589655919172003011874329705792829223512830659356540647622016
841194629645353280137831435903171972747493557
IDL> print, n->log2()
                   400

older posts »