FLOSS Weekly just talked to Jeff Squyres about Open MPI. Open MPI is one implementation of the message passing interface (MPI) standard. Open MPI can be used with mpIDL to use MPI within IDL. Particularly interesting was the discussion about connections with GPU computing (we recently made an mpiDL/GPULib example which ran on a “personal supercomputer” with three Tesla graphics cards).

FLOSS Weekly is a approximately weekly show about open source projects hosted by Leo Laporte and Randall Schwartz. Coming up in the next month is a show about Processing!

Full disclosure: I work for Tech-X Corporation and worked on the IDL bindings and examples for mpiDL and GPULib.

GPULib 1.0 was announced yesterday.

If you have been using the pre-release, the biggest new feature of the IDL bindings are the function forms of the routines. For example, it is possible to do:

IDL> gpuinit
IDL> x = findgen(360) * !dtor
IDL> y = 2 * findgen(360) * !dtor
IDL> z_gpu = gpusin(gpumult(x, y)) ; z = sin(x * y)
IDL> plot, gpugetarr(z_gpu)
IDL> gpuFree, z_gpu

This computes z = sin(x *y) in a much more readable form than the procedural interface and does not leak any memory. In general, there are two bottlenecks to fast GPU computation: allocating memory on the GPU and transfer between CPU and GPU. Keeping results on the GPU and doing many calculations before transferring a final result to the CPU is desirable. Also, reusing variables for multiple calculations can help. This was straight-forward to do using the procedure interface, but how to specify a return variable to a function? Use the LHS keyword. For instance, the above calculation could be done using a pre-defined z_gpu:

IDL> gpuinit
IDL> x = findgen(360) * !dtor
IDL> y = 2 * findgen(360) * !dtor
IDL> z_gpu = gpuFltarr(360)
IDL> z_gpu = gpusin(gpumult(x, y, LHS=z_gpu), LHS=z_gpu)
IDL> plot, gpugetarr(z_gpu)
IDL> gpuFree, z_gpu

It is possible to program as efficiently as the procedure forms with the function forms of the routines by using LHS, so make good use of them!

Full disclosure: I work for Tech-X Corporation and worked on the IDL bindings and examples for GPULib.

Here’s the difference between computations done on the CPU (like plain IDL) vs. those done on the GPU (like with GPULib) painted out by the Mythbusters crew at the NVISION conference this week.

Link via kottke.org.

Mort Canty, author of Image Analysis, Classification and Change Detection in Remote Sensing: With Algorithms for ENVI/IDL, is using GPULib and has started blogging his comments on it. So far, so good!

Full disclosure: I work for Tech-X Corporation and worked on the IDL bindings and examples for GPULib.

In this recent IT Conversations interview, Jon Udell discusses topics from Greg Wilson’s recent talk “High-Performance Computing Considered Dangerous”. Wilson places a new emphasis on testing and usability over more power and speed.

The interview focused on the difficulties of learning techniques in High-Performance Computing and, in general, about the split between scientific computing and “commercial” software development. Tools have to improve because there is no more time in the science curriculum to teach science graduate students more about software engineering practices. In particular, I thought the comments about version control aiding in the reproducibility of computations were interesting (version control as a “software lab notebook”). He also mentions that tools like “Matlab, IDL, and Mathematica” are scoffed at by the HPC gurus, but are the tools that many scientists are using to solve their problems (he put Excel in a category of popularity above these). Our focus should be adding easy to use HPC techniques to these platforms.

A pre-release of GPULib was made available today. GPULib is a library of common scientific computational routines that use modern graphics processing units (GPUs) to improve performance. I saw speedups of 5-50 times faster than normal IDL code for example code I was writing. Vectorized code that is still not fast enough can be improved with GPULib.

Bindings for Java, Python, MATLAB, and IDL are available. The IDL bindings are fairly simple to use and have a compatibility mode that allows running the code even without a GPU (performance was approximately equivalent to normal IDL code for the examples I wrote).

The documentation lists all the routines available (IDLdoc 3.0b3 spotted in the wild!).

Full disclosure: I work for Tech-X Corporation and worked on the IDL bindings and examples for GPULib.

FastDL 2.4 was released today. FastDL provides a way to do cluster computing from within IDL in two different flavors: mpiDL and TaskDL. For problems which require communication between nodes, mpiDL provides an interface to the MPI standard. If you are familiar with MPI, then mpiDL should be very easy to use. It can now be used with MPICH, MPICH2, or OpenMPI implementations of MPI. For problems where nodes do not need to communicate with each other, TaskDL is a task farming solution for IDL.

Full disclosure: I work for Tech-X Corporation and did some work on this project.