For some time, I have had a need for an easy way to make use of all of the cores of a single machine through multiple threads or processes, i.e., not a SIMD/vectorized paradigm. The IDL_IDLBridge is capable of doing this, but setup and usage is fairly painful. To make it easier, I have created a simple multicore library for IDL.

The simplest example is to create a process to execute an IDL command and then meet up with the main thread, like the mg_process_join_demo in the examples:

p = mg_process(name='subprocess', output='')
p->execute, 'mg_process_join_demo', /nowait
; main process free to do other stuff at this point
p->join ; main process and subprocess will meet here

Another example uses a pool class to create a pool of processes that work together to perform their part of a larger task. In this example, we will compute x^2 + y^3 for 100 values[1] using a pool with the number of processes corresponding to the number of cores available. We need a function to apply to each element of our x and y arrays:

function mg_pool_map_demo, x, y
  compile_opt strictarr

  return, x^2 + y^3
end

Now, we create a pool with the default number of processes being the number of cores available:

pool = mg_pool()

Create our datasets:

n = 100L
x = findgen(n)
y = 0.5 * findgen(n)

Then just pass our data to MG_Pool::map and indicate which function to apply to the data:

x_squared = pool->map('mg_pool_map_demo', x, y)

This will block, i.e., it returns control to the main process when finished processing. At this point, the result is computed:

print, x_squared, format='(8(F10.2))'

I have used this library to get fairly good performance on an 8 core laptop doing some image processing operations on a few hundred image files, as shown in the graph below.

Scaling

  1. This, of course, is not enough work to make this a good idea for performance and is only intended as a fast and simple example of using the library. ??