For some time, I have had a need for an easy way to make use of all of the cores of a single machine through multiple threads or processes, i.e., not a SIMD/vectorized paradigm. The `IDL_IDLBridge` is capable of doing this, but setup and usage is fairly painful. To make it easier, I have created a [simple multicore library][multiprocessing] for IDL.
[multiprocessing]: https://github.com/mgalloy/mglib/tree/master/src/multiprocessing "mglib/src/multiprocessing"
The simplest example is to create a process to execute an IDL command and then meet up with the main thread, like the `mg_process_join_demo` in the [examples]:
p = mg_process(name='subprocess', output='')
p->execute, 'mg_process_join_demo', /nowait
; main process free to do other stuff at this point
p->join ; main process and subprocess will meet here
Another example uses a pool class to create a pool of processes that work together to perform their part of a larger task. In this example, we will compute `x^2 + y^3` for 100 values[^1] using a pool with the number of processes corresponding to the number of cores available. We need a function to apply to each element of our `x` and `y` arrays:
function mg_pool_map_demo, x, y
return, x^2 + y^3
Now, we create a pool with the default number of processes being the number of cores available:
pool = mg_pool()
Create our datasets:
n = 100L
x = findgen(n)
y = 0.5 * findgen(n)
Then just pass our data to `MG_Pool::map` and indicate which function to apply to the data:
x_squared = pool->map('mg_pool_map_demo', x, y)
This will block, i.e., it returns control to the main process when finished processing. At this point, the result is computed:
print, x_squared, format='(8(F10.2))'
I have used this library to get fairly good performance on an 8 core laptop doing some image processing operations on a few hundred image files, as shown in the graph below.
[examples]: https://github.com/mgalloy/mglib/tree/master/examples/multiprocessing "mglib/examples/multiprocessing"
[^1]: This, of course, is not enough work to make this a good idea for performance and is only intended as a fast and simple example of using the library.