IDL 8.4 adds a new routine, CODE_COVERAGE ([docs]), which returns information about the lines of a routine that have been executed. Using CODE_COVERAGE is fairly straight-forward — you do not need to enable code coverage. Just call CODE_COVERAGE at any time to find the lines of a routine that have been executed. Note that the routine must have been at least compiled before you call CODE_COVERAGE (even if you are clearing the status of the routine). Also, pay particular definition of the definition of a "line of code" in the [docs], e.g., empty lines, comments, and END statements do not count. Between the return value and the output from the EXECUTED keyword, you should get all the lines of code in a routine. CODE_COVERAGE adds another useful developer tool to the timing routines like PROFILER[^1], TIC and TOC. I think CODE_COVERAGE has a range of uses, but most interesting for me is the ability to determine the coverage of your unit test suite, i.e., how much of my code base is executed by my test suite? I have already implemented some basic test coverage information in my unit testing framework, [mgunit]. For example, mgunit can now tell me that I'm missing coverage of a few lines in the helper routines for MG_SUBS: "mg_subs_ut" test case starting (5 tests) test_basic: passed (0.000223 seconds) test_derived: passed (0.000354 seconds) test_derived2: passed (0.000369 seconds) test_derived_perverse: passed (0.000477 seconds) test_not_found: passed (0.000222 seconds) Test coverage: 90.5% Untested lines mg_subs_iter: lines 135 mg_subs_getvalue: lines 72-73, 79 Completely covered routines mg_subs Results: 5 / 5 tests passed, 0 skipped This means that after the unit tests have been run, line 135 from MG_SUBS_ITER and lines 72-73, 79 from MG_SUBS_GETVALUE have not been executed. This is useful (though not complete) information for determining if you have enough unit tests. Grab [mgunit] from the master branch on GitHub to give it a try (see [mglib] for an example of unit tests that take advantage of it). I'm not sure of the exact format for displaying the results, but I am fairly certain of the mechanism for telling the unit tests which routines it is testing (an ::addTestingRoutine method). I intend to start using this for the unit tests of my products [GPULib] and [FastDL] soon! [docs]: http://www.exelisvis.com/docs/CODE_COVERAGE.html "CODE_COVERAGE (IDL Reference)" [mgunit]: http://github.com/mgalloy/mgunit "mgalloy/mgunit" [mglib]: http://github.com/mgalloy/mglib "mgalloy/mglib" [GPULib]: http://www.txcorp.com/home/gpulib "GPULib" [FastDL]: http://www.txcorp.com/fastdl "FastDL" [^1]: There is also a CODE_COVERAGE keyword to PROFILER now that displays the number of lines of a routine that were executed.
IDL 8.4 was released today with a slew of new features. Check out [What's New in IDL 8.4] for a list of the new features. Most interesting feature for me: code coverage. I am going to explore this a bit to see if I can get [mgunit] to report what code has been tested (and, more importantly, not tested) by your test suite. Stay tuned for more detailed information about the new features! [What's New in IDL 8.4]: http://www.exelisvis.com/docs/WhatsNew.html "What's New in IDL 8.4" [mgunit]: https://github.com/mgalloy/mgunit "mgunit on GitHub"
Paulo Penteado has updated his [Building cross-platform IDL runtime applications] article with an über-installation for IDL 8.3 on all current platforms: > I created a package for IDL 8.3. It contains all the files in the current manifest_rt.txt file, which cover all the current platforms: Linux (x86_64), Windows (x86 and x86_64), Mac (x86_64) and Solaris (x86_64 and sparc64). The über-installation ([download]) allows MAKE_RT to make all-inclusive IDL runtime applications that work on all platforms. [Building cross-platform IDL runtime applications]: http://www.ppenteado.net/idl/idl71_rt.html "Building cross-platform IDL runtime applications" [download]: http://www.ppenteado.net/idl/idl83rt.tar.gz "IDL 8.3 uber-installation"
Greg Wilson gave a great [talk] about Software Carpentry at SciPy this year. I think more efforts like the Software Carpentry seminars are greatly needed in science — I've mentioned Software Carpentry [several] times [before]. If you are interested in teaching, he highly recommends the book *[How Learning Works]*. It gives a summary of the current research in learning with links to the primary sources. I wish I had that when I was teaching. via [Astronomy Computing Today] [talk]: https://www.youtube.com/watch?feature=player_embedded&v=1e26rp6qPbA "Software Carpentry: Lessons Learned | SciPy 2014 | Greg Wilson" [several]: http://michaelgalloy.com/2012/11/20/best-practices.html "Best practices" [before]: http://michaelgalloy.com/2011/01/07/software-development-for-scientists.html "Software development for scientists" [How Learning Works]: http://smile.amazon.com/How-Learning-Works-Research-Based-Principles/dp/0470484101/ref=sr_1_1?s=books&ie=UTF8&qid=1409338149&sr=1-1&keywords=how+learning+works "How Learning Works" [Astronomy Computing Today]: http://astrocompute.wordpress.com/2014/08/22/software-carpentry-lessons-learned-a-talk-at-sci-py2014/
![Maps of floating pastic][debris-map] The National Geographic has created new [maps] showing the extent of floating plastic in the ocean: > Tens of thousands of tons of plastic garbage float on the surface waters in the world's oceans, according to researchers who mapped giant accumulation zones of trash in all five subtropical ocean gyres. Ocean currents act as "conveyor belts," researchers say, carrying debris into massive convergence zones that are estimated to contain millions of plastic items per square kilometer in their inner cores. Two ships covered in the world in nine months to collect this data. via [FlowingData] [maps]: http://news.nationalgeographic.com/news/2014/07/140715-ocean-plastic-debris-trash-pacific-garbage-patch/ "First of Its Kind Map Reveals Extent of Ocean Plastic" [FlowingData]: http://flowingdata.com/2014/08/21/mapping-plastic-in-the-ocean/ "Mapping plastic in the ocean" [debris-map]: http://michaelgalloy.com/wp-content/uploads/2014/08/debris-maps-thumbnail.jpg
When writing even small applications, it is often necessary to distribute resource files along with your code. For example, images and icons are frequently needed by GUI applications. Custom color table files or fonts might be needed by applications that create visualizations. Defaults might be stored in other data files. But how do you find these files, when the user could have installed your application anywhere on their system? The answer is to place these files in a directory that you know the location of relative to your source code. Then use [MG_SRC_ROOT][^1] (or one of the other alternatives to it) to determine the location of your source code. Finally, use FILEPATH to specify the location of the resource. For example, in MG_LOADCT, I do the following to find the Brewer color tables, which are in the same directory as MG_LOADCT: ctfilename = filepath('brewer.tbl', root=mg_src_root()) If I had placed the color tables in a *resources* subdirectory which was parallel to the *code* directory my source code lives in, I could just use the SUBDIR keyword: ctfilename = filepath('brewer.tbl', subdir=['..', 'resources'], $root=mg_src_root()) MG_SRC_ROOT is one of my most used routines[^2]. Get all the source code for [mglib] on GitHub. [^1]: Before the SCOPE_TRACEBACK routine was introduced in IDL 6.2, MG_SRC_ROOT had to do ugly things like parse the output from HELP. [^2]: I count 89 uses of MG_SRC_ROOT in my library mglib and it is also used in [IDLdoc] and [mgunit]. [MG_SRC_ROOT]: https://github.com/mgalloy/mglib/blob/master/src/dist_tools/mg_src_root.pro "mglib/mg_src_root.pro at master" [mglib]: https://github.com/mgalloy/mglib "mgalloy/mglib" [IDLdoc]: https://github.com/mgalloy/mglib "mgalloy/idldoc" [mgunit]: https://github.com/mgalloy/mglib "mgalloy/mgunit" ExelisVIS annonuced VISualize 2014 will focus on the following topics: >Presentations and discussions will focus on topics such as: > >* Using new data platforms such as UAS, microsatellites, and SAR sensors >* Remote sensing solutions for precision agriculture >* Drought, flood, and extreme precipitation event monitoring and assessment >* Wildfire and conservation area monitoring, management, mitigation, and planning >* Monitoring leaks from natural gas pipelines See the [video] for more information and then [register] or [submit an abstract]. UPDATE 9/18/14: postponed until 2015. [video]: https://www.youtube.com/watch?v=UGQhDxlyMC4 "Getting Ready for VISualize 2014" [register]: http://www.cvent.com/events/mproc.aspx?m=ebe94a38-b068-48c3-b826-99253ae0c926&u=http%3a%2f%2fexelisvis.cvent.com%2fevents%2fvisualize-2014%2fevent-summary-07ced8b3b24a40ceb6edb49352b5f790.aspx&l=%3cimg+width%3d%22153%22+height%3d%2232%22+alt%3d%22%22+style%3d%22border%3a+0px+solid%3b%22+src%3d%22http%3a%2f%2fwww.exelisvis.com%2fportals%2f0%2fimages%2fregister_now_btn.gif%22+width%3d%22153%22+height%3d%2232%22+%2f%3e [submit an abstract]: http://www.cvent.com/events/mproc.aspx?m=ebe94a38-b068-48c3-b826-99253ae0c926&u=http%3a%2f%2fexelisvis.cvent.com%2fevents%2fvisualize-2014%2fevent-summary-07ced8b3b24a40ceb6edb49352b5f790.aspx&l=%3cimg+width%3d%22247%22+height%3d%2232%22+alt%3d%22%22+style%3d%22border%3a+0px+solid%3b%22+src%3d%22http%3a%2f%2fwww.exelisvis.com%2fportals%2f0%2fimages%2fsubmit_abstract_btn.png%22+width%3d%22247%22+height%3d%2232%22+%2f%3e I've been dealing with [HDF 5] files for quite awhile, but IDL interface was as painful as the C interface. It did have H5_BROWSER and H5_PARSE to make things a bit easier, but these utilities are relevant for interactive browsing of a dataset and not for efficient, programmatic access. I created a [set of routines] for dealing with HDF 5 files that I have been extending as needed to other scientific data formats such as netCDF, HDF 4, and IDL Savefiles. [HDF 5]: http://www.hdfgroup.org/HDF5/ "HDF Group - HDF5" [set of routines]: http://michaelgalloy.com/2008/06/27/some-hdf5-helper-routines.html "Some HDF5 helper routines" For example, MG_H5_GETDATA can access variables stored in HDF 5 files: f = filepath('hdf5_test.h5', subdir=['examples', 'data']) arr = mg_h5_getdata(f, '/arrays/3D int array') Or slices of variables: slice = mg_h5_getdata(f, '/arrays/3D int array[3, 5:*:2, 0:49:3]') Or attributes: attr = mg_h5_getdata(f, '/images/Eskimo.CLASS') Similarly, MG_H5_PUTDATA can create and edit variables in HDF 5 files, while MG_H5_DUMP prints a listing of the variables and attributes of a file. I have adding corresponding routines for HDF 4, netCDF, and IDL Savefiles over the past few years. Since I have been dealing netCDF files more extensively recently, I have been adding to their routines, most notably I have created a LIST routine which provides an array of variable/attribute names available in a file. Here's the routines available in [mglib] currently: | Action | HDF 4 | HDF 5 | netCDF | Savefile | ---------- | ---------------- | --------------- | --------------- | ----------------- | GETDATA | MG_HDF_GETDATA | MG_H5_GETDATA | MG_NC_GETDATA | MG_SAVE_GETDATA | PUTDATA | MG_HDF_PUTDATA | MG_H5_PUTDATA | MG_NC_PUTDATA | | DUMP | MG_HDF_DUMP | MG_H5_DUMP | MG_NC_DUMP | MG_SAVE_DUMP | LIST | | | MG_NC_LIST | [mglib]: https://github.com/mgalloy/mglib "mgalloy/mglib" Recently, I have been writing a fairly large and generic system for ingesting various satellite images onto a common grid and producing user specified plots and reports from the results. Control of the system is done via a [configuration file], like [this one], which has been a great, flexible way to handle users extending and controlling the system. But reading the recent IDL Data Point article about Jim Pendleton's [DebuggerHelper] class reminded me how useful a logging framework is for medium to large sized projects. I use [mg_log] as my logging utility. It is simple to use, but has some powerful features for filtering and customizing output. It has five levels (debug, informational, warning, error, and critical) of messages which match the overall system level. This allows you to filter messages based on severity, e.g., during development you can set the logging level to "debug" and then all messages will appear. Later, when you have deploy the system, users may find setting the level to "warning" (which does not show the debug and informational messages) to be more appropriate. See [this article] to learn more about mg_log basics. [This sample log] shows what the typical output looks like, though the format for each line is completely configurable. mg_log is available on GitHub in my [mglib] repo. [this one]: http://michaelgalloy.com/wp-content/uploads/2014/08/viirs_small.ini "Sample configuration file" [configuration file]: http://michaelgalloy.com/2013/07/12/reading-configuration-files.html "Reading configuration files" [DebuggerHelper]: http://www.exelisvis.com/Company/PressRoom/Blogs/IDLDataPointDetail/TabId/902/ArtMID/2926/ArticleID/14005/DebuggerHelper---A-Handy-Debugging-Class-for-IDL-Developers.aspx "DebuggerHelper - A Handy Debugging Class for IDL Developers" [mg_log]: http://michaelgalloy.com/2011/11/17/logging.html "Logging" [this article]: http://michaelgalloy.com/2011/11/17/logging.html "Logging" [mglib]: https://github.com/mgalloy/mglib/ "mgalloy/mglib" [This sample log]: http://michaelgalloy.com/wp-content/uploads/2014/08/viirs_small.log "mg_log output" I have had multiple occasions where I needed to quickly generate bindings to an existing C library. The repetitive nature of creating these bindings calls out for a tool to automate this tool. For this purpose, I have written a class, MG_DLM, that allows: 1. creating wrapper binding for routines from a header prototype declaration (with some limitations from standard C) 2. creating routines which access variables and pound defines 3. allow adding custom routines written by the developer I have used MG_DLM to create bindings for the [GNU Scientific Library] (GSL), [CULA], [MAGMA], and even IDL itself. [GNU Scientific Library]: http://www.gnu.org/software/gsl/ "GSL - GNU Scientific Library" [CULA]: http://www.culatools.com "CULA" [MAGMA]: http://icl.cs.utk.edu/magma/ "MAGMA" #### Define the routines For our example, let's make some basic random number generation routines from GSL available. The relevant routines from the *gsl_rng.h* header file that we would like to access from IDL are: const gsl_rng_type *gsl_rng_env_setup(void); gsl_rng *gsl_rng_alloc(const gsl_rng_type *T); INLINE_DECL double gsl_rng_uniform(const gsl_rng *r); We need to write a modified header file that MG_DLM will be able to understand. Since the pointers to *gsl_rng_type* and *gsl_rng* are returned by one routine and passed into another, we can just define them as IDL_PTRINT. For example, GSL_RNG_ALLOC will just return a 64-bit integer that represents a pointer to a *gsl_rng*. This integer can then be passed to GSL_RNG_UNIFORM. Our header will be: void gsl_rng_env_setup(); IDL_PTRINT gsl_rng_alloc(IDL_PTRINT t); double gsl_rng_uniform(IDL_PTRINT r); We place these definitions in *mg_gsl_rng_bindings.h* for use when we create the DLM object. #### Create the DLM Next, we use MG_DLM to define our DLM containing these bindings. First, create a DLM object with basic metadata: dlm = mg_dlm(basename='mg_gsl',$ prefix='MG_', $name='mg_gsl',$ description='IDL bindings for GSL', $version='1.0',$ source='Michael Galloy') Next, tell the DLM where the include directory and library are: dlm->addInclude, 'gsl/gsl_rng.h', $header_directory='/usr/local/include/gsl' dlm->addLibrary, 'libgsl.a',$ lib_directory='/usr/local/lib', \$ /static dlm->addRoutinesFromHeaderFile, 'mg_gsl_rng_bindings.h' For our example, we will also need to access the *gsl_rng_default* variable in the GSL library. In *gsl_rng.h*, it is declared as: GSL_VAR const gsl_rng_type *gsl_rng_default; To add a routine that accesses this variable, we just do: dlm->addVariableAccessor, 'gsl_rng_default', type=14L Finally, tell the DLM to write itself, both the *.c* and *.dlm* files, and then build itself: dlm->write dlm->build We are now ready to use our DLM. #### Example of using We can now use our routines to generate random numbers from IDL: n = 1000L mg_gsl_rng_env_setup t = mg_gsl_rng_default() r = mg_gsl_rng_alloc(t) u = dblarr(n) for i = 0L, n - 1L do u[i] = mg_gsl_rng_uniform(r) Get the code for creating bindings (and a slightly more fleshed out set of GSL bindings) from the [mglib repo]. [mglib repo]: https://github.com/mgalloy/mglib "mgalloy/mglib"

older posts »

• GPULib

GPULib enables IDL developers to access the high-performance capabilities of modern NVIDIA graphics cards without knowledge of CUDA programming.

TaskDL is a task-farming solution for IDL designed for problems with loosely-coupled, parallel applications where no communication between nodes of a cluster is required.

mpiDL

mpiDL is a library of IDL bindings for Message Passing Interface (MPI) used for tightly-coupled parallel applications.

Remote Data Toolkit

The Remote Data Toolkit is a library of IDL routines allowing for easy access to various scientific data in formats such as OPeNDAP, HDF 5, and netCDF.

• Modern IDL

Modern IDL offers IDL programmers one place to look, for beginners and advanced users alike. This book also contains: a thorough tutorial on the core topics of IDL; a comprehensive introduction to the object graphics system; common problems and gotchas with many examples; advanced topics not normally found are discussed throughout the book: regular expressions, object graphics, advanced widget programming, performance, object-oriented programming, etc.

• IDLdoc

IDLdoc is an open source utility for generating documentation from IDL source code and specially formatted comments.

mgunit

mgunit is an open source unit testing framework for IDL.

rIDL

rIDL is an open source IDL command line replacement.

mglib

mglib is an open source library of IDL routines in areas of visualization, application development, command line utilities, analysis, data access, etc.