Exelis VIS is now accepting abstracts—due May 18! Not sure what I should talk about this year (I’ve talked about GPULib for the past couple of years), but I will definitely submit something and attend in any case.

I will definitely have to work this into an IDLdoc release soon.

I agree with most of their strengths/weaknesses lists for the tools, although I’m not sure why they say version control systems like Subversion or git are “intolerant of simultaneous edits”. Merging is quite good in these system unless two people edit the exact same line of a file. Usually collaborators have different areas of responsibility in the document and conflicts would only occur occasionally if people are doing global actions like proof-reading the entire document at the same time. Even then, the conflicts are detected by the system and can be manually corrected by the later committer.

I favor a Subversion/git repo for doing any serious work (although I do highly recommend Dropbox for sharing/syncing files, just not for collaborative writing). I use Subversion repos for all my proposals, although I am beginning experiments with git for certain internal proposals, papers, and other writing. I used a Subversion repo for Modern IDL. It’s flexible enough to handle your writing output (not the best for a single monolithic Word file, but great for any text based system like LaTeX), while also handling other related assets like code, images, diagrams, slides, etc. well. Data is stored locally, but also backed up on a server¹.

I have reservations about providing proprietary/sensitive/private data to a third-party service. Also, relying on a cloud service (especially like relative newcomers like ScribTeX and ShareLaTeX) might be disastrous if there was an outage just before a proposal deadline².

I’m retiring. No, really, this time I’m really retiring. :-)

Hopefully this is a false alarm, but in any case, good luck David and thanks for years of valuable information!

For example, Python’s doctest module provides a solution to these problems for Python. In your docs, you give a command and its expected output:

>>> [factorial(n) for n in range(6)]
[1, 1, 2, 6, 24, 120]

Then, in the main-level program, you say something like (I’ve omitted some Python details for simplicity here):

doctest.testmod()

When you do the equivalent of running the main-level program, the docs are checked for “>>>” lines which are executed and compared to the output shown in the docs.

It seems like there are several solutions that could be implemented in IDLdoc or mgunit to solve this problem.

1. Create a doctest routine for IDL. This would mean to create a routine like the dockets.testmod() routine in doctest that runs all the lines in the documentation that begin with “IDL>” and compares the results to the output shown below the “IDL>” line.

   Advantages: Can be used to easily run tests and compare results to provided output. Test commands and output are included in documentation, but are not present twice in the file.

   Disadvantages: None?

2. Conversely, create an IDLdoc include directive, like ..include:: [main-level] that includes the contents of the main-level of the current file. This could be used in the examples section of the docs to include the main-level program.

   Advantages: Could also be used to include other files which might be useful in other situations.

   Disadvantages: For files with multiple routines in a file, the entire main-level program would be included at the location, even if the main-level program addressed multiple of the routines. Doesn’t help with testing.

3. Allow IDLdoc to do the testing by creating a tests section that would be executed when IDLdoc is run.

   Advantages: Can be used to easily run tests and compare results to provided output. Test results appear in the IDLdoc output.

   Disadvantages: Need to run IDLdoc to run the tests.

Once nice thing here is that several of these strategies could be implemented. Any ideas, suggestions on what you would like to see? Creating an MG_DOCTEST routine seems like the strongest solution right now, but am I missing something?

percent cumulative  self
time     seconds   seconds  name
18.20      0.02     0.02    main (cuda-blas.cu:94 @ 401502)
 9.10      0.03     0.01    main (cuda-blas.cu:93 @ 4014c9)
 9.10      0.04     0.01    main (cuda-blas.cu:166 @ 4018c0)
 9.10      0.05     0.01    main (cuda-blas.cu:239 @ 401c3d)
 9.10      0.06     0.01    main (cuda-blas.cu:243 @ 401c91)
 9.10      0.07     0.01    main (cuda-blas.cu:318 @ 402039)
 9.10      0.08     0.01    main (cuda-blas.cu:319 @ 402060)
 9.10      0.09     0.01    main (cuda-blas.cu:322 @ 4020a1)
 9.10      0.10     0.01    main (cuda-blas.cu:321 @ 4020f0)

I wrote an IDL routine that takes the raw profile output along with the source code and creates HTML output that color codes the lines with high activity, like the following:

Get the code from my IDL library, available via Subversion:

svn co http://svn.idldev.com/idllib/trunk idllib

The mg_clineprofile.pro file is in the src/profiling directory. Call MG_CLINEPROFILE like:

IDL> mg_clineprofile, 'profile-output.txt', /all_files

The ALL_FILES keyword indicates you want output for each file listed in the profile output; you can also specify the files that you want output for via the FILES keyword.

The code is currently pretty ugly; expect changes to it coming up.

# Disable menu bar transparency
defaults write NSGlobalDomain AppleEnableMenuBarTransparency -bool false

# Disable the warning when changing a file extension
defaults write com.apple.finder FXEnableExtensionChangeWarning -bool false

# Show the ~/Library folder
chflags nohidden ~/Library

This is like a nerdy command line version of Secrets.

Pathline is a visualization tool for genetic data created in collaboration between designers and geneticists:

We recently attended an interdisciplinary visualization workshop that was all about creating a dialogue between scientists, technologists and designers. It was interesting to discuss the different ways in which these groups think about visualization and how they use it for different purposes. Very bluntly put, each group lacks something another group knows and cares deeply about, be it an understanding of colour [we met in the UK] or an understanding of statistics.

The before and after images show a striking simplification that apparently helps even trained geneticists:

Two main advantages of the new tool were found. First, there was a massive gain in efficiency. The study of a heatmap took up to a half-hour before but can be done at a glance at the curvemap now. More importantly, though, the scientists made new discoveries of gene properties they didn’t know about before. What was hidden in the data before, is now very clear, even to an untrained eye.

IDL doesn’t provide any way to access or manage OpenGL contexts. IDL Object Graphics does use OpenGL to do its drawing though, so there must be valid OpenGL contexts floating around in there somewhere. It’s a reasonable assumption (and testing shows it to be true) that when the Draw method of an IDLgrModel object is called, the current OpenGL context is set to the window into which the model should be drawn.

This is from one of a series of articles about using this trick along with Slither to have PyMOL draw into an IDL object graphics window.

I’ve been trying to get a way to get an OpenGL context for awhile. I definitely will be keeping an eye out for future articles from Jacquette.

Methods for producing scientific illustrations—and for reproducing them in publications—have been changing. Printing plates for figures were once engraved by hand, then made by a photographic process, and in recent years have been created by digital techniques. Now we are about to turn the page—if not close the book—on yet another chapter in publishing history. After centuries of reading and writing on paper, we seem to be headed for a world where most documents will be distributed online and viewed on a display screen of some kind. How will this transition to a new medium affect the practice of scientific illustration?

via FlowingData

But, while rich in depth and breath, the PDS databases have developed in a disparate fashion over the years with different architectures and formats for different scientific needs; thereby making acquisition of data problematic!

So, NASA is holding a series of Challenges to generate some simply awesome ideas for mobile or web based applications that will appeal to general users, to search and display compelling facts about the data. Instead of just scientists, our audience will be the millions of school age students, their teachers and parents, game designers and general civilians of the world. We want to deliver this incredible data to users in a way that excites them – and thus, to help them understand the value and potential of this data.

Prizes range from $500 to $10,000, as well as being named “Space Coder of the Galaxy 2012″. Some contests are restricted to teachers or high school students.

These images show a line-integral convolution (LIC) representation of the global NASA wind velocity data. The first image shows the LIC output, while the second and third images combine the LIC output with the Earth image in the IDL distribution. The fourth image shows the combination texture mapped on a sphere representing the Earth’s surface. Dark colors represent low velocity winds, lighter colors higher velocity winds. All aspects of the creation of the images were done in IDL. The code to compute the LIC was written in C and integrated with IDL using a DLM. The data access was done from NASA’s DAP server using OPeNDAP’s client IDL bindings (another DLM).

Images are retina sized, i.e., iPhone 4/4S only.

This is currently a proof of concept and works for views, models, and polygons with simple properties only, but expanding to more classes and properties should be straightforward. I have done similar things for various other new destination classes for outputs such as POV-Ray and anaglyphs.

This issue has been a problem for me before so I wrote a simple arithmetic parser (docs).

Evaluates a mathematical expression using the basic arithmetic operators +, -, *, /, and ^ along with parentheses for grouping and simple function calls of a single variable.

This routine is implemented with a recursive descent parser and does not use EXECUTE, so it is safe to use in the Virtual Machine.

For example, simple arithmetic expressions can be evaluated:

IDL> print, mg_evalexpr('1 + 2 + 3', error=error), error
                     6       0

Note that the ERROR keyword returns whether there was an error in evaluating the expression. Expressions can also take variables, if their values are provided via a structure or hash-like object:

IDL> print, mg_evalexpr('exp(i * pi)', { pi: !dpi, i: complex(0, 1) })
(      -1.0000000,   1.2246468e-16)

“Hash-like” here means that the object implements a hasKey method and subscripting via brackets like the IDL 8.0 Hash class.

1. Is it possible to dynamically resize a sparse matrix in the Petsc library?
2. Future of OpenCL?
3. Parallel I/O options, in particular parallel HDF5
4. Is it worthwhile to write unit tests for scientific research codes?

Currently, IDLdoc is built with IDL 6.4 to maximize who can use it, but this means any included IDL library routines could conflict with the routines provided by the user’s IDL distribution. IDLdoc is intended to be run from the command line, i.e., a full version of IDL, not a runtime or VM environment, so the IDL library routines should be provided by the user’s IDL distribution.

Get the new version here.

On-line documentation is now available for all 41 of the Coyote Graphics programs. (These are a subset of the programs in the Coyote Library.) This will make it much easier to use these Coyote Graphics programs.

Also, I have confirmation that IDLdoc will run on big libraries, i.e., over 4000 files, 7700 routines, and 850,000 lines of code!

Here’s our press release:

Tech-X Corporation invites you to visit our booth (Booth #1748) at the American Geophysical Union Fall Meeting 2011, December 5 – 9, at the Moscone Center in San Francisco.

We will be featuring GPU Computing, including GPULib, our library of mathematical kernels for GPU computing.

We will also be demonstrating the Remote Data Toolkit, which allows seamless access to remote data with IDL.

Personnel from Tech-X and/or their collaborators will be participating in the following activities:

Session A53C: Multisensor and Model Aerosol Data Inter-comparison and Synergy III Posters Friday, December 09 1:40PM – 6:00PM, Halls A – C

A53C-0382 Aerosol type estimations for the ERBE period (1985 – 1989) Jose R. Fernandez, SSAI, Hampton, VA; Seiji Kato, NASA Langley Research Center, Hampton, VA; Fred G. Rose, SSAI, Hampton, VA; David W. Fillmore, Tech-X Corporation, Boulder, CO

Exelis VIS will be at booth #1431; I haven’t heard about if there will be an IDL user group meeting.

UPDATE: Also, if you have an iPhone, I recommend the AGU app. It has been very useful already for me.

Full disclosure: I work for Tech-X Corporation.