Igor Chilingarian and Ivan Zolotukhin open their recent paper on the botteneck in scientific computing with:
Present-day astronomical instruments and large surveys produce the data flow increasing exponentially in time. The CPU power required to analyse these data is also growing with the same pace following the Moore’s law; the same applies to the data storage volume per price unit. However, in astronomy we do not see the exponential avalanche of scientific results produced with this computational power.
After providing good and bad examples of code in projects and discussing issues with bad code, they conclude:
It turns out that all “good examples” were developed either by professional astronomers with very strong IT/CS background or by IT/CS professionals working closely with astronomers for years and understanding astronomy. One cannot simply hire an industrial software engineer to develop astronomical software and/or an archive and/or a database.
A possible solution is to change the teaching paradigm for students in astronomy. Basic courses in algorithms, programming, software development and maintenance have to be made mandatory in the education of modern astronomers and physicists; advanced courses should be recommended to some of them. The Fortran language is now obsolete and we have to accept this. Instead of teaching research students to Fortran programming, one should teach how to interface legacy Fortran code in C/C++.
As soon as this bottleneck is resolved, the avalanche of discoveries will loom.
I agree. What to do with all the existing astronomers, though? I don’t think it is too late to learn good practices for those interested.
Link via Astrocompute.