IDL type promotion challenge
posted Fri 29 Aug 2008 by Michael Galloy under IDLI’m looking for a better solution to a sticky question. I had a fairly short “clever” solution until I realized it wasn’t correct in some cases i.e. it was wrong.
I’m creating a function to return the type code of the variable necessary to contain the precision of variables of the two input type codes passed to the routine. I’m using the chart to the right as the order of precision for the types. This is not what IDL uses for it’s promotion; the integer types don’t promote to a floating point type unless one of the types is already a floating point type.
For example, if the inputs are byte (type 1) and integer<br />(type 2), then an integer can hold them both:
IDL> print, mg_convert_type(1, 2)
2
But if the inputs are a double (type 5) and a complex (type 6), it takes a double complex (type 9) to hold them both:
IDL> print, mg_convert_type(5, 6)
9
In particular, the conversions between the signed and unsigned integer types caused my previous solution problems i.e. long and ulong64 produce float, but long and ulong produce long64.
Any clever solutions? Currently, I’m using a 16 by 16 lookup table with the precomputed answers.
August 29th, 2008 at 1:00 pm
I’m confused. In your diagram you have a long64 and a ulong64 promoting to a float? How can you “contain the precision” of a 64 bit type within a 32 bit type? Information will necessarily be lost.
In fact, you can’t promote a 64 bit integer to a double either, because of the way the ieee format is designed. If you want to promote 64 bit ints to floating point numbers you need to use double double’s but I don’t think those are available in IDL.
So long & ulong need to promote to double or long64. There is no idl type that can hold a long64 and a ulong64 without loss of information.
I think a lookup table is fine, but Maybe what you should do to solve this problem is write a BigInt or Double Double implementation as either a library in C or in Native IDL. IDL really needs this. It is available in other languages like python.
August 29th, 2008 at 1:08 pm
Yes, precision will be lost. The reason I need this is for visualization of images (of differing types), so the demands for precision are not as strict. I actually thought about converting everything to byte. Maybe I should say “contains the range”?
I know of a BigInt implementation for IDL, but not sure if it is available. Let me check on that.
August 29th, 2008 at 3:13 pm
Ya if you know about a BigInt, I’d be curious to see it.
“Contain the range” or maybe phrase it in terms of intervals.
I’m not sure the problem you mention is a problem though. A long & ulong64 should produce a float. If the ulong64 is greater than 2^63-1, then a long64 has insufficient range. In the same way, if a ulong is larger than 2^32-1 then we need to promote it. If you are concerned about these cases having different outcomes, why not always promote to float? Since you’re not stressing precision you can dump integral types.(Well maybe keep the 16 & 8 bits ones)
I like the idea of a lookup, it will probably be faster than alternatives. You could consider using a recursive function that traces out the tree in your diagram, it might be slower though.
September 4th, 2008 at 12:59 pm
Ron Kneusel is releasing his BigInt and arbitrary precision floating point packages to the ITT VIS Code Contrib site.
September 8th, 2008 at 1:57 pm
Awesome, thanks!
October 13th, 2008 at 3:16 pm
OK, Ron has two DLMs on the ITT VIS code contrib site. There doesn’t seem to be a way to give a link to their pages, but they are titled “Infinite precision integer and rational numbers” and “Arbitrary precision floating point”.