Topic
• 30 replies
• Latest Post - ‏2015-12-10T10:09:10Z by Adamarla
jester76
68 Posts

# Pinned topic real in DXL

‏2013-04-24T17:27:26Z | . 6 99.0000000000 a after digits example for get is more than there to way
• llandale
3035 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-04-24T20:48:29Z
• jester76
• ‏2013-04-24T18:42:59Z

The "real" type has a lot of precision, but when you print it only 6 decimal digits are displayed.  Some time ago I posted a "fStringOfReal()" function which clumsily handled it; other folks posted more elegant solutions.

-Louie

• jester76
68 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-04-24T18:42:59Z
• llandale
3035 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-04-24T20:48:29Z
• jester76
• ‏2013-04-24T18:42:59Z

The "real" type has a lot of precision, but when you print it only 6 decimal digits are displayed.  Some time ago I posted a "fStringOfReal()" function which clumsily handled it; other folks posted more elegant solutions.

-Louie

• Wolfgang Uhr
339 Posts

#### Re: real in DXL

‏2013-04-25T07:28:43Z

As you can see in the attachment (I hope the upload will work):

I do not know what you are speaking about, but it sounds interesting ...

• jester76
68 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-04-25T15:34:28Z
• llandale
• ‏2013-04-24T20:48:29Z

The "real" type has a lot of precision, but when you print it only 6 decimal digits are displayed.  Some time ago I posted a "fStringOfReal()" function which clumsily handled it; other folks posted more elegant solutions.

-Louie

Thanks LLandale. Your function worked great for me. Sorry I had to post everything in the subject line as site wouldn't let me edit the body of the post.

• jester76
68 Posts

#### Re: real in DXL

‏2013-04-25T15:37:51Z

As you can see in the attachment (I hope the upload will work):

I do not know what you are speaking about, but it sounds interesting ...

When you print or assign real to object in DXL it only prints/assigns 6 digits after decimal points. So the function will help you go beyond 6 decimal points.

• jester76
68 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-10T21:58:49Z
• llandale
• ‏2013-04-24T20:48:29Z

The "real" type has a lot of precision, but when you print it only 6 decimal digits are displayed.  Some time ago I posted a "fStringOfReal()" function which clumsily handled it; other folks posted more elegant solutions.

-Louie

Hello,

I am facinng an issue here. When supplying a value of 0.0000009999 to the function. It returns 0.0000009998. Can you please check why this is happening.

Thanks

Jester

• llandale
3035 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-11T15:27:12Z
• jester76
• ‏2013-06-10T21:58:49Z

Hello,

I am facinng an issue here. When supplying a value of 0.0000009999 to the function. It returns 0.0000009998. Can you please check why this is happening.

Thanks

Jester

"real" is by nature imprecise.

1.  On a computer, [a] "2*2" is not equal to [b] "2^^2" (exponent).  [a] uses integers which are precise, [b] converts to real which is not.

2.  You cannot accurately express "1/3" in a real number.  Looks like "0.33333333333333333" which is very close but not quite.

When I run this code:

• void Test(real r)
• { print fStringOfReal(r, 0,true) "\t" fStringOfReal(r, 10,true) "\t[" (r*1000000000000.0) "]\n"
• }
• Test(0.00000099)
• Test(0.000000999)
• Test(0.0000009999)
• Test(0.00000099999)

I get these results (bolding added post results)

• 0.00000099000000000000004 0.00000099     [990000.000000]
• 0.00000099900000000000018 0.000000999   [999000.000000]
• 0.00000099989999999999998 0.0000009998 [999900.000000]  <<-- your example
• 0.00000099999000000000012 0.0000009999 [999990.000000]

Perhaps at step [8] in the code I could "round" instead of just "truncating".

Could you post the URL you found this?  IIRC, someone else responded with a far more elegant solution.

-Louie

Updated on 2013-06-11T15:30:59Z at 2013-06-11T15:30:59Z by llandale
• jester76
68 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-11T15:43:33Z
• llandale
• ‏2013-06-11T15:27:12Z

"real" is by nature imprecise.

1.  On a computer, [a] "2*2" is not equal to [b] "2^^2" (exponent).  [a] uses integers which are precise, [b] converts to real which is not.

2.  You cannot accurately express "1/3" in a real number.  Looks like "0.33333333333333333" which is very close but not quite.

When I run this code:

• void Test(real r)
• { print fStringOfReal(r, 0,true) "\t" fStringOfReal(r, 10,true) "\t[" (r*1000000000000.0) "]\n"
• }
• Test(0.00000099)
• Test(0.000000999)
• Test(0.0000009999)
• Test(0.00000099999)

I get these results (bolding added post results)

• 0.00000099000000000000004 0.00000099     [990000.000000]
• 0.00000099900000000000018 0.000000999   [999000.000000]
• 0.00000099989999999999998 0.0000009998 [999900.000000]  <<-- your example
• 0.00000099999000000000012 0.0000009999 [999990.000000]

Perhaps at step [8] in the code I could "round" instead of just "truncating".

Could you post the URL you found this?  IIRC, someone else responded with a far more elegant solution.

-Louie

Hello,

Here is the link I found this @.

I think rounding would be better then truncating. But the funny thing is that it does not do it with .0000008888 or any other number like this.

Can you please post the solution with rounding.

Thanks

Jester

• llandale
3035 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-11T17:07:03Z
• jester76
• ‏2013-06-11T15:43:33Z

Hello,

Here is the link I found this @.

I think rounding would be better then truncating. But the funny thing is that it does not do it with .0000008888 or any other number like this.

Can you please post the solution with rounding.

Thanks

Jester

I'll let you debug it.  May look like:

• int iLast, iNext
•                                 // [8] Truncate Decimal Part to max DesiredPrecision
• DecimalLen = length(DecimalPart)
• if(DesiredPrecision > 0  and
•    DesiredPrecision < DecimalLen) // Desired is non-zero, but less than Actual -
• {  iLast = intOf(DecimalPart[DesiredPrecision-1] "")  // last digit desired
•    iNext = intOf(DecimalPart[DesiredPrecision] "")    // next digit, ignored
•    if (iNext >= 5) then iLast++    // Round
•    DecimalPart = DecimalPart[0:DesiredPrecision-2] iLast ""
• }

-Louie

• jester76
68 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-11T18:09:55Z
• llandale
• ‏2013-06-11T17:07:03Z

I'll let you debug it.  May look like:

• int iLast, iNext
•                                 // [8] Truncate Decimal Part to max DesiredPrecision
• DecimalLen = length(DecimalPart)
• if(DesiredPrecision > 0  and
•    DesiredPrecision < DecimalLen) // Desired is non-zero, but less than Actual -
• {  iLast = intOf(DecimalPart[DesiredPrecision-1] "")  // last digit desired
•    iNext = intOf(DecimalPart[DesiredPrecision] "")    // next digit, ignored
•    if (iNext >= 5) then iLast++    // Round
•    DecimalPart = DecimalPart[0:DesiredPrecision-2] iLast ""
• }

-Louie

Actually it is not the question of rounding. if I supply 0.0000009999 it should return same right. Why is last 9 being converted to 8? That is the real question.

Update:

I found the issue:

if I change the follwoign variable anything more than 25 the behavior changes

cl_fStringOfRealExponent = 25 // Arbitrary limit of Digits away from decimal point

Not sure why that is.

Updated on 2013-06-11T19:23:00Z at 2013-06-11T19:23:00Z by jester76
• llandale
3035 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-11T19:25:04Z
• jester76
• ‏2013-06-11T18:09:55Z

Actually it is not the question of rounding. if I supply 0.0000009999 it should return same right. Why is last 9 being converted to 8? That is the real question.

Update:

I found the issue:

if I change the follwoign variable anything more than 25 the behavior changes

cl_fStringOfRealExponent = 25 // Arbitrary limit of Digits away from decimal point

Not sure why that is.

It isn't.

• 0.0000009999   is being converted to 0.00000099989999999999998, and then truncated to 0.0000009998.
• 0.00000099999 is being converted to 0.00000099999000000000012, and then truncated to 0.0000009999

Now that I look at it, if we correctly "round" the 1st one to 0.0000009999 then we would "correctly?" "round" the 2nd one to 0.0000010000.

I think this im-precision is the nature of the beast when manipulating real numbers.  I don't know what to do about it.

-Louie

• llandale
3035 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-06-12T13:54:01Z
• llandale
• ‏2013-04-24T20:48:29Z

The "real" type has a lot of precision, but when you print it only 6 decimal digits are displayed.  Some time ago I posted a "fStringOfReal()" function which clumsily handled it; other folks posted more elegant solutions.

-Louie

I misspoke here.  This has nothing to do with "printing".  Only 6 fractional digits are preserverd when it is converted to a string printing converts it to a string.

82 Posts

#### Re: real in DXL

‏2013-06-13T07:44:06Z

I have attached my real formatting library code.

It provides conversion between real and string (Base 10, decimal or scientific notation) in both directions.

• llandale
3035 Posts

#### Re: real in DXL

‏2013-06-14T20:47:23Z
• ‏2013-06-13T07:44:06Z

I have attached my real formatting library code.

It provides conversion between real and string (Base 10, decimal or scientific notation) in both directions.

Wow, zero can be positive or negative and a real can be positive or negative infinity.

Ran a couple routine tests of the code like this and confirmed the ToString and FromString functions didn't lose any precision, which is a real good sign.

• void Test(real r1)
• {
•  string s1 = Real_ToString(r1)
•  real   r2 = Real_FromString(s1)
•  print (r1 == r2) "\t[" s1 "]\t"
•  print "[" r1 "]\n"  // Doors convert to string
• }
• Test(1.4455555555555555555555555557e63)
• Test( 2.0^1025.0)
• Test( 2.0^1024.0)
• Test( 2.0^1023.9999)
Test(-2.0^1023.9999)

OK, so that last one DOES lose something.  I wonder what print result [-1.#IND00] means.  Even if IEEE doesn't recognize it, it appears that DOORS does.

Didn't really study the code, but have these comments when reading it:

• Yes, while my comments are too verbose, this is far too lean.  What is a "glyph" for example?
• You have too many hard-coded arbitrary constants; like "20.0" and "1023", "1022"
• Too many hard-coded "precise" constants, which I think are not quite precise enough.  "0.3010299956639811" doesn't feel right to me.  Should that not be some calculated Exponenet or Logrithm or something?

In any event it's an order of magnatude better than what I wrote.

-Louie

82 Posts

#### Re: real in DXL

‏2013-06-17T08:22:05Z
• llandale
• ‏2013-06-14T20:47:23Z

Wow, zero can be positive or negative and a real can be positive or negative infinity.

Ran a couple routine tests of the code like this and confirmed the ToString and FromString functions didn't lose any precision, which is a real good sign.

• void Test(real r1)
• {
•  string s1 = Real_ToString(r1)
•  real   r2 = Real_FromString(s1)
•  print (r1 == r2) "\t[" s1 "]\t"
•  print "[" r1 "]\n"  // Doors convert to string
• }
• Test(1.4455555555555555555555555557e63)
• Test( 2.0^1025.0)
• Test( 2.0^1024.0)
• Test( 2.0^1023.9999)
Test(-2.0^1023.9999)

OK, so that last one DOES lose something.  I wonder what print result [-1.#IND00] means.  Even if IEEE doesn't recognize it, it appears that DOORS does.

Didn't really study the code, but have these comments when reading it:

• Yes, while my comments are too verbose, this is far too lean.  What is a "glyph" for example?
• You have too many hard-coded arbitrary constants; like "20.0" and "1023", "1022"
• Too many hard-coded "precise" constants, which I think are not quite precise enough.  "0.3010299956639811" doesn't feel right to me.  Should that not be some calculated Exponenet or Logrithm or something?

In any event it's an order of magnatude better than what I wrote.

-Louie

Ah, you managed to get an 'Indeterminate' (Not a Number) value with your last test.

I have added it to the library.

Note: It will still fail your  (r1 == r2) test because NAN != NAN (which seems stupid to me!)

Magic Numbers:

20.0 : to bit shift by 20 because no bit shift operator in dxl

1023, 1022 : IEEE specified for Normalise / Denormalize

0.3010299956639811: approximation of [ln(2) / ln(10)] which constrains the result to prevent it overshooting the precise answer

• jester76
68 Posts

#### Re: real in DXL (truncates) (Is there a way to make real type have 99.000000000)

‏2013-07-03T20:13:56Z
• llandale
• ‏2013-06-11T19:25:04Z

It isn't.

• 0.0000009999   is being converted to 0.00000099989999999999998, and then truncated to 0.0000009998.
• 0.00000099999 is being converted to 0.00000099999000000000012, and then truncated to 0.0000009999

Now that I look at it, if we correctly "round" the 1st one to 0.0000009999 then we would "correctly?" "round" the 2nd one to 0.0000010000.

I think this im-precision is the nature of the beast when manipulating real numbers.  I don't know what to do about it.

-Louie

Found something weird here. If I supply "0.9999999" to the it rounds it to 1.0000000999. Not sure why. Can you please please look into it.

Thanks for the help

Jester

• jester76
68 Posts

#### Re: real in DXL

‏2013-07-03T20:15:13Z
• ‏2013-06-17T08:22:05Z

Ah, you managed to get an 'Indeterminate' (Not a Number) value with your last test.

I have added it to the library.

Note: It will still fail your  (r1 == r2) test because NAN != NAN (which seems stupid to me!)

Magic Numbers:

20.0 : to bit shift by 20 because no bit shift operator in dxl

1023, 1022 : IEEE specified for Normalise / Denormalize

0.3010299956639811: approximation of [ln(2) / ln(10)] which constrains the result to prevent it overshooting the precise answer

I will try to test this one as well. What is the diff. between private and public funcations.

How do you restrict the number of digits after the decimal place.

thanks jester

Updated on 2013-07-04T02:56:57Z at 2013-07-04T02:56:57Z by jester76
82 Posts

#### Re: real in DXL

‏2013-07-04T08:43:01Z
• jester76
• ‏2013-07-03T20:15:13Z

I will try to test this one as well. What is the diff. between private and public funcations.

How do you restrict the number of digits after the decimal place.

thanks jester

I've attached my UnitTests that I used during development.

There is no real difference between private and public funcations. It is just my naming convention to make it clear which functions are safe to call and which are not. The private ones are just internal helpers for the safe functions.

It converts the numbers without precision loss (a round trip of [Real -> String -> Real] will result in exactly the same real), so there is no functionality to truncate or round. If you need to do that, do it as a separate step afterwards.

82 Posts

#### Re: real in DXL

‏2013-07-04T08:57:56Z
• llandale
• ‏2013-06-14T20:47:23Z

Wow, zero can be positive or negative and a real can be positive or negative infinity.

Ran a couple routine tests of the code like this and confirmed the ToString and FromString functions didn't lose any precision, which is a real good sign.

• void Test(real r1)
• {
•  string s1 = Real_ToString(r1)
•  real   r2 = Real_FromString(s1)
•  print (r1 == r2) "\t[" s1 "]\t"
•  print "[" r1 "]\n"  // Doors convert to string
• }
• Test(1.4455555555555555555555555557e63)
• Test( 2.0^1025.0)
• Test( 2.0^1024.0)
• Test( 2.0^1023.9999)
Test(-2.0^1023.9999)

OK, so that last one DOES lose something.  I wonder what print result [-1.#IND00] means.  Even if IEEE doesn't recognize it, it appears that DOORS does.

Didn't really study the code, but have these comments when reading it:

• Yes, while my comments are too verbose, this is far too lean.  What is a "glyph" for example?
• You have too many hard-coded arbitrary constants; like "20.0" and "1023", "1022"
• Too many hard-coded "precise" constants, which I think are not quite precise enough.  "0.3010299956639811" doesn't feel right to me.  Should that not be some calculated Exponenet or Logrithm or something?

In any event it's an order of magnatude better than what I wrote.

-Louie

Oh, and I don't calculate 0.3010299956639811, because we need to avoid using reals in the conversion or we just introduce more precision loss. It is what it is to ensure the resulting number is suitable for the remaining algorithm.

I used the term Glyph to mean a character in decimal notation {0123456789}. I didn't call it number, because at that point in the algorithm it is just a sequence of Glyphs which need to be altered and arranged to form the number:

• Correcting for the wrong Glyphs
• Truncating Glyphs which add no precision
• Putting the decimal point in the correct place
Updated on 2013-07-04T09:08:50Z at 2013-07-04T09:08:50Z by Adamarla
• jester76
68 Posts

#### Re: real in DXL

‏2013-10-11T12:40:27Z
• jester76
• ‏2013-07-03T20:15:13Z

I will try to test this one as well. What is the diff. between private and public funcations.

How do you restrict the number of digits after the decimal place.

thanks jester

Here are couple of cases I ran into:

1> 0.009 / 3 returns wrong results => 0.0029999999999999996

2> 0.008/3 returns wrong results => 0.0026666666666666664 should be 0.0026666666666666667

Did you know about it. If so can you post fix for it.

82 Posts

#### Re: real in DXL

‏2013-10-11T13:13:18Z
• jester76
• ‏2013-10-11T12:40:27Z

Here are couple of cases I ran into:

1> 0.009 / 3 returns wrong results => 0.0029999999999999996

2> 0.008/3 returns wrong results => 0.0026666666666666664 should be 0.0026666666666666667

Did you know about it. If so can you post fix for it.

Initially, I thought you had found a bug in my library, but no...

The problem is not in this code: it is correctly reporting the result of the calculation.

The problem is that calculations with real (floating point) numbers do not always result in exactly the correct answer.

[NB. Not really a 'problem', it is just the way it is]

Classic example: (0.1+0.2 == 0.3) is false. Why?

Because the result of the calculation is actually 0.30000000000000004. Why?

Because the numbers are stored as the closest number that can be represented in floating point (real). Not all numbers can be represented in floating point (eg. 0.1).

This happens in all languages which use floating point - javascript, python etc

real r01 = 0.3
real r02 = 0.1 + 0.2
real r03 = 0.30000000000000004

print(r01 "\t" Real_ToString(r01) "\n")
print(r02 "\t" Real_ToString(r02) "\n")
print(r03 "\t" Real_ToString(r03) "\n")

print(r01 == r02)
print(r02 == r03)
print(r01 == r03)
real r11 = 0.003
real r12 = 0.009 / 3.0
real r13 = 0.0029999999999999996

print(r11 "\t" Real_ToString(r11) "\n")
print(r12 "\t" Real_ToString(r12) "\n")
print(r13 "\t" Real_ToString(r13) "\n")

print(r11 == r12)
print(r12 == r13)
print(r11 == r13)

real r21 = 0.002667
real r22 = 0.008/3.0
real r23 = 0.0026666666666666664

print(r21 "\t" Real_ToString(r21) "\n")
print(r22 "\t" Real_ToString(r22) "\n")
print(r23 "\t" Real_ToString(r23) "\n")

print(r21 == r22)
print(r22 == r23)
print(r21 == r23)

So why does the DOORS string seem correct... It is rounding it to six decimal places.

Updated on 2013-10-11T13:34:25Z at 2013-10-11T13:34:25Z by Adamarla
• llandale
3035 Posts

#### Re: real in DXL

‏2013-10-11T15:34:36Z
• ‏2013-10-11T13:13:18Z

Initially, I thought you had found a bug in my library, but no...

The problem is not in this code: it is correctly reporting the result of the calculation.

The problem is that calculations with real (floating point) numbers do not always result in exactly the correct answer.

[NB. Not really a 'problem', it is just the way it is]

Classic example: (0.1+0.2 == 0.3) is false. Why?

Because the result of the calculation is actually 0.30000000000000004. Why?

Because the numbers are stored as the closest number that can be represented in floating point (real). Not all numbers can be represented in floating point (eg. 0.1).

This happens in all languages which use floating point - javascript, python etc

real r01 = 0.3
real r02 = 0.1 + 0.2
real r03 = 0.30000000000000004

print(r01 "\t" Real_ToString(r01) "\n")
print(r02 "\t" Real_ToString(r02) "\n")
print(r03 "\t" Real_ToString(r03) "\n")

print(r01 == r02)
print(r02 == r03)
print(r01 == r03)
real r11 = 0.003
real r12 = 0.009 / 3.0
real r13 = 0.0029999999999999996

print(r11 "\t" Real_ToString(r11) "\n")
print(r12 "\t" Real_ToString(r12) "\n")
print(r13 "\t" Real_ToString(r13) "\n")

print(r11 == r12)
print(r12 == r13)
print(r11 == r13)

real r21 = 0.002667
real r22 = 0.008/3.0
real r23 = 0.0026666666666666664

print(r21 "\t" Real_ToString(r21) "\n")
print(r22 "\t" Real_ToString(r22) "\n")
print(r23 "\t" Real_ToString(r23) "\n")

print(r21 == r22)
print(r22 == r23)
print(r21 == r23)

So why does the DOORS string seem correct... It is rounding it to six decimal places.

I am not "real" good with "real"s.  I wonder if you could do real comparisons

• (0.1+0.2 == 0.3)

By converting to a string, "rounding" to perhaps 15 characters, convertinb back to real and THEN comparing?  or perhaps multiplying by 2**15, rounding to whole numbers, and comparing them?  I suppose we'd have to write some "rounding" function.

-Louie

82 Posts

#### Re: real in DXL

‏2013-10-11T16:17:59Z
• llandale
• ‏2013-10-11T15:34:36Z

I am not "real" good with "real"s.  I wonder if you could do real comparisons

• (0.1+0.2 == 0.3)

By converting to a string, "rounding" to perhaps 15 characters, convertinb back to real and THEN comparing?  or perhaps multiplying by 2**15, rounding to whole numbers, and comparing them?  I suppose we'd have to write some "rounding" function.

-Louie

Just accept that you cannot represent an infinate number of values with a finite number of bits. Most values have to be stored as a slightly different number. It is just the way it works.

The normal way to compare floating point numbers is to check they are within some tolarance:

if( | r1 - r2 | < 0.0000001 ) { ... }

Most of the complexity in the library is due to the inherant precision loss of floating point calculations, trying to avoid it and to recover from it when unavoidable. That is why I don't calculate the logarithm.

• EHcnck
189 Posts

#### Re: real in DXL

‏2014-01-21T05:33:08Z
• llandale
• ‏2013-10-11T15:34:36Z

I am not "real" good with "real"s.  I wonder if you could do real comparisons

• (0.1+0.2 == 0.3)

By converting to a string, "rounding" to perhaps 15 characters, convertinb back to real and THEN comparing?  or perhaps multiplying by 2**15, rounding to whole numbers, and comparing them?  I suppose we'd have to write some "rounding" function.

-Louie

when comparing numbers(real/double/float) you should only use <, > never ==, <=, >=