HD62958: DRAFTING (INTDRAFT) / PRECISION / THE DIMENSIONS VALUE IS NOT R OUNDED OFF CORRECTLY.

APAR status

• Closed as user error.

Error description

• Drafting (IntDraft) / Precision / The dimensions
  value is not rounded off correctly.
  .
  SCENARIO
  .
  ) Start CATIA V5R17SP03.
  2) Open the file : "Drawing1.CATDrawing".
  3) Click "Dimension" icon.
  4) Select Two Pink line which Figure shows.
  Dimension is created. - Dimension is 41.7mm.
  .
  Check the dimension by changing Precision.
  .
  5) Click the dimension by MB3,
  Select "Properties" in the context menu.
  "Properties" dialog box is displayed.
  6) Change the value of "Format/ Precision"
  to "0.01" .
  The Precision of Dimension is changed.
  Dimension is 41.75mm.
  .
  PROBLEM
  .
  The dimensions value is not rounded off correctly.
  .
  EXPECTED RESULT
  .
  The dimensions value should be rounded off
  correctly.
  In this case, the Dimensions should be
  created "41.8mm".
  .
  .
  

Local fix

Problem summary

Problem conclusion

Temporary fix

Comments

• Incident Diagnosis
  In the given drawing, when you create a dimension
  between the purple dotted lines, you get a
  dimension value of 41.7, while the result is 41.75
  with precision set to 0.01 or lower
  Correct Application
  This is normal behaviour. The real value is:
  41.749999999959172
  - rounded (not cut) to 0.000001, it leads to
  41.750000 because the 7th digit is 9. Note that
  following the chosen numerical display format, the
  trailing zeros may not be displayed in dimension
  value
  - rounded to 0.01, it leads to 41.75, because the
  3rd digit is a 9.
  - rounded to 0.1, it leads to 41.7, because the
  2nd digit is a 4.
  That's it.
  http://www.math.com/school/subject1/lessons/S1U
  1L3GL.html
  With your example: 41.745999, the round off
  With 0.000001 is - 41.745999 =>OK
  With 0.00001 is - 41.74600 =>OK
  With 0.0001 is - 41.7460 =>OK
  With 0.001 is - 41. 746 =>OK
  With 0.01 is - 41.75 =>OK
  With 0.1 is - 41.8 => NOT OK, the 2nd digit in
  your original value is 4 so it as to be rounded to
  41.7, not 41.8.
  It is the real value that is rounded and must be
  rounded. We shouldn't round off already rounded
  values as it would introduce even more
  incoherences.
  You can try the same thing with excel: enter the
  given value and then play around with Cell
  format/Number/Number/Decimal place. You'll get
  the same result.
  Documentation Reference
  Interactive Drafting
  Getting Started
  .
  

APAR Information

• APAR is sysrouted FROM one or more of the following:

• APAR is sysrouted TO one or more of the following:

Fix information

Applicable component levels