DAX 2009 Real Data Type

Hi guys,

In DAX 2009, for the real data type, just want to ask the following?

(1) what is the effective precision/scale for real data type? When I create a new field with real data type, it creates the field in SQL server 2008 as numeric (28,12). Why was it created this with this precision (28) and scale (12)? Is it default in DAX ? Can you override it?

(2) I created an EDT for real with the NoOfDecimals = 12.

I opened table browser and tried to input this data :

1.123456789012 - OK

1234.123456789012 - OK

12345.123456789012 - not ok. The last decimal digit was rounded off… resulting to 12345.123456789010. Why is this so? I did not expect the round off since it still meets the NoOfDecimals value set. It seems it is always rounding off to the 17th digit. Is this standard behavior, as well ?

Thanks.

Just an update, 10 minutes after I posted the above, i realized that this might be the answer… :).

The documented precision of the real data type is 16 significant digits. So this seems to be the reason for my question (2).

As for question (1), SQL server maximum precision is 38 but older versions is 28. So it seems the underlying field is created with that precision, though within DAX, we can only be precised up to 16 significant digits. Why the scale is 12, that I have no clue.

Hi IDig,

Thanks in advance…!

Please help me I am facing the same problem… which is I cant able to get a 7 digit precision value into the dynamics AX AOT Table. The dynamics ax only supports the 2 precision value. As you told that your EDT is supporting the High Precision value like 12. Please let me know if any other alternate way to get and store the 7 digit precision float value… [:(]

You will be appreciated…

Thanks In advance…