Please start any new threads on our new site at https://forums.sqlteam.com. We've got lots of great SQL Server experts to answer whatever question you can come up with.

 All Forums
 SQL Server 2005 Forums
 Other SQL Server Topics (2005)
 Decimal data being rounded

Author  Topic 

KurtW
Starting Member

6 Posts

Posted - 2011-02-10 : 18:02:33
Hello all:

I am having an issue storing latitude and longitude as decimals. I have a Sql Server (2005) datatype of decimal(12, 6). If I enter
-140.8899 in my C#.Net 2.0 WinForms control, I end up with a rounded value of -141.000000 in the column.

Debugging shows that the value taken from the control into my business object remains -140.8899. Further, the value that is stored inside the command’s parameter debugs as -140.8899 as well. So that is the value being sent to the stored procedure.

It must be the Sql Server engine itself that is causing the rounding.
I suspected that since I was not explicitly setting the precision and scale of the parameter this was causing the issue. If I poll the parameter’s precision and scale after creation I see that it has been set automatically based on the length of the decimal passed in. But even if I set precision and scale to match my Sql Server type decimal(12, 6) I still get the rounding.

Anyone have an idea why?

Kurt

khtan
In (Som, Ni, Yak)

17689 Posts

Posted - 2011-02-10 : 19:06:36
what is the data type used in the stored procedure ?


KH
[spoiler]Time is always against us[/spoiler]

Go to Top of Page

KurtW
Starting Member

6 Posts

Posted - 2011-02-11 : 10:21:46
Khtan:

OY! <Head smack!> That was the issue! I declared a variable of decimal with NO precision or scale. Thanks for the catch! Namaste.

Kurt
Go to Top of Page

khtan
In (Som, Ni, Yak)

17689 Posts

Posted - 2011-02-11 : 10:23:49
you are welcome


KH
[spoiler]Time is always against us[/spoiler]

Go to Top of Page
   

- Advertisement -