[Top][All Lists]
[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
[Pnet-developers] Fixing bugs in floating-point formatting
From: |
Marcus |
Subject: |
[Pnet-developers] Fixing bugs in floating-point formatting |
Date: |
Sun, 18 Dec 2005 19:00:24 -0600 |
User-agent: |
KMail/1.9.1 |
I am trying to fix some bugs the output formatting of floating point types,
specifically System.Single and System.Double. (System.Decimal seems to be a
bit different.) I have found two problems.
One problem is that the output string always has one too many digits. I fixed
that by changing the lines in FormatFloat() to
if (sb.Length > precision + exponent)
{
sb.Remove(precision+exponent,
sb.Length-(precision+exponent));
}
Another problem is that the output digits are always truncated (by removing
characters from the StringBuilder). In MS, the digits are rounded instead
(and not using "banker's rounding) of truncated.
I am also confused about the role of the local variable "exponent" because it
always seems to have the valve 0 for the cases I tried.
[Prev in Thread] |
Current Thread |
[Next in Thread] |
- [Pnet-developers] Fixing bugs in floating-point formatting,
Marcus <=