[identity profile] belovedcrown.livejournal.com posting in [community profile] techrecovery
found this solution online to trim the excess decimals caused by the floating point error, but it doesn't round, which means the total pricing for each item i am trying to calculate may be off by a penny:

function dp(price)

{

string = "" + price;

number = string.length - string.indexOf('.');

if (string.indexOf('.') == -1)

return string + '.00';

if (number == 1)

return string + '00';

if (number == 2)

return string + '0';

if (number > 3)

return string.substring(0,string.length-number+3);

return string;

}

can someone help me make this round?

Date: 2003-11-22 10:30 pm (UTC)
From: [identity profile] insaint.livejournal.com
What language is this written in?

For Java, check out DecimalFormat class:
http://java.sun.com/j2se/1.3/docs/api/java/text/DecimalFormat.html

For C++, use this code:
cout.setf(ios_base::fixed);
cout.precision(2);

Language-independent:
int temp = (price + 0.005) * 100;
double result = (double)temp / 100;
return double;

Hope this helps. And with that said, what in the hell does your post have to do with this community?

Date: 2003-11-24 09:42 pm (UTC)
ximinez: (Default)
From: [personal profile] ximinez
I'm assuming this is related to your last post and you are using Javascript. The safest way to get rid of rounding errors is to get out of floating point and into integer. eg.

function FloatToInt(number, factor)
// factor is 1/precision, so if you want to round to two decimal
// places, precision is 0.01, and 1/0.01 is 100.
{
return Math.round(number * factor);
}

Now that you've got an int, you can divide it again by the factor

number / factor

to get the most accurate representation that Javascript can give you.

Alternatively, since the factor is a power of 10, you could just format it as a string

number.substring(0, number.length - 1 - 2) + "." + number.substring(number.length - 2, number.length - 1)

Another possibility, since you're working with money, is to do all of your computations in cents instead of dollars.

This is off the top of my head, so please double check my syntax, and function parameters.

Profile

techrecovery: (Default)
Elitist Computer Nerd Posse

April 2017

S M T W T F S
      1
2345678
91011121314 15
16171819202122
23242526272829
30      

Most Popular Tags

Style Credit

Expand Cut Tags

No cut tags
Page generated Mar. 20th, 2026 05:38 am
Powered by Dreamwidth Studios