Questions tagged [floating-point]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
5 answers
328 views

How to handle precision problems with the correct terminology

I'm writing a math library and I've encountered trouble with floating point imprecision. I want to know if there are any standard workarounds that are used often. I have doubts about my current method ...
Scene's user avatar
  • 109
-4 votes
2 answers
101 views

Successor in a floating point system [closed]

I’m starting at numerical methods and I am introducing to the machine epsilon and succesor. I have this question, which is the succesor of .13579x10^2 at a floating point system of (10,5,-2,6). My ...
RES's user avatar
  • 3
5 votes
2 answers
300 views

How did the IEEE 754 standard choose how many bits to allocate to each field (i.e. sign, exponent, mantissa)?

I've been reading about the IEEE 754 standard, but can't understand how or why they chose those exact number of bits for the exponent and mantissa. For example, why 8 bits for the exponent and not 7 ...
hackm's user avatar
  • 53
-4 votes
1 answer
158 views

if I write 16 bit floating point number representation of 2044 and 2045.5 it's seems like the same, which is 0 1001001 11111111 why is that? [closed]

floating point representations## exponent == 7 bits ,mantissa == 8 bits 2044    = 0 1001001 11111111 2045.5 = 0 1001001 11111111 why is this happening?? what is the error? Is anything wrong with the ...
Senu Nawamina's user avatar
-1 votes
3 answers
560 views

How do you represent a negative floating point in two's complement?

I was looking into a datasheet of a Real Time Clock (RTC) with a temperature sensor MAX31328. (You can find the datasheet easily). I'd like to convert the raw binary value into decimal format. I have ...
Nuz's user avatar
  • 51
7 votes
2 answers
1k views

Why are angles usually represented by floats in programming?

My idea is that, if an unsigned 32-bit integer is used for representing angles in a way that 0 is 0°, 231 is 180° and 232−1 is just under 360°, all possible values are used and evenly distributed ...
matj1's user avatar
  • 87
2 votes
4 answers
253 views

Swift: Best way to store crypto amounts?

Having a bit of trouble on deciding the best way to store crypto values which are usually in decimals. Do I go with Double or Float? Or is there an even better type that Swift offers? My initial gut ...
Tank12's user avatar
  • 139
7 votes
6 answers
1k views

When are equality comparisons between IEEE 754 floating point values acceptable?

My understanding is that it's rarely a good idea to compare two floating point numbers due to inherent inaccuracies. However, my understanding of those inaccuracies are that they are deterministic and ...
user avatar
-4 votes
1 answer
210 views

What is the Difference Between Float and Double Data Types? [duplicate]

Apart from the memory size of both data types, I want to know any other differences and also want to know when we use float and when we use double?
lokeshjoshi's user avatar
2 votes
2 answers
706 views

How to use chebyshev polynomials to calculate exponents (antilogs)?

I am trying to write an algorithm to accurately calculate exponents (antilogs) for a variable precision floating point library I am working on. The base is not relevant since I can convert between ...
user264480's user avatar
-1 votes
5 answers
2k views

Why is What Every Computer Scientist Should Know About Floating-point Arithmetic considered mandatory reading? [closed]

What Every Computer Scientist Should Know About Floating-point Arithmetic is widely considered absolutely mandatory reading for every programmer. Why is this the case? What aspects of the article make ...
J. Mini's user avatar
  • 1,007
6 votes
4 answers
4k views

Which is preferred: subclass double or create extension methods to test (relative) equality due to floating point differences?

I am writing numerical calculation software using .NET C#, which needs to be blazingly fast. There is a lot of fractional math. So using decimal type is pretty much out of the question, given its poor ...
Conrad's user avatar
  • 288
5 votes
2 answers
3k views

How different are float numbers across different hardwares?

Firstly i am aware that different hardware architectures handle floats differently, so I can't guarantee the same result on different machines even with the same inputs. But in my case i do not have a ...
WDUK's user avatar
  • 2,072
-2 votes
2 answers
1k views

Sending float over TCP

I need to send a series of floats over a TCP socket. Firstly, is it safe to assume that most platforms (e.g off the shelf Windows PC/Laptop) and architectures represent floats in the same way (i.e ...
19172281's user avatar
  • 133
2 votes
5 answers
709 views

Will two equal floating point compare equal after the exact same set or operations?

Say if i have two floating point numbers double a = 0.1 double b = 0.1 Then a, and b, after going through the EXACT SAME set of complicated arithmetic operations, on the same computer, then compare. ...
Yituo's user avatar
  • 123
4 votes
5 answers
776 views

How do you assess the damage in a system that has been using floats or doubles for money?

I've been assigned to a code base responsible for millions of dollars of transactions, per quarter, and has been in use for over a decade. Sifting through the solution, I see doubles used everywhere ...
8protons's user avatar
  • 1,369
4 votes
3 answers
406 views

How to do grid systems with floats given precision causes problems

I've been trying to create a grid system that can use any grid size and start at any given anchor point rather than always at 0,0. The problem however is imprecision makes it impossible, even the ...
WDUK's user avatar
  • 2,072
-2 votes
3 answers
708 views

How are floating point representations (say single precision for instance) able to represent more numbers than what a 32 bit memory allows?

I am trying to understand the floating point system. Can the floating-point system represent all the real numbers within the range of (10^38) accurately? If so, then how is it possible? Because ...
Viper's user avatar
  • 29
3 votes
1 answer
196 views

Should equality operators be approximate when defining numeric types that rely on floating-point numbers?

For example, say you have a class Point which has floating point components. It's tempting to overload the equality operator so you can do something like a = Point(1.1, 2.2) b = Point(3.3, 6.6) b /= ...
Josie Thompson's user avatar
1 vote
4 answers
825 views

Why do many programming languages and applications use integer instead of floating point to represent time?

In my experience, the value of time mostly is represented by integer number, by programming languages (C/Java/Golang/Javascript(*)...), and by databases (MySQL, Postgres...), exceptions maybe some GUI/...
Yoshi's user avatar
  • 757
27 votes
5 answers
7k views

When conversion from Integer to Single may lose precision

I was reading an article from Microsoft regarding Widening Conversions and Option Strict On when I got to the part The following conversions may lose precision: Integer to Single Long ...
Vinicius V's user avatar
2 votes
1 answer
1k views

Should "NaN" default to SNaN or QNaN

I created a function to read floats (IEEE 754 binary 32-bit) from strings, for developers. I have everything finished and working perfectly, except i'm wondering if a user/or programmer enters a ...
b.sullender's user avatar
16 votes
6 answers
8k views

How to implement float hashing with approximate equality

Let's say we have the following Python class (the problem exists in Java just the same with equals and hashCode) class Temperature: def __init__(self, degrees): self.degrees = degrees ...
Marten's user avatar
  • 269
13 votes
6 answers
2k views

Do i still need to use fixed point to guarantee computers get the same result for math operations anymore?

I was told most modern computers follow the same floating point standard, does this mean they will all get the same float answer for a given math operation if the inputs are the same? I ask because i ...
WDUK's user avatar
  • 2,072
1 vote
1 answer
357 views

How can I used fixed-point arithmetic to convert from Celsius to Fahrenheit?

Disclaimer: I realize this is not the easiest or even best way to handle this and I am asking largely out of a sense of curiosity, not because I'm convinced this is the proper way. It's a fun puzzle I'...
brenzo's user avatar
  • 129
1 vote
1 answer
202 views

what *should* you get when you add minus-zero to plus-zero? [closed]

this Wikipedia article did not address this contingency and i am having trouble finding a definitive IEEE-754 document that addresses this.
robert bristow-johnson's user avatar
31 votes
9 answers
18k views

Does comparing equality of float numbers mislead junior developers even if no rounding error occurs in my case?

For example, I want to show a list of buttons from 0,0.5,... 5, which jumps for each 0.5. I use a for loop to do that, and have different color at button STANDARD_LINE: var MAX=5.0; var DIFF=0.5 var ...
ocomfd's user avatar
  • 5,712
1 vote
4 answers
1k views

Unit test a generic floating point equality function

I've written a function that tests two floating point numbers for approximate equality (see this Code Review question). I'd like to unit test my function, but I'm not positive of the best way to do ...
Dan Oberlam's user avatar
  • 1,281
4 votes
0 answers
191 views

Who first said, "If you think floating point is the answer, you don't understand the problem." [closed]

I had a C (programming language) teacher at the Lowell Institute at MIT who said this in the Spring of 1996. It's an overstatement, but still seems like a classic quote to me. I've had no luck ...
GlenPeterson's user avatar
  • 14.9k
6 votes
0 answers
155 views

Rounding floats after modf()

I am trying to make an effective modf() function that uses floats instead of doubles. An embedded application I am working on only supports the float type. My modf() function works as expected with ...
user264480's user avatar
1 vote
3 answers
2k views

Why can less precise data like float be faster than larger, more precise data like double?

I am currently reading a chapter in a textbook on Processor Architecture and saw the following statement: The less precision there is, the less space is occupied by a program variable in memory. ...
forgetaboutme's user avatar
1 vote
3 answers
3k views

Choosing an epsilon when comparing floating point numbers across different systems

I am transcribing thousands of lines of computational code from MATLAB to C++. I don't fully understand the math myself, but I can run it with MATLAB, assume it's correct, and compare the results to ...
user avatar
86 votes
6 answers
20k views

Why are floats still part of the Java language when doubles are mostly recommended instead?

In every place I've looked, it says that double is superior to float in almost every way. float has been made obsolete by double in Java, so why is it still used? I program a lot with Libgdx, and ...
Eames's user avatar
  • 885
0 votes
2 answers
775 views

Floating point accuracy and conversions

I am trying to build a class where user can compute floating operations with a particular accuracy. For example, if the user wants to have up to 4 digit accuracy, everything less than 0.0001 is zero ...
MaLiN2223's user avatar
  • 103
5 votes
2 answers
2k views

Floating point calculation being too exact

In Java, I've run this method: private static void floatAddition() { float a = 0.1F, b = 0.2F, c = a+b; System.out.println(a + " + " + b + " = " + c); } and the result is 0.3 for ...
Baustein's user avatar
-1 votes
1 answer
486 views

Floating point absorption phenomena and ULP

Numerical analysis text books talks about the absorption phenomena (Introduction to Numerical Analysis and Scientific Computing; Nabil Nassif; Dolly Khoueiri Fayyad; CRC Press; 2014) when adding ...
ArgBat's user avatar
  • 1
8 votes
4 answers
20k views

Avoiding Division by Zero Using Float Comparison

Source code analyzers (like SonarQube) complain about float (or double) equality comparisons because equality is a tricky thing with floats; the values compared can be the results of computations ...
Alfe's user avatar
  • 241
9 votes
2 answers
581 views

Should `Vector<float>.Equals` be reflexive or should it follow IEEE 754 semantics?

When comparing floating point values for equality, there are two different approaches: NaN not being equal to itself, which matches the IEEE 754 specification. NaN being equal to itself, which ...
CodesInChaos's user avatar
  • 5,717
-3 votes
1 answer
1k views

How to convert byte-array (4 bytes) back into float? [closed]

A piece of code I'm trying to debug generated a sequence of 4 bytes from a known float number. I'm trying to verify the float number given the four bytes. The (known) float 8.03 generated the bytes ...
Minh Tran's user avatar
  • 115
4 votes
0 answers
865 views

Rationale of IEEE754 direct rounding behavior near infinity [closed]

A preface: it's well known that IEEE754 defines five rounding modes (in 2008 edition terms, with my abbreviations): rounding to nearest, ties to even (RNE) - the default mode for binary arithmetic; ...
Netch's user avatar
  • 1,520
9 votes
3 answers
3k views

Is order of arguments in an arithmetic expression important to achieve as most exact result as possible (speed is not necessary)?

Actually, in this question I do not ask about particular language or architecture, however I understand there might be some differences. In physics / engineering it is usually better to handle ...
Voitcus's user avatar
  • 201
11 votes
2 answers
44k views

How to calculate min/max values of floating point numbers?

I'm trying to calculate the min/max, or the lowest to highest value range of a 48 bit floating point type MIL-STD-1750A (PDF) (WIKI). Ex: How a double range is 1.7E +/- 308 I've looked around for ...
CodeMonkey's user avatar
7 votes
2 answers
5k views

Why does Python's math.ceil return a float?

While I understand that the difference between integral and long values is blurred in Python, the difference between floats and integral values is not. Therefore, I'm having a difficult time ...
Naftuli Kay's user avatar
  • 1,611
0 votes
1 answer
752 views

floating-point number stored in float variable has other value [duplicate]

I'm reading a book about C programming By "King.K.N" and I'm reading the following statement: If we store 0.1 in a float variable, we may later find that the variable has a value such as 0....
Higgs's user avatar
  • 169
10 votes
1 answer
545 views

How can unums emulate IEEE's negative zero?

I am currently reading "The End of Error - Unum Computing" by John Gustafson (Youtube). What I am still not sure about is how the cases handled in IEEE by negatively signed zero are handled ...
false's user avatar
  • 233
2 votes
3 answers
3k views

How can a computer round the last digit in a floating point representation?

I'm confused by how a computer rounds off the last digit in the floating point representation. For example, I'm told that x=1.24327789 is stored in a computer with a 6-digit capacity then it;s ...
DLV's user avatar
  • 301
1 vote
2 answers
172 views

How to implement rounding in an all-purpose stack language using different types?

Disclaimer: If you are not terribly interested in numerics and mathematical processes, this is most likely nothing for you. I am currently a bit stuck in a development process of a private project I ...
Thorsten S.'s user avatar
86 votes
6 answers
32k views

Why is negative zero important?

I'm confused about why we care about different representations for positive and negative zero. I vaguely recall reading claims that having a negative zero representation is extremely important in ...
jpmc26's user avatar
  • 5,409
4 votes
1 answer
407 views

Is the behaviour of floating-point arithmetic defined by the c++ standard?

I started writing this as a SO question, but I think it's better suited here. While I'm happy for a simple answer I'm also fond of the "Teach a Man to Fish" philosophy so I'm happy for people to point ...
quant's user avatar
  • 1,288
1 vote
1 answer
3k views

adding the digits of a number

I am trying to write a program that asks the user for a decimal number and then calculates the sum of its digits. for example if the number is 123.25 then the sum will be 1+2+3+2+5=13. I decided to ...
user170800's user avatar