Ask HN: Is float divergence/drift a real issue in many industries?
8 pointsby Swapnopama day ago3 comments
  • sigbottle12 hours ago
    For physics simulations, or even in RF work, the tolerances you need are quite a bit more than say, AI work. For example, for one of our RF algorithms, we "have" to use doubles (I actually can think of an optimization that can let us use floats, but it's quite involved to fix it) to get the required precision. That precision requirement is 10^-15. For physics simulations, I've heard of accuracies of up to 10^-20.

    I will say, generally, it depends on what you mean. There's like, numerical drift, and then there's converting to the discrete part. Generally, I'd say that "getting into the discrete part" of an algorithm, unless you're trying to match behavior of another piece of software, is fine. Like, if you're playing a game, people are not going to care if the shot just grazed them and missed, but the "Actual True Value" is like 10^-9 off or something. They will just chalk it up to "close, unlucky". If you have one canonical place for your floating point errors from which your "discrete" parts derive, it's no biggie.

    But at work, I recently debugged an algorithm in which the author, effectively wanted to compute the integer and fractional parts of a double type. Their reasoning was that, for the fractional part, there can be imprecision, so you should do some fancy math to recover the fractional part.

    This is giga wrong for the application we were building. You never want to try computing the "same double" in two different ways and compose the results like this. Intuitively, if one path "thinks" your double is 15.00001, and the other path "thinks" its 14.99999, you're now saying that the double is 15 (integer) + .99999 (fractional) = 15.99999: this is way off now!

    You should just use one double, and extract out the integer and fractional parts. It doesn't matter if the double is imprecise - it's authoritatively imprecise. Much easier to reason about.

    This is the first time I've debugged an edge case like this since I wouldn't implement something like this, but it's an example of how subtle floating point errors can be.

    • Swapnopam36 minutes ago
      Agreed with you on that. One of the reasons for my post was, I was told by some engineers over at aeronautical companies that such float drift is trivial — anything beyond 10^-5 or 10^-8 becomes meaningless. Nearly the same was said by some engineers at HFT and quant firms. Which led me to wonder if cross-hardware drift is really a big deal. From what I've tested, even differences between an Intel i5 and Intel i7 pile up, let alone between Intel and AMD. Curious if the "it's within tolerance" answer holds in your RF work or if you've hit cases where the hardware boundary actually mattered.
  • mitchellpkt17 hours ago
    In my own experience with data science work:

    On multiple occasions I’ve seen noticeable accumulated rounding error crop up by the end of a cumprod over very long timeseries of floats.

    But in my cases it has just been a minor nuisance that I have to double check whether a mismatch came from float-related quirkiness or a code mistake.

    I got into the habit of just using integers where applicable for cases that need to match down to the last bit, and numpy.isclose() for inherently floaty cases where it’s anticipated and reasonable.

    Only speaking from my own experience, which doesn’t involve chaotic systems or anything like that. I’m curious to hear from people in other domains who may have more exciting stories.

  • 17 hours ago
    undefined