go – Unmarshal json to float. Why is float64 required?-ThrowExceptions

Exception or error:

I’ve noticed some strange behavior with the way go unmarshals json floats. Some numbers, but not all, refuse to unmarshal correctly. Fixing this is as easy as using a float64 instead of a float32 in the destination variable, but for the life of me I can’t find a good reason why this is the case.

Here is code that demonstrates the problem:

package main

import (
    "encoding/json"
    "fmt"
    . "github.com/shopspring/decimal"
)

func main() {
    bytes, _ := json.Marshal(369.1368) // not every number is broken, but this one is
    fmt.Println("bytes", string(bytes))

    var f32 float32
    json.Unmarshal(bytes, &f32)
    fmt.Printf("f32 %f\n", f32) // adds an extra 0.00001 to the number

    var d Decimal
    json.Unmarshal(bytes, &d)
    fmt.Printf("d %s\n", d) // 3rd party packages work

    // naw, you can just float64
    var f64 float64
    json.Unmarshal(bytes, &f64)
    fmt.Printf("f64 %f\n", f64) // float64 works
}

A float64 isn’t required to accurately represent my example number, so why is required here?

Go playground link: https://play.golang.org/p/tHkonQtZoCt

How to solve:

Your assertion is wrong: 369.1368 cannot be represented exactly by either float32 or float64.

The closest float32 value is (approximately) 369.136810302734375, whcih rounds to 369.13681 which is where your extra digit comes from. The closest float64 value is (approximately) 369.13679999999999382, which rounds more nicely for your purposes.

(Of course, if you round either of these to just four digits after the decimal point, you get the number you expected.)

A Decimal representation is exact: there is no rounding error.

JSON transmits and receives floating point values expressed in decimal, but actual implementations, in various languages, then encode those numbers in different ways. Depending on what sort of entity you’re talking to via JSON, encoding and decoding via Decimal could preserve the number exactly as you’d like, but be aware that programs written in, say, C++ or Python might decode your number to a different floating-point precision and introduce various rounding errors.

This Go Playground example uses the newly-added %x format, and shows you how the numbers are stored internally:

as float32 = 369.13681030273437500 (float32), which is really 12095875p-15 or 0x1.712306p+08

and:

as float64 = 369.13679999999999382 (float64), which is really 6493923261440380p-44 or 0x1.712305532617cp+08

That is, the number 369.whatever is internally represented in binary. It’s between 28 = 256 and 29 = 512. In binary, it is 1 256, no 128, 1 64, 1 32, 1 16, no 8, no 4, no 2, and 1 1: 1.01110001something x 28. The %b format expresses this one way and the %x format another, with %x starting with 1.72 (1 . 0111 0010).

See Is floating point math broken? (as jub0bs linked in a comment) for more.

Leave a Reply

Your email address will not be published. Required fields are marked *