http – Content Length in Golang-ThrowExceptions

Exception or error:

I couldn’t find anything helpful online on this one.

I am writing an REST API, and I want to log the size of the body of the request in bytes for metrics. Go net/http API does not provide that directly. http.Request does have Content-Length field, but that field can be empty or the client might send false data.

Is there a way to get that in the middlware level? The bruteforce method would be to read the full body and check the size. But if I do that in the middleware, the handler will not have access to the body because it would have been read and closed.

How to solve:

Why do you want a middle in here?
The simple way is b, err = io.Copy(anyWriterOrMultiwriter, r.Body)
b is total content length of request when err == nil
Use request body as you want. Also b, err = io.Copy(ioutil.Discard, r.Body)

Answer:

You could write a custom ReadCloser that proxies an existing one and counts bytes as it goes. Something like:

type LengthReader struct {
    Source io.ReadCloser
    Length int
}

func (r *LengthReader) Read(b []byte) (int, error) {
    n, err := r.Source.Read(b)
    r.Length += n
    return n, err
}

func (r *LengthReader) Close() error {
    var buf [32]byte
    var n int
    var err error
    for err == nil {
        n, err = r.Source.Read(buf[:])
        r.Length += n
    }
    closeerr := r.Source.Close()
    if err != nil && err != io.EOF {
        return err
    }
    return closeerr
}

This will count bytes as you read them from the stream, and when closed it will consume and count all remaining unread bytes first. After you’re finished with the stream, you can then access the length.

Answer:

Option 1

Use TeeReader and this is scalable. It splits reader into two and one of them calculates the size using allocated memory. Also, in the first case

  maxmem := 4096
  var buf bytes.Buffer
  // comment this line out if you want to disable gathering metrics
  resp.Body = io.TeeReader(resp.Body, &buf) 

  readsize := func(r io.Reader) int {
    bytes := make([]byte, maxmem)
    var size int
      for {
        read, err := r.Read(bytes)
        if err == io.EOF {
        break
      }
      size += read
    }
    return size
  }

  log.Printf("Size is %d", readsize(&buf))

Option 2 unscalable way (original answer)

You can just read the body, calculate the size, then unmarshal into struct, so that it becomes:

    b, _ := ioutil.ReadAll(r.Body)

    size := len(b) // can be nil so check err in your app

    if err := json.Unmarshal(b, &input); err != nil {
        s.BadReq(w, errors.New("error reading body"))
        return
    }

Leave a Reply

Your email address will not be published. Required fields are marked *