How to model JSON array as protobuf definition-ThrowExceptions

Exception or error:

I am writing a new service in golang using protobuf. I want to model the following Request JSON in the .proto file.

[
   {
    "var": ["myVariable1","myVariable2"], 
    "key1": 123123,
    "key2": 1122,
    "key3": "abcd-0101"
   },
  { 
    "var": ["myVariable1"], 
    "key1": 123124,
    "key2": 1123,
    "key3": "abcd-0102"
  },
] 

There are two problems currently :

  • The keys in each array element are not known beforehand, hence I cannot create a message in .proto file and make it repeated. I need to keep it map
  • I am unable to model a json which is just an array with no key. Everytime i do this, following error is shown : failed to decode request: json: cannot unmarshal array into Go value

Following is my .proto file :

syntax = "proto3";

package pb;

import "google/protobuf/empty.proto";
import "google/api/annotations.proto";

service Transmitter {
  rpc GetVariables(GetVariablesRequest) returns (GetVariablesResponse) {
    option (google.api.http) = {
                post: "/api/v1/{Service}/getVars"
                body: "*"
            };
  };
}


message GetVariablesRequest {
  string Service = 1;
  repeated GetVarInput in = 2;
}

message GetVariablesResponse {
  string msg = 1;
}

message GetVarInput {
  map<string,string> Input = 2;
}

I have tried with bytes instead of repeated GetVarInput, but it always comes empty. Also tried body: “*” and body : “in”

Please provide some pointers.

How to solve:

You can write a message for your json like this:

message RequestMessage {
   string var = 0;
   double key1 = 1;
   double key2 = 2;
   string key3 = 3;
}

Further, you can create another message which contains an array of RequestMessage

message Request {
   repeated RequestMessage request = 0;
}

Answer:

Another approach is usage of Any message in your .proto file. In this case you could have a repeated map with any GO types you need. But this approach is a bit overhead as it gives you a flexibility using strongly typed programming language. Here’s an example:

message Any {
    string type = 1;
    bytes value = 2;
}

and in your GO code:

func GetRPCAnyValue(value interface{}) *pb.Any {
    bValue, type_val, _ := MarshlizeValue(value)
    return &pb.Any{
        Type: type_val,
        Value: bValue,
    }
}

Answer:

The answer by Satyam Zode seems like the most reasonable one.

In your original question, you are trying to keep all the flexibility and dynamic nature of JSON, and in that case trying to convert it into protobuf makes little sense. You could just have a string field that contains the json.

However, protobuf does handle future changes to RequestMessage contents quite well, so you do not have to know the final set of fields yet. Just follow Updating a Message Type to keep them compatible between different .proto file versions.

Leave a Reply

Your email address will not be published. Required fields are marked *