Protocol Buffers: Benchmark and mobile
Going faster on a mobile has become essential. Putting aside the actual choice on the means of communication, the data format used weighs in quite a bit when it comes to the speed. As of Today, JSON has proven to be a standard media for APIs. Still, is this data format suitable for mobile? For instance, handling JSON in an Android environment is difficult.
Other data formats are emerging in recent years like Thrift, Avro, Message Pack or Protocol Buffers.
Protocol Buffers has the ability to have a binary format that is easily adaptable and which can be manipulated. It also has a very basic structure to write and understand and easily generate source code for several languages.
This blog featured two articles on Protocol Buffers (protobuf) in 2012. The version used was 2.4.1, and the standard was proto2.
Evolution
Protobuf development began in 2001 and the latest stable version was released in October 2014. This time around, the beta-3 3.0.0 will be discussed (Edit (01/08/2016): V3 was finally released!). With this beta, standard proto3 appeared. The purpose of this standard is to simplify a little more protobuf structure declaration files.
| message Hello {<br> message Bye {<br> required string name = 1;<br> optional int32 count = 2 [default = 1];<br> }<br><br> repeated Bye bye = 1;<br> optional bool check = 2;<br>} | message Hello { <br> repeated Bye bye = 1;<br> bool check = 2;<br>}<br><br>message Bye {<br> string name = 1;<br> int32 count = 2;<br>} |
| proto2 | proto3 |
Among visible changes between proto2 and proto3, we note the disappearance of attributes ("optional" and "required"). The attribute "required" was already not recommended by Google: it prevents subsequent suppression of a field in the object future versions. Indeed, delete, edit or add a field "required" brings reading bugs of the protobuf data. In addition, now, default values can not be changed. The first argument in favor of these changes remains the simplification of the objects. The second is the protobuf extension to other languages, which they do not necessarily accept the defaults values. This choice allows to standardize the generation in present and future languages. Nevertheless, the proto2 standard is always accepted by the Protocol Buffers generator.
In its version 2.6.1, Protocol Buffers can be declined in four languages: C ++, Java, Python and Go. Currently, the 3.0.0-beta 3 version includes new variations. C # and Objective-C were added in beta. JavaScript, Ruby and Javanano are currently in alpha. Protocol Buffers is BSD licensed. Everyone then has the opportunity to participate in its evolution. This allows quick expansion. We can now find a lot of protobuf generators for many different languages. This list contains some of the existing implementations.
Benchmarks
To visualize Protocol Buffers performance, a client (Nexus 5, Android 6.0.1) calls a server (Go) to transmit text content. The call is made locally via HTTP/2.
Tests Environment
To compare results, other data formats are used: XML, JSON and MessagePack. For JSON, three libraries are tested: Jackson, Jackson Jr and Moshi. In Protobuf side, there are two libraries used: the official implementation of Google (bêta-3.0.0) and Square Wire (2.2.0)
Processing Time
Processing time is the time between end of packets reception and the moment to send the final objects for display. The client requests the server for 10 files of varying numbers of subsections (50 to 500). These subsection are randomly selected so, the data received are different everytime. Each request is repeated 50 times for each number of paragraphs to get an average. The "parser" initialization time (whether for JSON, MessagePack or Protobuf) which occurs in the first round, is diluted. This choice is made to approach reality: the "parser" object is set on the first turn and is reused throughout the application usage.
Benchmark Results: ms time by the number of subsections by file
First finding already known XML is totally disqualified. For readability, we remove XML in the schema. Results:
without XML - Benchmark Time in ms by the number of subsections by file
For other formats, processing time is already superior than Protocol Buffers. For gzipped JSON, the processing cost is multiplied by 1.2 compared to JSON. The factor between JSON (with Jackson, the lowest) and Protocol Buffers is almost 2.
The conclusion is, for processing time, on text data, Protocol Buffers is clearly to his advantage.
Memory
Weight
At the compression of the initial data, we can see that the binary protobuf is lighter than MessagePack or JSON gzipped.
Android
Despite these encouraging results, the establishment of a new solution may seem perilous. For Protocol Buffers, it is relatively simple. Take Android as an instance. We have the choice between the Implementation of Official Google (IOG) and Square Wire version. Beyond efficiency, compare several other points is necessary.
Code Generation:
Two ways to generate your code: command line or pre-build. Command line:
For IOG, the latest version package is available and contains an executable:
protoc --java_out *.proto
For Wire, a jar is also available.
java -jar wire-compiler-2.2.0-jar-with-dependencies.jar --java_out =. --path_proto =. *.proto
Pre-build:
On the IOG side, it's not possible for the proto3 standard. A Gradle plugin exists but requires gradle 2.12 and includes only proto2. Wire is more effective in this regard. Square officially has a Gradle version of Wire plugin, but the repository is not updated since a year. Moreover, there is no tutorial on how to use it and it cannot be found in the official repositories of jcenter or mavenCentral. One of the forks solves these problems: the plugin Wire Gradle by Jiechic. Then, establishment is then very simple:
In project build.gradle
classpath 'com.jiechic.librairy.wire: wire-gradle-plugin: 1.0.0'
In app build.gradle:
apply plugin: 'com.squareup.wire' dependencies { compile 'com.squareup.wire: wire-runtime:2.2.0' }
Then simply place the proto files in src/main/proto. At compile time, objects will be generated in build/generated/source/proto.
Generated Objects:
Square wins. The generated files by IOG are unreadable and repulsive. The Square ones are much simpler to understand. In addition, the generated objects are also simpler to use.
Encode / Decode:
In my case, only the part 'decode' of each version was used. The syntax is simple. For example, the structure is:
syntax = “proto3”; package hello; option java_package = “com.sdu.testprotoreceive”;
message Hello { string name = 1; }
IOG Version:
HelloOuterClass.Hello hello = HelloOuterClass.Hello.parseFrom(byteArray);
Wire Version:
Hello hello = hello.ADAPTER.decode(byteArray);
The objects generated by Wire is simple, they are also more natural to use.
Other languages
The tests server is in Go. Objects generation from the proto3 file is:
protoc --go_out=. *.proto
The command is generic a language to another, which makes very easy to use. To interpret generated files, the library is not on the same than other languages:
go get -u github.com/golang/protobuf/protoc-gen-go go get -u github.com/golang/protobuf/proto
Example of use:
func sendProto(w http.ResponseWriter, r *http.Request) { hello := new(hello.Hello) hello.Name = “Marie” hello.Number = 32 w.Header().Set("Content-Type", "application/x-protobuf") helloCompress, err := proto.Marshal(hello) if err != nil { log.Fatal("Compress Problem (Marshal error)") } w.Write(helloCompress) }
The last tested language is C#. Again, it is easy.
public static void Main (string[] args) { WebRequest wrGETURL = WebRequest.Create( "http://xx.xx.xx.xx:8000/"); Stream streamHello = wrGETURL.GetResponse().GetResponseStream(); Hello hello = Hello.Parser.ParseFrom(ReadFully(streamHello)); Console.WriteLine (hello.Name+" "+hello.Number+" "+hello.Bool); }
Here, the function ReadFully is only to turn Stream in byte array.
Implementation
Implementation: repository, Go Client, Go Server, Android Clients
A Playground on Github allows you to see an implementation more real. A Go repository contains the protobuf file and an application to generate objects in different languages. A Go server find in this repository a Go object, a Go client did the same. The Android Wire client find protobuf file and generates its objects in pre-build. The Android IOG client retrieves the jar file created by the Go repository.
Conclusion
Objects generation for Java, Golang and C# was done in three command lines with one protobuf file. Objects use is very similar in language tested in this article (Android, C# and Go), and looks like decode functions for JSON. For a text content, on Android client, Protobuf has several advantages such as simplicity ("build") or the gain in processing time ( "run"). Substitute JSON may even be advisable in a new project. It should nevertheless think that some tools are little or not updated (as plugin gradle by Square).
The V3 is trying to simplify use and expand the user fields, with more new languages. The binary apprehension, using new formats and JSON in place since a long time curb changement. This, I think, is the reasons which leaves Protocol Buffers little used.