Home .NET Using Protocol Bufferson the .Net Platform (Part 1)

Using Protocol Bufferson the .Net Platform (Part 1)

by admin

Here is an introduction to the use of Protocol Buffers on the .Net platform in a discussion format.I will explain and show what it is and why a .Net developer needs it. This topic requires the reader to have a basic knowledge of C# and the SVN version control system. Since the amount of material exceeds the average amount of threads on hubra that do not bore the hubrauders and force them to scroll to the comments, it was decided to split it into two parts. In the first part, we’ll learn the basics and even write (not) much code!

Hello, what are Protocol Buffers?

According to the definition on the official page, Protocol Buffers (protobuf) is a way of encoding structured data in an efficient and extensible format that Google uses in almost all of its products. For most platforms, including .Net, this process is called serialization

I use Google services, but how does protobuf help me in .Net application development?

Yes you are right, Google does not write specialized libraries for.Net developers. However, there is a project protobuf-net (one of several platform implementations of protobuf), which allows you to use protobuf in .Net. It is managed by Marc Gravell – regular stackoverflow.com And a contributor to a multitude of other excellent projects So you can always ask him a question, and he’ll be happy to answer it (which is what the author of the topic abused).

Why should I use this library instead of the built-in tools?

When it comes to serialization in .Net, everyone usually thinks of the existence of binary formatter and xml serializer The next thing the developers pointout is that the former is fast and highly compressed, but only works within the .Net platform;and the latter presents data in human-readable format and serves as the basis for SOAP which in turn provides cross-platform. In fact, the claim that you always have to choose between speed and portability is taken as axiomatic! But protobuf lets you solve both problems at once.

So you claim that protobuf is as good as binary serialization and is also portable?

Yes that’s exactly it. Let’s look at a small example and see how to use protobuf-net. Let’s assume we have the following entities :

using System;
using ProtoBuf;
namespace Proto.Sample
{
public enum TaskPriority
{
Low,
Medium,
High
}
[ Serializable ] //<-- Only for BinaryFormatter
[ProtoContract]
public class Task
{
[ProtoMember(1)]
public int Id { get ; set ;}
[ProtoMember(2)]
public DateTime CreatedAt { get ; set ;}
[ProtoMember(3)]
public string CreatedBy { get ; set ;}
[ProtoMember(4)]
public TaskPriority Priority { get ; set ; }
[ProtoMember(5)]
public string Content { get ; set ; }
}
}
* This source code was highlighted with Source Code Highlighter

Protobuf-net requires special attributes, which directly follows from the main feature of the format – dependency on the order of the fields. Let’s write a performance and compression test for :

using System;
using System.Collections. Generic ;
using System.Diagnostics;
using System.IO;
using System.Runtime.Serialization.Formatters.Binary;
using ProtoBuf;
namespace Proto.Sample
{
internal class Program
{
private static void Main( string [] args)
{
var tasks = new List <Task>
{
new Task
{
Id = 1,
CreatedBy = "Steve Jobs" ,
CreatedAt = DateTime Now,
Priority = TaskPriority.High,
Content = "Invent newiPhone"
},
new Task
{
Id = 2,
CreatedBy = "Steve Ballmer" ,
CreatedAt = DateTime Now.AddDays(-7),
Priority = TaskPriority.Low,
Content = "Install own Skype"
}
};
Console WriteLine( "The test of binary formatter:" );
const string file1 = "tasks1.bin" ;
TestBinaryFormatter(tasks, file1, 1000);
TestBinaryFormatter(tasks, file1, 2000);
TestBinaryFormatter(tasks, file1, 3000);
TestBinaryFormatter(tasks, file1, 4000);
TestBinaryFormatter(tasks, file1, 5000);
Console WriteLine( The test of protobuf-net:" );
const string file2 = "tasks2.bin" ;
TestProtoBuf(tasks, file2, 1000);
TestProtoBuf(tasks, file2, 2000);
TestProtoBuf(tasks, file2, 3000);
TestProtoBuf(tasks, file2, 4000);
TestProtoBuf(tasks, file2, 5000);
Console WriteLine( "nThe comparision of file size:" );
Console WriteLine( "The size of {0}is {1}bytes" , file1, ( new FileInfo(file1)).Length);
Console WriteLine( "The size of {0}is {1}bytes" , file2, ( new FileInfo(file2)).Length);
Console ReadKey();
}
private static void TestBinaryFormatter(IList<Task> tasks, string fileName, int iterationCount)
{
var stopwatch = new Stopwatch();
var formatter = new BinaryFormatter();
using ( var file = File create(fileName))
{
stopwatch.Restart();
for ( var i = 0; i < iterationCount; i++)
{
file.Position = 0;
formatter.Serialize(file, tasks);
file.Position = 0;
var restoredTasks = ( List <Task> )formatter.Deserialize(file);
}
stopwatch.Stop();
Console WriteLine( "{0} iterations in {1} ms" , iterationCount, stopwatch.ElapsedMilliseconds);
}
}
private static void TestProtoBuf(IList<Task> tasks, string fileName, int iterationCount)
{
var stopwatch = new Stopwatch();
using ( var file = File Create(fileName))
{
stopwatch.Restart();
for ( var i = 0; i < iterationCount; i++)
{
file.Position = 0;
Serializer.Serialize(file, tasks);
file.Position = 0;
var restoredTasks = Serializer.Deserialize< List <Task> > (file);
}
stopwatch.Stop();
Console WriteLine( "{0}iterations in {1}ms" , iterationCount, stopwatch.ElapsedMilliseconds);
}
}
}
}
* This source code was highlighted with Source Code Highlighter

Results :

The test of binary formatter:
1000 iterations in 423 ms
2000 iterations in 381 ms
3000 iterations in 532 ms
4000 iterations in 660 ms
5000 iterations in 814 ms
The test of protobuf-net:
1000 iterations in 1056 ms
2000 iterations in 76 ms
3000 iterations in 129 ms
4000 iterations in 152 ms
5000 iterations in 202 ms
The comparision of file size:
The size of tasks1.bin is 710 bytes
The size of tasks2.bin is 101 bytes

* This source code was highlighted with Source Code Highlighter

As you can see, we outperformed binary serialization not only in speed, but also in compression rate. The only drawback is that protobuf-net took longer to cold-start. But you can solve this problem by using the following helper code :

var model = TypeModel.Create();
model.Add( typeof (Task), true );
var compiledModel = model.Compile(path);
compiledModel.Serialize(file, tasks);
* This source code was highlighted with Source Code Highlighter

You can see the rest of the tests and results here

Ok. Regarding speed and compression you have convinced me, but how is the portability problem solved?

You see, if there are implementations for the platform you need, the portability issue is mostly solved. And there are implementations of protobuf for more than 20 languages. You can see the full list here I should just note that for some languages there is more than one implementation. So you always have a choice.
UPD: Part 2

You may also like