Skip to content

Commit

Permalink
3.0.0 Changeset (#31)
Browse files Browse the repository at this point in the history 
* Add helpers and read only memory option

* Test case and additions to FlatBufferSerializer

* Test cases and fix a couple of bugs

* Implement read only semantics

* works a bit

* Updates to samples

* Comments and test updates

* Update README.md

* Experimental: sorted vectors

* Support quotes on field metadata

* Add oracles for sorted vectors, input buffer refactor

* Tests for sorted vectors, Making copy constructors default option

* String comparer tests

* Sorted vectors operational, enum bug fix

* More test -- performance optimization for binary search of string

* Revert back to attribute

* Basic include support

* More tests, espcially for includes

* Support for includes from different directories. Add copyright notice to samples. Prepare for release

* Update samples

* Update README.md

* Consolidate built in types in one location.

* In place vector sort

* Sorted vector performance optimizations.

* Fix deserialization performance regression

* Tests for comparers, hide some things from intellisense

* More tests, documentation

* Last few test cases, always serialize keys

* Update benchmarks

* Add 2.1.0 benchmarks

* Transparency around benchmarks

Transparency around benchmarks
  • Loading branch information
@jamescourtney
jamescourtney authored Mar 30, 2020
1 parent 6710a36 commit 1792322
Show file tree
Hide file tree
Showing 102 changed files with 5,542 additions and 856 deletions.
10 changes: 7 additions & 3 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
## FlatSharp

FlatSharp is Google's FlatBuffers serialization format implemented in C#, for C#. FlatBuffers is a zero-copy binary serialization format intended for high-performance scenarios. FlatSharp leverages the latest and greatest from .NET in the form of ```Memory<T>``` and ```Span<T>```. As such, FlatSharp's safe-code implementations are often faster than other implementations using unsafe code. FlatSharp aims to provide 3 core priorities:
- Full safety (no unsafe code or IL generation -- more on that below).
- Speed
Expand Down Expand Up @@ -32,7 +31,7 @@ public class Person : object
[FlatBufferItem(1)] public virtual string Name { get; set; }
[FlatBufferItem(2, Deprecated = true)] public virtual Person Parent { get; set; }
[FlatBufferItem(3)] public virtual IList<Person> Children { get; set; }
[FlatBufferItem(4, DefaultValue = Color.Blue)] public virtual double FavoriteColor { get; set; } = Color.Blue;
[FlatBufferItem(4, DefaultValue = Color.Blue)] public virtual Color FavoriteColor { get; set; } = Color.Blue;
[FlatBufferItem(5)] public virtual Location Position { get; set; }
}

Expand Down Expand Up @@ -96,6 +95,9 @@ FlatSharp supports some interesting features not covered here. Please visit the
- [Deserialization options (Lazy, Greedy, and everything in between)](samples/Example1-SerializerOptions/SerializerOptionsExample.cs)
- [IO Options](samples/Example4-IOOptions/)
- [gRPC](samples/Example5-gRPC/)
- [Copy Constructors](samples/Example6-CopyConstructors/)
- [FBS Includes](samples/Example7-Includes/)
- [Sorted Vectors](samples/Example8-SortedVectors/)

### Internals
FlatSharp works by generating subclasses of your data contracts based on the schema that you define. That is, when you attempt to deserialize a ```MonsterTable``` object, you actually get back a subclass of ```MonsterTable```, which has properties defined in such a way as to index into the buffer, according to the deserialization mode specified (greedy, lazy, etc).
Expand All @@ -112,10 +114,12 @@ At its core, FlatSharp is a tool to convert a FlatBuffer schema into a pile of s
### Performance & Benchmarks
FlatSharp is really fast. This is primarily thanks to new changes in C# with Memory and Span, as well as FlatBuffers itself exposing a very simple type system that makes optimization simple. FlatSharp has a default serializer instance (```FlatBuffersSerializer.Default```), however it is possible to tune the serializer by creating your own with a custom ```FlatBufferSerializerOptions``` instance. More details are available in the [samples solution](samples/Example1-SerializerOptions/SerializerOptionsExample.cs).

The FlatSharp benchmarks were run on .NET Core 2.1, using a C# approximation of [Google's FlatBuffer benchmark](https://github.com/google/flatbuffers/tree/benchmarks/benchmarks/cpp/FB). The FlatSharp benchmarks use this schema, but with the following parameters:
The FlatSharp benchmarks were run on .NET Core 2.1, using a C# approximation of [Google's FlatBuffer benchmark](https://github.com/google/flatbuffers/tree/benchmarks/benchmarks/cpp/FB), which can be found [here](src/Benchmark).The FlatSharp benchmarks use this schema, but with the following parameters:
- Vector length = 3 or 30
- Traversal count = 1 or 5

The full results for each version of FlatSharp can be viewed in the [benchmarks folder](benchmarks).

The benchmarks test 4 different serialization frameworks:
- FlatSharp
- Protobuf.NET
Expand Down
62 changes: 62 additions & 0 deletions benchmarks/2.1.0/Benchmark.FBBench.FBDeserializeBench-report-github.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
``` ini

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.18363
AMD Ryzen 9 3900X 12-Core Processor (Max: 3.80GHz), 1 CPU, 24 logical and 12 physical cores
.NET Core SDK=3.1.200
[Host] : .NET Core 2.1.16 (CoreCLR 4.6.28516.03, CoreFX 4.6.28516.10), 64bit RyuJIT
ShortRun : .NET Core 2.1.16 (CoreCLR 4.6.28516.03, CoreFX 4.6.28516.10), 64bit RyuJIT

Job=ShortRun IterationCount=3 LaunchCount=1
WarmupCount=3

```
| Method | TraversalCount | DeserializeOption | VectorLength | Mean | Error | StdDev |
|---------------------------------- |--------------- |------------------- |------------- |------------:|-------------:|-----------:|
| **FlatSharp_ParseAndTraverse** | **1** | **Greedy** | **3** | **760.9 ns** | **32.386 ns** | **1.8299 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | Greedy | 3 | 700.3 ns | 44.398 ns | 2.5086 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **Greedy** | **30** | **5,238.0 ns** | **233.865 ns** | **13.2138 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | Greedy | 30 | 5,083.7 ns | 349.848 ns | 19.7671 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **GreedyMutable** | **3** | **709.9 ns** | **16.872 ns** | **0.9533 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | GreedyMutable | 3 | 685.6 ns | 49.776 ns | 2.8125 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **GreedyMutable** | **30** | **4,950.2 ns** | **409.436 ns** | **23.1339 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | GreedyMutable | 30 | 4,908.3 ns | 1,555.206 ns | 87.8720 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **Lazy** | **3** | **650.9 ns** | **25.902 ns** | **1.4635 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | Lazy | 3 | 509.8 ns | 3.948 ns | 0.2231 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **Lazy** | **30** | **5,013.5 ns** | **132.336 ns** | **7.4772 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | Lazy | 30 | 3,582.7 ns | 34.916 ns | 1.9728 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **PropertyCache** | **3** | **691.1 ns** | **14.409 ns** | **0.8141 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | PropertyCache | 3 | 526.5 ns | 13.909 ns | 0.7859 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **PropertyCache** | **30** | **5,362.0 ns** | **61.570 ns** | **3.4788 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | PropertyCache | 30 | 3,710.4 ns | 102.218 ns | 5.7755 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **VectorCache** | **3** | **751.7 ns** | **8.854 ns** | **0.5002 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | VectorCache | 3 | 596.5 ns | 6.371 ns | 0.3600 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **VectorCache** | **30** | **5,570.1 ns** | **130.303 ns** | **7.3623 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | VectorCache | 30 | 4,157.6 ns | 40.305 ns | 2.2773 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **VectorCacheMutable** | **3** | **739.3 ns** | **16.095 ns** | **0.9094 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | VectorCacheMutable | 3 | 582.9 ns | 16.944 ns | 0.9573 ns |
| **FlatSharp_ParseAndTraverse** | **1** | **VectorCacheMutable** | **30** | **5,557.5 ns** | **462.247 ns** | **26.1178 ns** |
| FlatSharp_ParseAndTraversePartial | 1 | VectorCacheMutable | 30 | 3,921.4 ns | 236.591 ns | 13.3678 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **Greedy** | **3** | **962.4 ns** | **25.491 ns** | **1.4403 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | Greedy | 3 | 856.1 ns | 49.425 ns | 2.7926 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **Greedy** | **30** | **7,577.8 ns** | **176.417 ns** | **9.9679 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | Greedy | 30 | 6,211.7 ns | 89.155 ns | 5.0374 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **GreedyMutable** | **3** | **928.1 ns** | **30.454 ns** | **1.7207 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | GreedyMutable | 3 | 803.1 ns | 12.615 ns | 0.7128 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **GreedyMutable** | **30** | **6,886.8 ns** | **85.276 ns** | **4.8182 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | GreedyMutable | 30 | 5,769.5 ns | 816.899 ns | 46.1563 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **Lazy** | **3** | **3,006.8 ns** | **57.782 ns** | **3.2648 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | Lazy | 3 | 2,312.6 ns | 61.479 ns | 3.4737 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **Lazy** | **30** | **24,528.2 ns** | **571.328 ns** | **32.2811 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | Lazy | 30 | 17,534.4 ns | 128.936 ns | 7.2851 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **PropertyCache** | **3** | **2,771.8 ns** | **89.119 ns** | **5.0354 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | PropertyCache | 3 | 1,949.8 ns | 80.360 ns | 4.5405 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **PropertyCache** | **30** | **25,256.0 ns** | **308.697 ns** | **17.4419 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | PropertyCache | 30 | 17,697.7 ns | 345.900 ns | 19.5440 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **VectorCache** | **3** | **1,105.8 ns** | **34.505 ns** | **1.9496 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | VectorCache | 3 | 795.8 ns | 33.950 ns | 1.9183 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **VectorCache** | **30** | **8,486.3 ns** | **909.611 ns** | **51.3947 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | VectorCache | 30 | 5,476.0 ns | 82.795 ns | 4.6781 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **VectorCacheMutable** | **3** | **1,057.9 ns** | **18.340 ns** | **1.0363 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | VectorCacheMutable | 3 | 756.3 ns | 51.704 ns | 2.9214 ns |
| **FlatSharp_ParseAndTraverse** | **5** | **VectorCacheMutable** | **30** | **8,259.9 ns** | **511.685 ns** | **28.9112 ns** |
| FlatSharp_ParseAndTraversePartial | 5 | VectorCacheMutable | 30 | 5,183.4 ns | 361.414 ns | 20.4206 ns |
22 changes: 22 additions & 0 deletions benchmarks/2.1.0/Benchmark.FBBench.FBSerializeBench-report-github.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
``` ini

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.18363
AMD Ryzen 9 3900X 12-Core Processor (Max: 3.80GHz), 1 CPU, 24 logical and 12 physical cores
.NET Core SDK=3.1.200
[Host] : .NET Core 2.1.16 (CoreCLR 4.6.28516.03, CoreFX 4.6.28516.10), 64bit RyuJIT
ShortRun : .NET Core 2.1.16 (CoreCLR 4.6.28516.03, CoreFX 4.6.28516.10), 64bit RyuJIT

Job=ShortRun IterationCount=3 LaunchCount=1
WarmupCount=3

```
| Method | VectorLength | Mean | Error | StdDev |
|----------------------------- |------------- |------------:|-----------:|-----------:|
| **Google_FlatBuffers_Serialize** | **3** | **1,160.86 ns** | **32.828 ns** | **1.8548 ns** |
| FlatSharp_GetMaxSize | 3 | 64.94 ns | 17.989 ns | 1.0164 ns |
| FlatSharp_Serialize | 3 | 615.20 ns | 35.624 ns | 2.0128 ns |
| PBDN_Serialize | 3 | 1,331.25 ns | 27.770 ns | 1.5690 ns |
| **Google_FlatBuffers_Serialize** | **30** | **9,276.51 ns** | **430.944 ns** | **24.3491 ns** |
| FlatSharp_GetMaxSize | 30 | 254.81 ns | 4.097 ns | 0.2315 ns |
| FlatSharp_Serialize | 30 | 4,600.13 ns | 167.040 ns | 9.4381 ns |
| PBDN_Serialize | 30 | 9,034.99 ns | 566.687 ns | 32.0188 ns |
30 changes: 30 additions & 0 deletions benchmarks/2.1.0/Benchmark.FBBench.OthersDeserializeBench-report-github.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
``` ini

BenchmarkDotNet=v0.11.1, OS=Windows 10.0.18363
AMD Ryzen 9 3900X 12-Core Processor (Max: 3.80GHz), 1 CPU, 24 logical and 12 physical cores
.NET Core SDK=3.1.200
[Host] : .NET Core 2.1.16 (CoreCLR 4.6.28516.03, CoreFX 4.6.28516.10), 64bit RyuJIT
ShortRun : .NET Core 2.1.16 (CoreCLR 4.6.28516.03, CoreFX 4.6.28516.10), 64bit RyuJIT

Job=ShortRun IterationCount=3 LaunchCount=1
WarmupCount=3

```
| Method | TraversalCount | VectorLength | Mean | Error | StdDev |
|------------------------------------------- |--------------- |------------- |------------:|------------:|-----------:|
| **Google_Flatbuffers_ParseAndTraverse** | **1** | **3** | **1,014.3 ns** | **18.46 ns** | **1.043 ns** |
| Google_Flatbuffers_ParseAndTraversePartial | 1 | 3 | 698.7 ns | 110.71 ns | 6.255 ns |
| PBDN_ParseAndTraverse | 1 | 3 | 3,302.1 ns | 109.82 ns | 6.205 ns |
| PBDN_ParseAndTraversePartial | 1 | 3 | 3,257.4 ns | 123.37 ns | 6.970 ns |
| **Google_Flatbuffers_ParseAndTraverse** | **1** | **30** | **8,277.3 ns** | **82.52 ns** | **4.663 ns** |
| Google_Flatbuffers_ParseAndTraversePartial | 1 | 30 | 5,381.1 ns | 68.28 ns | 3.858 ns |
| PBDN_ParseAndTraverse | 1 | 30 | 24,073.8 ns | 2,280.50 ns | 128.852 ns |
| PBDN_ParseAndTraversePartial | 1 | 30 | 22,870.8 ns | 935.38 ns | 52.851 ns |
| **Google_Flatbuffers_ParseAndTraverse** | **5** | **3** | **4,746.7 ns** | **119.53 ns** | **6.754 ns** |
| Google_Flatbuffers_ParseAndTraversePartial | 5 | 3 | 3,267.7 ns | 61.69 ns | 3.485 ns |
| PBDN_ParseAndTraverse | 5 | 3 | 3,535.4 ns | 179.76 ns | 10.157 ns |
| PBDN_ParseAndTraversePartial | 5 | 3 | 3,379.7 ns | 269.58 ns | 15.232 ns |
| **Google_Flatbuffers_ParseAndTraverse** | **5** | **30** | **42,207.3 ns** | **1,014.03 ns** | **57.294 ns** |
| Google_Flatbuffers_ParseAndTraversePartial | 5 | 30 | 25,942.9 ns | 1,282.82 ns | 72.482 ns |
| PBDN_ParseAndTraverse | 5 | 30 | 25,402.2 ns | 2,386.51 ns | 134.842 ns |
| PBDN_ParseAndTraversePartial | 5 | 30 | 24,091.7 ns | 759.31 ns | 42.903 ns |
Loading

0 comments on commit 1792322

Please sign in to comment.