Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more efficient bucket implementation #5

Open
helje5 opened this issue Nov 4, 2016 · 0 comments
Open

Add more efficient bucket implementation #5

helje5 opened this issue Nov 4, 2016 · 0 comments

Comments

@helje5
Copy link
Member

helje5 commented Nov 4, 2016

Noze.io streams work on batches of items. The items T can be any Swift type due to the use of generics. You can have streams that read and/or write bytes (UInt8) or Characters, or String-lines, or database records, or tweets, etc. In Noze the items are always processed in batches to be efficient. E.g. if a parser reads 5 lines, it won't issue 5 read calls, but just one containing all 5 lines.

The original idea was to make a 'bucket of items' a protocol, like:

protocol Bucket<T> { ... }

And then make existing bucket data structures conform to that, say:

Array<T>: Bucket<T>

and

DispatchData : Bucket<UInt8>

Any stream which would process Bucket<UInt8>'s would have been able to deal with [ UInt8 ] or DispatchData or a memory mapped file handle, etc.

Unfortunately that didn't/doesn't fly (yet?) in Swift 2/3 because you can't use a generic protocol as a type. (essentially this doesn't work: let someBucket : Bucket<UInt8>).

So as a temporary measure we skipped the issue and just used Swift arrays as 'buckets'. That is [T].
Now of course this is pretty slow in a lot of circumstances :-)

Summary: we need more efficient buckets.

One non-ideal solution is to use a base class for buckets and subclasses which wrap the various container classes. Like:

class Bucket<T> {}

class ArrayBucket<T> : Bucket<T> { let items: [ T ] }

class DispatchDataBucket : Bucket<UInt8> { let items : DispatchData }

Presumably this is much better than the [T] and may be the way to go, but obviously having an extra heap allocated wrapper isn't 1337.
Better suggestions are very welcome.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant