Better handling of Future handles #53
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Completely refactored the Subspace API to fix most common issues and prevent bugs when dealing with embedded tuples
- subspace.Keys.xxx to concat/extract/work with binary keys, ex: subspace.Keys[Slice.FromString("Hello")]
- subspace.Partition.xxx to create sub-partitions of subspaces, ex: subspace.Partition.By("Messages")
- subspace.Tuples.xxx to deal with everything Tuple related, ex: subspace.Tuples.EncodeKey(123, "abc")
- Made the interface "IFdbSubspace" the primary entry point, instead of class FdbSubspace
- Added low-level methods on IFdbSubspace, moved most extension methods on Keys/Partition/Tuples helper struct
- See #40 for discussion, and how to update your code with the new API !
- Don't hate me too much (^_^;)
…NILs inside tuples - Changed encoding of embedded tuples to be <03>...<00>, instead of <03>...<04> - NILs inside embedded tuples are now escaped as <00><FF> instead of only <00> - Ordering should now be: (1, (2)) then (1, (2), 3) then (1, (2, 3)) then (1, (3)) - Introduced the TupleReader and TupleWriter structs, which track the current context of encoding/decoding (currently only the Depth) - Regular top-level tuples are at depth 0, embedded tuples starts at depth 1 or more - ToRange() on partial embedded tuples are still broken regarding ordering!
- Renamed methods on staticclass FdbTuple to be in synch with the new naming conventions - FdbTuple.EncodeKey() *could* be used to encoded values, but the name suggest that it tries to keep ordering properties that may be overkill for values - Simplified names with replacing "Range" by either the plural (EncodeKeys) or nothing if there are no overload resolution ambiguity possible
- Responding to feedback on the new names - ByKey(...) better matches the conventions of location.Tuples.EncodeKey(...)
- Removed subspace.Tuples.EncodeKey(object[]) because it encourages bad practice - To make up for it, added overloads of EncodeKey, Append that take up to 8 generic types - Also added a struct tuple with 5 items, because composite keys with 5 items are not infrequent. - There is no DecodeKey with 6 or more items because there is no generic typed tuple with that many items
…, and Wrap - There are too many variants to create a tuple from an array, so we need to simplifiy - Testing things with a FdbTuple.FromArray<T> and FromSequence<T>, which should be explicit enough - Still need to decide what to do between Create and Wrap. If they both stay, one need to imply copying the array, the other simply wrapping the array?
… and subpsace.Tuples - subspace.Tuples.ToRange() has the same behavior as the old subspace.ToRange() (prefix+\x00 .. prefix+\xFF) - subspace.Keys.ToRange() includes ALL the keys, and is the equivalent of FdbKeyRange.StartsWith(subspace.Prefix) - If you had "subspace.ToRange()" before, you should first change it to "subspace.Tuples.ToRange()" to stay compatible, and then decide if you actually needed "subspace.Keys.ToRange()" or not.
- FdbTuple.Create(123, "abc").With((x, y) => ... ) will have x == 123 and y == "abc" - tuple.OfSize(3).Foo(...) will verify that tuple.Count == 3, and throw if this is not the case. - inspired by https://github.com/louthy/language-ext !
- allows custom type codecs to output embedded tuples
…, and ElementAtOrDefaultAsync()
- the Stopwatch is now private, and we only expose TimeSpans - tr.Context.Elapsed is the time since the last reset/retry - tr.Context.ElapsedTotal is the time since the creation of the transaction
…ansforms, added observers and initial implementation for sorting
- AsyncFilterExpression and AsyncTransformExpression help combine filters and selector that could be either sync or async.
- Fixed bug where in a foo.Select(...).Where(...) the Where could be evaluated before the Select
- Added .Observe((x) => { ...}) that can execute actions on each element of the sequence, but without changing the result
- Added the notion of QueryStatistics<T> that can be used to measure what flows through some part of the query
- Added TakeWhile(.., out QueryStatistics<bool> stopped), and other like WithCountStatistics(..) or WithSizeStatistics(...)
- Fixed most aggregate methods to call into FdbAsyncIterator<T>.ExecuteAsync(..) whenever possible, instead of calling MoveNext() manually.
- Added initial support for sorting (not complete)
- try to preserve the callstack if a lambda in a Select(..) or Where(..) throws
- Added enum FdbTupleSegmentType - Made all the FdbTupleParser.ParseXYZ public - Added FdbTupleParser.VisitNext<T>(...) helper to decode one segment at a time - Added FdbTupleParser.Skip(...) to skip N first tuple segments (ie: decoder wants to decode only the 3rd segment...)
…able to build on .NET 4.0 - ExceptionDispatchInfo does not exist in NET40 so we use regular Exceptions - renamed Maybe<T>.ThrowIfFailed() to ThrowForNonSuccess() so that it matches TaskAwaiter<T>.ThrowForNonSuccess()
…ecodeKeys and EncodeValues/DecodeValues - Matches with the names for tuples, and also remove ambiguity between encoding keys and values
…, and somes fixed for building on .NET 4.0
- IFdbTransactional don't compose very well with each other - It is better to only expose methods that take IFdb[ReadOnly]Transaction arguments, and use the various retry loops (db.ReadWriteAsync, etc...) to compose one or more layer calls inside the same transaction.
…ides in the various experimental layers - See Issue #45 for rationale being the change
- add Last<R>() on IFdbTuple
- added the "Base" property that returns the untyped original subspace
- split subspaces into 3 categories: regular "untyped" IFdbSubspace, dynamic "tuple-based" IFdbDynamicSubspace, and encoder-based IFdbEncoderSubspace<....> - each subspace category comes with its own set of Keys / Partition fields that have different methods, for each use case - a regular subspace can be "enhanced" via subspace.Using(TypeSystem.Tuples), or subspace.UsingEncoder(myCustomEncoder) - TypeSystem.Tuples contains the basic of a "Type System" which uses the Tuple Encoding format to generate Keys
…-dotnet-client into refac_tuples
- The Type System is a "Key Encoding" which has methods like GetDynamicEncoder(), GetEncoder<T>(..) and returns specialized encoders - Key Encoders have a specific shape (dynamic vs static) and produce keys using the correct encoding scheme - Added subspace encoders up to T4 - cleaned up some old classes, moved some files around
- FdbTuple.Wrap(..) returns a tuple without copying the items - FdbTuple.FromObjects(...) returns a tuple with a copy of the items
…e<T1, .., TN> - Tuple<..> to FdbTuple<..> is implicit - FdbTuple<..> to Tuple<..> is explicit
- Similar to With<...>(...), but return a typed tuple instead
… n) items of a Tuples - allows things like t.Truncate(3).As<string, int, Guid>() or t.Truncate(3).With((string x, int y, Guid z) => ...)
- must encode '<' in code blocks to please GitHub
Stupid GitHub is Stupid
Fix error code for invalid_mutation_type
…llbacks - maybe remove this when we are done?
…hread, and defer the handling to the thread pool - When callback are invoked inline, we have to return ASAP to the caller (who holds a lock) so we move to the TP - If we have already been moved to the TP, then we can call OnReady inline (already on the TP!) else we defer to the TP at that time. - Use the 32 lower bits as the key in the future dictionary, because IntPtr is not IEquatable<IntPtr> and uses a boxing key comparer - Added a "dead" flag to future context, and implemented refcounting down to 0 to remove the context from the global MapToException - Changed the way transaction are cancelled, by waiting for FDB_C to call us back for each pending future (need to map `transaction_cancelled` into a TaskCancelledException for this to work) - Started working on a Watch refactoring
…ost of copying the execution context for nothing - Used internally to trigger cancellation, and we do not care about the ExecutionContext to do that.
Contributor
Author
|
I had to move this PR to the refac_tuples branch, which creates a lot of noise in the commits. I'm closing this PR and re-opening it as #54. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is tracking PR for the work of rewriting the way FDBFuture* handles are wrapped to .NET's
Task<T>, as detailed in #48.This PR changes the following:
FdbFutureHandlehas been removed, and Futures handles are now back to IntPtr. This helps avoid problems when the GC wants to destroy handles at unwanted times.IFdbFuture, so that they can be put in the same dictionary or list.