[DTP-1033] Add LiveObjects edit API #1948
Conversation
This reduces code duplication and prepares for the changes in next commits where a StateMessage from LiveObjects plugin will be using these encoding functions to encode its state.
…a `MessageEncoding`
…existing objects
|
Warning There were issues while running some tools. Please review the errors and either fix the tool’s configuration or disable the tool if it’s a critical failure. 🔧 ast-grep (0.31.1)test/realtime/live_objects.test.jsAn unexpected error occurred while running ast-grep. WalkthroughThis pull request introduces enhancements to the message encoding and state management system across multiple files. Key modifications include the addition of a new Changes
Assessment against linked issues
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (1)
⏰ Context from checks skipped due to timeout of 90000ms (6)
🔇 Additional comments (5)
Finishing Touches
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
github-actions
bot
temporarily deployed
to
staging/pull/1948/bundle-report
github-actions
bot
temporarily deployed
to
staging/pull/1948/features
github-actions
bot
temporarily deployed
to
staging/pull/1948/typedoc
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (10)
src/plugins/liveobjects/statemessage.ts (1)
165-179: Simplify Null Checks with Optional ChainingConsider using optional chaining to simplify the assignment of
message.operationandmessage.object.Apply this diff to refactor the code:
- message.operation = message.operation ? StateMessage._encodeStateOperation(message.operation, encodeFn) : undefined; - message.object = message.object ? StateMessage._encodeStateObject(message.object, encodeFn) : undefined; + message.operation = StateMessage._encodeStateOperation(message.operation, encodeFn); + message.object = StateMessage._encodeStateObject(message.object, encodeFn);Since the
_encodeStateOperationand_encodeStateObjectmethods likely handleundefinedinputs appropriately, this refactor simplifies the code.src/common/lib/types/message.ts (1)
171-181: Guard Against Null Cipher OptionsIn the
encodefunction, the checkif (cipherOptions != null && cipherOptions.cipher)may not properly handle all falsy values. Consider simplifying the condition.Apply this diff to improve the condition:
- if (cipherOptions != null && cipherOptions.cipher) { + if (cipherOptions?.cipher) { return encrypt(msg, cipherOptions); } else { return msg; }This uses optional chaining to make the code more concise.
🧰 Tools
🪛 Biome (1.9.4)
[error] 176-176: Change to an optional chain.
Unsafe fix: Change to an optional chain.
(lint/complexity/useOptionalChain)
src/plugins/liveobjects/livemap.ts (2)
522-522: Update Log Message with Correct Object IDIn the
_applyMapCreatemethod, correct the object ID reference in the log message to provide accurate debugging information.Apply this diff to fix the log message:
`skipping applying MAP_CREATE op on a map instance as it was already applied before; objectId=${this.getObjectId()}`Ensure that the object ID is correctly included in the message.
601-601: Consistent Logging in_applyMapRemoveMethodIn the
_applyMapRemovemethod, ensure that the logging is consistent with other methods and includes all relevant information for debugging.Verify and adjust the log message as necessary.
src/common/lib/types/presencemessage.ts (1)
137-141: Consider removing the non-null assertion operator.The code assumes encoding will always be defined by using the non-null assertion operator (
!). Consider handling the undefined case explicitly for better type safety.- encoding: encoding!, + encoding: encoding ?? '',Also applies to: 151-155
src/common/lib/types/protocolmessage.ts (1)
183-185: Consider using array join for better performance.The string concatenation could be optimized by using array join instead of string concatenation.
- result += - '; state=' + toStringArray(liveObjectsPlugin.StateMessage.fromValuesArray(msg.state, Utils, MessageEncoding)); + const stateArray = liveObjectsPlugin.StateMessage.fromValuesArray(msg.state, Utils, MessageEncoding); + result += ['; state=', toStringArray(stateArray)].join('');src/plugins/liveobjects/livecounter.ts (2)
53-74: Consider adding error message constants.The error message and code are hardcoded. Consider extracting these to constants for better maintainability and reusability.
+const COUNTER_INCREMENT_ERROR = { + message: 'Counter value increment should be a number', + code: 40013, + statusCode: 400 +}; createCounterIncMessage(amount: number): StateMessage { if (typeof amount !== 'number') { - throw new this._client.ErrorInfo('Counter value increment should be a number', 40013, 400); + throw new this._client.ErrorInfo(COUNTER_INCREMENT_ERROR.message, COUNTER_INCREMENT_ERROR.code, COUNTER_INCREMENT_ERROR.statusCode); } // ... rest of the code
76-87: Consider reusing error constants for decrement.The error message for decrement is similar to increment. Consider reusing the same error constants with a different message.
+const COUNTER_DECREMENT_ERROR = { + message: 'Counter value decrement should be a number', + code: 40013, + statusCode: 400 +}; decrement(amount: number): Promise<void> { if (typeof amount !== 'number') { - throw new this._client.ErrorInfo('Counter value decrement should be a number', 40013, 400); + throw new this._client.ErrorInfo(COUNTER_DECREMENT_ERROR.message, COUNTER_DECREMENT_ERROR.code, COUNTER_DECREMENT_ERROR.statusCode); } return this.increment(-amount); }src/plugins/liveobjects/liveobjects.ts (1)
153-168: Consider adding batch size validation.The method accepts an array of state messages but doesn't validate the batch size. Consider adding a maximum batch size check to prevent memory issues with large batches.
+const MAX_BATCH_SIZE = 100; publish(stateMessages: StateMessage[]): Promise<void> { + if (stateMessages.length > MAX_BATCH_SIZE) { + throw this._client.ErrorInfo.fromValues({ + message: `Maximum batch size exceeded (was ${stateMessages.length}; limit is ${MAX_BATCH_SIZE})`, + code: 40009, + statusCode: 400 + }); + } if (!this._channel.connectionManager.activeState()) { throw this._channel.connectionManager.getError(); } // ... rest of the codesrc/common/lib/client/realtimechannel.ts (1)
629-629: Consider adding error handling for missing plugin.The ternary operation could throw if the plugin is missing. Consider adding a more descriptive error message.
- ? this.client._LiveObjectsPlugin.StateMessage.decode(msg, options, MessageEncoding) - : Utils.throwMissingPluginError('LiveObjects'), + ? this.client._LiveObjectsPlugin.StateMessage.decode(msg, options, MessageEncoding) + : Utils.throwMissingPluginError('LiveObjects', 'Cannot decode state message without LiveObjects plugin'),
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (11)
src/common/lib/client/baseclient.ts(2 hunks)src/common/lib/client/realtimechannel.ts(3 hunks)src/common/lib/transport/protocol.ts(1 hunks)src/common/lib/types/message.ts(6 hunks)src/common/lib/types/presencemessage.ts(3 hunks)src/common/lib/types/protocolmessage.ts(3 hunks)src/plugins/liveobjects/livecounter.ts(3 hunks)src/plugins/liveobjects/livemap.ts(5 hunks)src/plugins/liveobjects/liveobjects.ts(1 hunks)src/plugins/liveobjects/statemessage.ts(5 hunks)test/realtime/live_objects.test.js(1 hunks)
🧰 Additional context used
📓 Learnings (1)
src/plugins/liveobjects/livecounter.ts (1)
Learnt from: VeskeR
PR: ably/ably-js#1897
File: src/plugins/liveobjects/livecounter.ts:93-93
Timestamp: 2024-11-12T07:31:53.691Z
Learning: In the `LiveCounter` class's `_applyCounterCreate` method, it's intentional to increment the counter's value using `+=` instead of initializing it with `=` because the counter may have a pre-existing non-zero value.
🪛 Biome (1.9.4)
src/common/lib/types/message.ts
[error] 176-176: Change to an optional chain.
Unsafe fix: Change to an optional chain.
(lint/complexity/useOptionalChain)
⏰ Context from checks skipped due to timeout of 90000ms (6)
- GitHub Check: test-browser (webkit)
- GitHub Check: test-node (20.x)
- GitHub Check: test-browser (firefox)
- GitHub Check: test-node (18.x)
- GitHub Check: test-node (16.x)
- GitHub Check: test-browser (chromium)
🔇 Additional comments (21)
src/plugins/liveobjects/statemessage.ts (4)
1-2: Import Consistency: Ensure Correct PathsVerify that the import paths for
MessageEncodingandUtilsare correct and accessible in the project structure to prevent any module resolution issues.
5-8: Type Definition ClarityThe
StateDataEncodeFunctiontype is well-defined and clear. This enhances code readability and maintainability.
154-157: Constructor Parameter UpdateUpdating the constructor to accept
_utilsand_messageEncodingpromotes better dependency injection and modularity.
250-254: Ensure Proper Error Handling in Data DecodingIn the
_decodeStateDatamethod, after decoding the data, ensure that any errors are appropriately handled to prevent potential runtime exceptions.src/common/lib/types/message.ts (4)
134-137: Parameter Renaming for ClarityRenaming the parameter from
optionstocipherOptionsin theencryptfunction enhances readability by making the purpose of the parameter clear.
146-162: Proper Handling of Encoding StringsEnsure that the encoding string concatenation correctly handles cases where
encodingisnullor an empty string to prevent leading or trailing slashes.Test with different values of
encodingto confirm the correctness of the concatenated encoding string.
220-247: Handle Binary Data Consistently Across ProtocolsIn
encodeDataForWireProtocol, ensure that binary data is handled consistently for both JSON and MsgPack protocols, and that any required transformations are correctly applied.Review the handling of binary data to confirm that it adheres to protocol specifications.
445-451: ExposeMessageEncodingAppropriatelyExporting
MessageEncodingprovides a convenient way to access encoding functions, but ensure that this does not inadvertently expose internal implementations or create security risks.Review the accessibility of the encoding functions to confirm that they are intended for public use.
src/plugins/liveobjects/livemap.ts (4)
173-209: Validate Map Key and Value TypesThe
createMapSetMessagemethod correctly validates the key and value types, which is essential for data integrity.
228-246: Validate Map Key Type increateMapRemoveMessageThe validation of the key in
createMapRemoveMessageis appropriate, ensuring only string keys are accepted.
548-548: Ensure Correct Timeserial ComparisonIn the
_applyMapSetmethod, verify that the timeserial comparison logic accurately determines whether to apply the operation, accounting for potential edge cases.Review the timeserial comparison to confirm that it's consistent with the desired CRDT semantics.
267-267: Correct Log Message FormattingIn the
applyOperationmethod, ensure that template literals are used correctly to avoid any runtime errors.Check line 267 for any formatting issues in the log message.
Apply this diff if needed:
`skipping ${op.action} op: op timeserial ${opOriginTimeserial.toString()} <= site timeserial ${this._siteTimeserials[opSiteCode]?.toString()}; objectId=${this.getObjectId()}`Ensure that all variables used in the template string are correctly defined.
src/common/lib/transport/protocol.ts (1)
23-24: LGTM! Enhanced ack requirement logic.The changes improve type safety and readability while extending acknowledgment support to state messages.
src/common/lib/types/presencemessage.ts (1)
3-9: LGTM! Clean import organization.The imports are well-organized and properly grouped.
src/common/lib/client/baseclient.ts (1)
21-21: LGTM! Consistent export pattern.The MessageEncoding export follows the established pattern for exposing utilities to plugins.
Also applies to: 185-185
src/common/lib/types/protocolmessage.ts (1)
6-10: LGTM! Clean import organization.The imports are well-organized and properly grouped.
src/plugins/liveobjects/livecounter.ts (1)
39-51: LGTM! Well-documented increment method.The implementation is clean and follows the async pattern correctly. The documentation clearly explains that the change is applied when the operation is echoed back.
src/common/lib/client/realtimechannel.ts (1)
514-523: LGTM! Clean implementation of state message sending.The implementation follows the promise pattern correctly and uses the protocol message construction appropriately.
test/realtime/live_objects.test.js (3)
2063-2089: LGTM! Comprehensive test coverage for LiveCounter increment.The test cases cover a good range of values including edge cases like MAX_SAFE_INTEGER.
2102-2124: LGTM! Thorough input validation testing.The test cases cover all possible invalid input types for increment operation.
2343-2348: LGTM! Well-organized test scenarios.The test scenarios are well-organized into logical groups making the test suite maintainable.
| // we can infer the format used by client by inspecting with what arguments this method was called. | ||
| // if JSON protocol is being used, the JSON.stringify() will be called and this toJSON() method will have a non-empty arguments list. | ||
| // MSGPack protocol implementation also calls toJSON(), but with an empty arguments list. | ||
| const format = arguments.length > 0 ? this._utils.Format.json : this._utils.Format.msgpack; |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Clarify Format Detection Logic
In the toJSON method, the logic for inferring the format based on the arguments object can be unreliable and may cause issues in strict mode. Consider refactoring this logic to explicitly pass the format as a parameter or use a more reliable detection method.
Would you like assistance in refactoring this method to improve reliability?
| private static _encodeStateData(data: StateData, encodeFn: StateDataEncodeFunction): StateData { | ||
| const { value: newValue, encoding: newEncoding } = encodeFn(data?.value, data?.encoding); | ||
|
|
||
| // toBuffer returns a datatype understandable by | ||
| // that platform's msgpack implementation (Buffer in node, Uint8Array in browsers) | ||
| return { | ||
| value: platform.BufferUtils.toBuffer(value), | ||
| encoding, | ||
| ...data, | ||
| value: newValue, | ||
| encoding: newEncoding!, |
There was a problem hiding this comment.
Avoid Non-Null Assertion Operator
Avoid using the non-null assertion operator ! for newEncoding. Instead, ensure that newEncoding is properly handled to prevent potential undefined or null values.
Apply this diff to safely handle newEncoding:
return {
...data,
value: newValue,
- encoding: newEncoding!,
+ encoding: newEncoding ?? data.encoding,
};This change ensures that if newEncoding is undefined, the original data.encoding is preserved.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| private static _encodeStateData(data: StateData, encodeFn: StateDataEncodeFunction): StateData { | |
| const { value: newValue, encoding: newEncoding } = encodeFn(data?.value, data?.encoding); | |
| // toBuffer returns a datatype understandable by | |
| // that platform's msgpack implementation (Buffer in node, Uint8Array in browsers) | |
| return { | |
| value: platform.BufferUtils.toBuffer(value), | |
| encoding, | |
| ...data, | |
| value: newValue, | |
| encoding: newEncoding!, | |
| private static _encodeStateData(data: StateData, encodeFn: StateDataEncodeFunction): StateData { | |
| const { value: newValue, encoding: newEncoding } = encodeFn(data?.value, data?.encoding); | |
| return { | |
| ...data, | |
| value: newValue, | |
| encoding: newEncoding ?? data.encoding, |
| // data can be decoded partially and throw an error on a later decoding step. | ||
| // so we need to reassign the data and encoding values we got, and only then throw an error if there is one | ||
| const { data, encoding, error } = await decodeData(message.data, message.encoding, inputContext); | ||
| message.data = data; |
There was a problem hiding this comment.
Graceful Error Handling in Data Decoding
In the decode function, after decoding data, ensure that any errors are handled gracefully and that the message object remains in a valid state even if partial decoding occurs.
Consider updating the error handling to avoid the message being left in an inconsistent state.
| // we can infer the format used by client by inspecting with what arguments this method was called. | ||
| // if JSON protocol is being used, the JSON.stringify() will be called and this toJSON() method will have a non-empty arguments list. | ||
| // MSGPack protocol implementation also calls toJSON(), but with an empty arguments list. | ||
| const format = arguments.length > 0 ? Utils.Format.json : Utils.Format.msgpack; | ||
| const { data, encoding } = encodeDataForWireProtocol(this.data, this.encoding, format); | ||
|
|
There was a problem hiding this comment.
🛠️ Refactor suggestion
Avoid Using arguments Object for Format Detection
Using the arguments object for format detection in the toJSON method can lead to unreliable behavior, especially in strict mode. Consider passing the format explicitly or using a different detection mechanism.
Would you like assistance in refactoring this method to improve reliability?
| remove<TKey extends keyof T & string>(key: TKey): Promise<void> { | ||
| const stateMessage = this.createMapRemoveMessage(key); | ||
| return this._liveObjects.publish([stateMessage]); | ||
| } |
There was a problem hiding this comment.
Ensure Promise Rejection Handling in remove Method
Similar to the set method, the remove method should handle any rejections from the publish method to prevent unhandled promise rejections.
Apply this diff to handle possible rejections:
remove<TKey extends keyof T & string>(key: TKey): Promise<void> {
const stateMessage = this.createMapRemoveMessage(key);
- return this._liveObjects.publish([stateMessage]);
+ return this._liveObjects.publish([stateMessage]).catch((error) => {
+ // Handle or rethrow the error appropriately
+ return Promise.reject(error);
+ });
}Ensure that the calling context properly handles rejections.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| remove<TKey extends keyof T & string>(key: TKey): Promise<void> { | |
| const stateMessage = this.createMapRemoveMessage(key); | |
| return this._liveObjects.publish([stateMessage]); | |
| } | |
| remove<TKey extends keyof T & string>(key: TKey): Promise<void> { | |
| const stateMessage = this.createMapRemoveMessage(key); | |
| return this._liveObjects.publish([stateMessage]).catch((error) => { | |
| // Handle or rethrow the error appropriately | |
| return Promise.reject(error); | |
| }); | |
| } |
| /** | ||
| * Send a MAP_SET operation to the realtime system to set a key on this LiveMap object to a specified value. | ||
| * | ||
| * This does not modify the underlying data of this LiveMap object. Instead, the change will be applied when | ||
| * the published MAP_SET operation is echoed back to the client and applied to the object following the regular | ||
| * operation application procedure. | ||
| * | ||
| * @returns A promise which resolves upon receiving the ACK message for the published operation message. | ||
| */ | ||
| set<TKey extends keyof T & string>(key: TKey, value: T[TKey]): Promise<void> { | ||
| const stateMessage = this.createMapSetMessage(key, value); | ||
| return this._liveObjects.publish([stateMessage]); | ||
| } |
There was a problem hiding this comment.
Implement Asynchronous Error Handling in set Method
The set method returns a promise but may not properly handle rejections from the publish method. Ensure that any errors during publishing are caught and propagated.
Apply this diff to handle possible rejections:
set<TKey extends keyof T & string>(key: TKey, value: T[TKey]): Promise<void> {
const stateMessage = this.createMapSetMessage(key, value);
- return this._liveObjects.publish([stateMessage]);
+ return this._liveObjects.publish([stateMessage]).catch((error) => {
+ // Handle or rethrow the error appropriately
+ return Promise.reject(error);
+ });
}Alternatively, ensure that the calling context handles the promise rejections appropriately.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /** | |
| * Send a MAP_SET operation to the realtime system to set a key on this LiveMap object to a specified value. | |
| * | |
| * This does not modify the underlying data of this LiveMap object. Instead, the change will be applied when | |
| * the published MAP_SET operation is echoed back to the client and applied to the object following the regular | |
| * operation application procedure. | |
| * | |
| * @returns A promise which resolves upon receiving the ACK message for the published operation message. | |
| */ | |
| set<TKey extends keyof T & string>(key: TKey, value: T[TKey]): Promise<void> { | |
| const stateMessage = this.createMapSetMessage(key, value); | |
| return this._liveObjects.publish([stateMessage]); | |
| } | |
| /** | |
| * Send a MAP_SET operation to the realtime system to set a key on this LiveMap object to a specified value. | |
| * | |
| * This does not modify the underlying data of this LiveMap object. Instead, the change will be applied when | |
| * the published MAP_SET operation is echoed back to the client and applied to the object following the regular | |
| * operation application procedure. | |
| * | |
| * @returns A promise which resolves upon receiving the ACK message for the published operation message. | |
| */ | |
| set<TKey extends keyof T & string>(key: TKey, value: T[TKey]): Promise<void> { | |
| const stateMessage = this.createMapSetMessage(key, value); | |
| return this._liveObjects.publish([stateMessage]).catch((error) => { | |
| // Handle or rethrow the error appropriately | |
| return Promise.reject(error); | |
| }); | |
| } |
test/realtime/live_objects.test.js
Outdated
| expect(() => counter.increment(BigInt(1))).to.throw('Counter value increment should be a number'); | ||
| expect(() => counter.decrement(true)).to.throw('Counter value decrement should be a number'); | ||
| expect(() => counter.increment(Symbol())).to.throw('Counter value increment should be a number'); | ||
| expect(() => counter.decrement({})).to.throw('Counter value decrement should be a number'); |
There was a problem hiding this comment.
Fix inconsistent error message testing.
The test is checking for increment error messages while testing decrement operation.
-expect(() => counter.increment(BigInt(1))).to.throw('Counter value increment should be a number');
+expect(() => counter.decrement(BigInt(1))).to.throw('Counter value decrement should be a number');
-expect(() => counter.increment(Symbol())).to.throw('Counter value increment should be a number');
+expect(() => counter.decrement(Symbol())).to.throw('Counter value decrement should be a number');📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| expect(() => counter.increment(BigInt(1))).to.throw('Counter value increment should be a number'); | |
| expect(() => counter.decrement(true)).to.throw('Counter value decrement should be a number'); | |
| expect(() => counter.increment(Symbol())).to.throw('Counter value increment should be a number'); | |
| expect(() => counter.decrement({})).to.throw('Counter value decrement should be a number'); | |
| expect(() => counter.decrement(BigInt(1))).to.throw('Counter value decrement should be a number'); | |
| expect(() => counter.decrement(true)).to.throw('Counter value decrement should be a number'); | |
| expect(() => counter.decrement(Symbol())).to.throw('Counter value decrement should be a number'); | |
| expect(() => counter.decrement({})).to.throw('Counter value decrement should be a number'); |
VeskeR
force-pushed
the
liveobjects/object-mutation-api
branch
from
2412b3f to
6b66b7c
Compare
Adds common parts to support the write API for LiveObjects and implements the edit API part of the DTP-1033 (map set/remove, counter increment)
Resolves DTP-1033
Summary by CodeRabbit
Summary by CodeRabbit
New Features
sendStatemethod toRealtimeChannelfor sending state messages.incrementanddecrementmethods forLiveCounter.setandremovemethods forLiveMap.publishmethod inLiveObjectsfor state message publishing.Improvements
BaseClientwithMessageEncoding.Bug Fixes