Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update and optimize use of ws library in socks #1409

Merged
merged 19 commits into from
Jun 14, 2024
Merged

Update and optimize use of ws library in socks #1409

merged 19 commits into from
Jun 14, 2024

Conversation

BrendanChou
Copy link
Contributor

@BrendanChou BrendanChou commented Apr 25, 2024

Changelist

  • [chore]: Update ws library in socks service from 8.8.1 to 8.16.0.
  • [bugfix]: Do not explicitly handle ping as the ws library handles this by default. Explicitly set autoPong: true in server initialization to show this
  • [performance]: Flip value of allowSynchronousEvents to true which will be the default value in the next version of ws (commit here). This is expected to improve performance

Test Plan

  • Current behavior double-pongs:
$ wscat -c <indexer_endpoint> --slash -P
Connected (press CTRL+C to quit)
< {"type":"connected","connection_id":"...","message_id":0}
> /ping
< Received pong (data: "")
< Received pong (data: "")
  • Unit tests
  • Will test behavior in staging

Author/Reviewer Checklist

  • If this PR has changes that result in a different app state given the same prior state and transaction list, manually add the state-breaking label.
  • If the PR has breaking postgres changes to the indexer add the indexer-postgres-breaking label.
  • If this PR isn't state-breaking but has changes that modify behavior in PrepareProposal or ProcessProposal, manually add the label proposal-breaking.
  • If this PR is one of many that implement a specific feature, manually label them all feature:[feature-name].
  • If you wish to for mergify-bot to automatically create a PR to backport your change to a release branch, manually add the label backport/[branch-name].
  • Manually add any of the following labels: refactor, chore, bug.

Summary by CodeRabbit

  • Bug Fixes

    • Enhanced WebSocket server initialization to support synchronous events and automatic pong responses for improved stability.
    • Adjusted WebSocket test cases for consistent behavior and accurate test results.
  • Chores

    • Updated Docker configurations to ensure higher security during npm module installations by adding the --unsafe-perm flag.

@BrendanChou BrendanChou requested a review from jiajames April 25, 2024 15:39
Copy link
Contributor

coderabbitai bot commented Apr 25, 2024

Walkthrough

The updates mainly involve adding the --unsafe-perm flag to various pnpm install commands across multiple Dockerfiles in the indexer directory to enable elevated permissions during installation. Additionally, there are enhancements to the WebSocket server initialization in the Wss class for better performance and synchronous event handling. The associated test files were updated to accommodate these changes, removing obsolete spies and adjusting test parameters.

Changes

Files Summary
indexer/Dockerfile.auxo.remote, indexer/Dockerfile.bazooka.remote, indexer/Dockerfile.postgres-package.local, indexer/Dockerfile.service.local, indexer/Dockerfile.service.remote Added the --unsafe-perm flag to pnpm install commands to enable elevated permissions during installation.
indexer/services/socks/src/helpers/wss.ts Updated Wss constructor to initialize the WebSocket server with additional options like allowSynchronousEvents and autoPong.
indexer/services/socks/__tests__/websocket/index.test.ts Modified the WebSocket instantiation and removed wsPongSpy related tests.

Poem

🐇🌐
In Docker's realm, with scripts so tight,
We've granted them a special might,
With --unsafe-perm they rise, alight,
Permissions high, they now take flight.

WebSockets, with their pings so bright,
AutoPong sets the timing right,
Code refined, tests out of sight,
The indexer's future looks crisp and bright.


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@BrendanChou BrendanChou requested a review from jayy04 April 25, 2024 15:39
@BrendanChou BrendanChou requested a review from dydxwill April 25, 2024 15:39
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between efa1a70 and f907d7a.
Files ignored due to path filters (2)
  • indexer/package.json is excluded by !**/*.json
  • indexer/pnpm-lock.yaml is excluded by !**/*.yaml
Files selected for processing (2)
  • indexer/patches/@types+ws+8.5.10.patch (1 hunks)
  • indexer/services/socks/src/helpers/wss.ts (1 hunks)
Additional Context Used
Path-based Instructions (2)
indexer/patches/@types+ws+8.5.10.patch (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

indexer/services/socks/src/helpers/wss.ts (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

Additional comments not posted (2)
indexer/patches/@types+ws+8.5.10.patch (1)

16-19: The patch correctly introduces allowSynchronousEvents and autoPong properties to the WebSocket server options.

Verification successful

The new properties allowSynchronousEvents and autoPong have been successfully integrated into the serverOptions object in the wss.ts file.

  • indexer/services/socks/src/helpers/wss.ts:
    • allowSynchronousEvents: false
    • autoPong: true
Scripts Executed

The following scripts were executed for the analysis:


Script:

#!/bin/bash
# Description: Verify the integration of the new properties in the WebSocket server implementation.

# Test: Search for the usage of the new properties in the WebSocket server initialization.
rg --type typescript $'new WebSocket.Server'

Length of output: 80



Script:

#!/bin/bash
# Description: Verify the integration of the new properties in the WebSocket server implementation.

# Test: Search for the usage of the new properties in the WebSocket server initialization.
rg --type ts 'new WebSocket.Server'

Length of output: 131



Script:

#!/bin/bash
# Description: Verify the integration of the new properties in the WebSocket server implementation.

# Test: Search for the definition of the `serverOptions` object and its properties in the `wss.ts` file.
rg 'serverOptions' indexer/services/socks/src/helpers/wss.ts

Length of output: 165



Script:

#!/bin/bash
# Description: Verify the integration of the new properties in the WebSocket server implementation.

# Test: Display the contents of the `serverOptions` object in the `wss.ts` file.
rg -A 10 'const serverOptions: WebSocket.ServerOptions =' indexer/services/socks/src/helpers/wss.ts

Length of output: 415

indexer/services/socks/src/helpers/wss.ts (1)

21-25: The WebSocket server options have been updated to include autoPong: true. This aligns with the PR's objectives to simplify ping/pong handling.

port: config.WS_PORT,
});
allowSynchronousEvents: false,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider setting allowSynchronousEvents to true as per the PR's performance improvement plan.

-      allowSynchronousEvents: false,
+      allowSynchronousEvents: true,

Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
allowSynchronousEvents: false,
allowSynchronousEvents: true,

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between f907d7a and 8565bc1.
Files ignored due to path filters (1)
  • indexer/pnpm-lock.yaml is excluded by !**/*.yaml
Files selected for processing (5)
  • indexer/Dockerfile.auxo.remote (1 hunks)
  • indexer/Dockerfile.bazooka.remote (1 hunks)
  • indexer/Dockerfile.postgres-package.local (1 hunks)
  • indexer/Dockerfile.service.local (1 hunks)
  • indexer/Dockerfile.service.remote (1 hunks)
Files skipped from review due to trivial changes (1)
  • indexer/Dockerfile.postgres-package.local
Additional Context Used
Path-based Instructions (4)
indexer/Dockerfile.auxo.remote (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

indexer/Dockerfile.bazooka.remote (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

indexer/Dockerfile.service.local (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

indexer/Dockerfile.service.remote (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

Additional comments not posted (4)
indexer/Dockerfile.auxo.remote (1)

22-22: The use of --loglevel warn, --frozen-lockfile, and --unsafe-perm flags in the pnpm install command is appropriate for ensuring a stable and secure build environment.

indexer/Dockerfile.bazooka.remote (1)

25-25: Consistent use of pnpm install flags across different service Dockerfiles helps maintain a uniform build process.

indexer/Dockerfile.service.local (1)

45-45: Using --production flag in development environment installations is a good practice to mirror production settings, helping catch potential deployment issues early.

indexer/Dockerfile.service.remote (1)

45-45: Maintaining consistency in dependency installation commands between local and remote environments is crucial for ensuring that services behave similarly in both contexts.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between 8565bc1 and fe8e220.
Files ignored due to path filters (2)
  • indexer/pnpm-lock.yaml is excluded by !**/*.yaml
  • indexer/services/socks/package.json is excluded by !**/*.json
Files selected for processing (5)
  • indexer/Dockerfile.auxo.remote (2 hunks)
  • indexer/Dockerfile.bazooka.remote (2 hunks)
  • indexer/Dockerfile.postgres-package.local (2 hunks)
  • indexer/Dockerfile.service.local (2 hunks)
  • indexer/Dockerfile.service.remote (2 hunks)
Files skipped from review as they are similar to previous changes (5)
  • indexer/Dockerfile.auxo.remote
  • indexer/Dockerfile.bazooka.remote
  • indexer/Dockerfile.postgres-package.local
  • indexer/Dockerfile.service.local
  • indexer/Dockerfile.service.remote

@@ -22,7 +25,7 @@ COPY tsconfig.json ./
RUN npm install -g pnpm@6

# Install npm modules using pnpm
RUN pnpm install --loglevel warn --frozen-lockfile
RUN pnpm install --loglevel warn --frozen-lockfile --unsafe-perm
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

--unsafe-perm needed, otherwise npm skips the postinstall since it thinks its malicious

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review Details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits Files that changed from the base of the PR and between fe8e220 and ff9aa92.
Files selected for processing (6)
  • indexer/Dockerfile.auxo.remote (2 hunks)
  • indexer/Dockerfile.bazooka.remote (2 hunks)
  • indexer/Dockerfile.postgres-package.local (2 hunks)
  • indexer/Dockerfile.service.local (2 hunks)
  • indexer/Dockerfile.service.remote (2 hunks)
  • indexer/services/socks/src/helpers/wss.ts (1 hunks)
Files skipped from review as they are similar to previous changes (6)
  • indexer/Dockerfile.auxo.remote
  • indexer/Dockerfile.bazooka.remote
  • indexer/Dockerfile.postgres-package.local
  • indexer/Dockerfile.service.local
  • indexer/Dockerfile.service.remote
  • indexer/services/socks/src/helpers/wss.ts

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range and nitpick comments (3)
indexer/services/socks/__tests__/websocket/index.test.ts (3)

Line range hint 5-5: Import WebSocket with the node: protocol.

- import WebSocket from 'ws';
+ import WebSocket from 'node:ws';

This change is recommended to explicitly distinguish Node.js built-in modules from npm packages, improving clarity and potentially preventing naming conflicts.


Line range hint 6-6: Import other Node.js built-in modules with the node: protocol.

- import { IncomingMessage } from 'http';
+ import { IncomingMessage } from 'node:http';

This change follows best practices for importing built-in modules in Node.js, ensuring consistency and clarity in imports.


Line range hint 321-321: Specify a more precise type instead of any.

- function createIncomingMessage(properties: any): IncomingMessage {
+ function createIncomingMessage(properties: Record<string, unknown>): IncomingMessage {

Using Record<string, unknown> instead of any provides better type safety by enforcing that properties is an object with string keys, which aligns with TypeScript's best practices for type safety.

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between ff9aa92 and 87da7e3.

Files selected for processing (1)
  • indexer/services/socks/tests/websocket/index.test.ts (2 hunks)
Additional context used
Path-based instructions (1)
indexer/services/socks/__tests__/websocket/index.test.ts (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

Biome
indexer/services/socks/__tests__/websocket/index.test.ts

[error] 5-5: A Node.js builtin module should be imported with the node: protocol.


[error] 6-6: A Node.js builtin module should be imported with the node: protocol.


[error] 321-321: Unexpected any. Specify a different type.

Additional comments not posted (2)
indexer/services/socks/__tests__/websocket/index.test.ts (2)

52-52: Updated WebSocket constructor to include additional parameters.

This change aligns with the PR's objective to handle WebSocket connections more robustly by explicitly setting options, even if they are empty.


267-267: Adjusted expectation for wsPongSpy call count.

This change reflects the updated behavior due to the autoPong: true setting in the WebSocket server, which now handles pongs automatically, reducing the need for explicit pong handling in tests.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 87da7e3 and 01b203c.

Files selected for processing (1)
  • indexer/services/socks/tests/websocket/index.test.ts (4 hunks)
Additional context used
Path-based instructions (1)
indexer/services/socks/__tests__/websocket/index.test.ts (1)

Pattern **/**: Do not include a walkthrough. For all comments on line changes that are longer than 15 lines, add a prefix to the comment of what lines you are commenting on (for example: "The changes from lines 100 to 200...". Add your comment to be on the last 15 lines of the chunk rather than adding your comment to the entire chunk.

Biome
indexer/services/socks/__tests__/websocket/index.test.ts

[error] 5-5: A Node.js builtin module should be imported with the node: protocol.


[error] 6-6: A Node.js builtin module should be imported with the node: protocol.


[error] 51-51: Unexpected any. Specify a different type.


[error] 309-309: Unexpected any. Specify a different type.

Additional comments not posted (1)
indexer/services/socks/__tests__/websocket/index.test.ts (1)

75-75: Removal of the wsPongSpy call count expectation is appropriate given the automatic pong handling by the updated ws library.

@@ -49,10 +48,9 @@ describe('Index', () => {
(Subscriptions as unknown as jest.Mock).mockClear();
(sendMessage as unknown as jest.Mock).mockClear();
mockWss = new Wss();
websocket = new WebSocket(null);
websocket = new WebSocket(null, [], { autoPong: true } as any);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update to WebSocket constructor to include autoPong: true aligns with PR objectives. However, avoid using as any for type safety.

- websocket = new WebSocket(null, [], { autoPong: true } as any);
+ // Define a proper type for the options or extend existing types to include autoPong
+ websocket = new WebSocket(null, [], { autoPong: true } as WebSocketOptions);

Committable suggestion was skipped due low confidence.

Tools
Biome

[error] 51-51: Unexpected any. Specify a different type.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 01b203c and 7bf3954.

Files ignored due to path filters (1)
  • indexer/pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
Files selected for processing (1)
  • indexer/services/socks/tests/websocket/index.test.ts (4 hunks)
Files skipped from review due to trivial changes (1)
  • indexer/services/socks/tests/websocket/index.test.ts

@dydxwill dydxwill merged commit 5141daf into main Jun 14, 2024
18 checks passed
@dydxwill dydxwill deleted the bc/socks-ws branch June 14, 2024 17:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

4 participants