Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DataClone Errors #873

Merged
merged 7 commits into from
Feb 13, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions integration-tests/execute/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# @openfn/integration-tests-execute

## 1.0.15

### Patch Changes

- Updated dependencies [beb4617]
- @openfn/[email protected]

## 1.0.14

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/execute/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@openfn/integration-tests-execute",
"private": true,
"version": "1.0.14",
"version": "1.0.15",
"description": "Job execution tests",
"author": "Open Function Group <[email protected]>",
"license": "ISC",
Expand Down
9 changes: 9 additions & 0 deletions integration-tests/worker/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# @openfn/integration-tests-worker

## 1.0.75

### Patch Changes

- Updated dependencies [beb4617]
- @openfn/[email protected]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.0.74

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion integration-tests/worker/package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@openfn/integration-tests-worker",
"private": true,
"version": "1.0.74",
"version": "1.0.75",
"description": "Lightning WOrker integration tests",
"author": "Open Function Group <[email protected]>",
"license": "ISC",
Expand Down
7 changes: 7 additions & 0 deletions packages/cli/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# @openfn/cli

## 1.11.1

### Patch Changes

- Updated dependencies [beb4617]
- @openfn/[email protected]

## 1.11.0

### Minor Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/cli/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/cli",
"version": "1.11.0",
"version": "1.11.1",
"description": "CLI devtools for the openfn toolchain.",
"engines": {
"node": ">=18",
Expand Down
7 changes: 7 additions & 0 deletions packages/engine-multi/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# engine-multi

## 1.4.9

### Patch Changes

- Updated dependencies [beb4617]
- @openfn/[email protected]

## 1.4.8

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/engine-multi/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/engine-multi",
"version": "1.4.8",
"version": "1.4.9",
"description": "Multi-process runtime engine",
"main": "dist/index.js",
"type": "module",
Expand Down
8 changes: 8 additions & 0 deletions packages/lightning-mock/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# @openfn/lightning-mock

## 2.0.30

### Patch Changes

- Updated dependencies [beb4617]
- @openfn/[email protected]
- @openfn/[email protected]

## 2.0.29

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/lightning-mock/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/lightning-mock",
"version": "2.0.29",
"version": "2.0.30",
"private": true,
"description": "A mock Lightning server",
"main": "dist/index.js",
Expand Down
6 changes: 6 additions & 0 deletions packages/runtime/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,11 @@
# @openfn/runtime

## 1.6.2

### Patch Changes

- beb4617: Ensure that AdaptorError details are safely serialised

## 1.6.1

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/runtime/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/runtime",
"version": "1.6.1",
"version": "1.6.2",
"description": "Job processing runtime.",
"type": "module",
"exports": {
Expand Down
6 changes: 4 additions & 2 deletions packages/runtime/src/errors.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import stringify from 'fast-safe-stringify';
import { ErrorPosition } from './types';

export function assertImportError(e: any) {
Expand Down Expand Up @@ -202,10 +203,11 @@ export class AdaptorError extends RTError {

this.details = Object.assign(
{
type: error.type || error.name,
message: error.message,
},
error
// The incoming error is untrusted but MUST be serialisable
// Or else we'll have problems in threaded environments
JSON.parse(stringify(error))
);

if (typeof error === 'string') {
Expand Down
9 changes: 7 additions & 2 deletions packages/runtime/src/execute/expression.ts
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,6 @@ export const wrapOperation = (
try {
result = await fn(newState);
} catch (e: any) {
console.log(e);
if (e.stack) {
const containsVMFrame = e.stack.match(/at vm:module\(0\)/);

Expand All @@ -151,9 +150,15 @@ export const wrapOperation = (
// (this cuts out low level language errors like TypeError)
do {
const next = frames.shift();
// If we hit a frame in runtime code, this is not an adaptor error
if (/@openfn\/runtime/.test(next)) {
break;
}
if (
// detect an adaptor prod, adaptor monorepo, or vm frame
/(@openfn\/language-)|(packages\/.+\/dist)|(vm:module)/.test(next)
/(@openfn\/language-)|(packages\/.+\/dist)|(vm:module)|(node_modules)/.test(
next
)
) {
firstFrame = next;
break;
Expand Down
16 changes: 14 additions & 2 deletions packages/runtime/test/errors.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,6 @@ test('fail on adaptor error and map to the top operation', async (t) => {
details: {
code: 1234,
message: 'adaptor err',
type: 'Error',
},
message: 'adaptor err',
name: 'AdaptorError',
Expand Down Expand Up @@ -460,7 +459,6 @@ test('fail on nested adaptor error and map to a position in the vm', async (t) =
details: {
code: 1234,
message: 'adaptor err',
type: 'Error',
},
message: 'adaptor err',
name: 'AdaptorError',
Expand Down Expand Up @@ -548,3 +546,17 @@ test('AdaptorError: if no operation, extract position from stack', (t) => {
t.deepEqual(adaptorError.pos, { column: 27, line: 2 });
t.falsy(adaptorError.operationName);
});

test('AdaptorError: ensure that error objects are safely serialised', (t) => {
const originalError = {
message: 'e',
name: 'AxiosError',
fn: () => {},
};

const adaptorError = new AdaptorError(originalError);
t.deepEqual(adaptorError.details, {
message: 'e',
name: 'AxiosError',
});
});
9 changes: 9 additions & 0 deletions packages/ws-worker/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,14 @@
# ws-worker

## 1.9.2

### Patch Changes

- beb4617: Fix an issue where a DataCloneError can occur after an exception is thrown
- Updated dependencies [beb4617]
- @openfn/[email protected]
- @openfn/[email protected]

## 1.9.1

### Patch Changes
Expand Down
2 changes: 1 addition & 1 deletion packages/ws-worker/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@openfn/ws-worker",
"version": "1.9.1",
"version": "1.9.2",
"description": "A Websocket Worker to connect Lightning to a Runtime Engine",
"main": "dist/index.js",
"type": "module",
Expand Down
Loading