Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ERROR]: TypeError [ERR_INVALID_ARG_TYPE] when running playwright service with node v20.10.0 and set qase=true #106

Closed
RF2023 opened this issue May 1, 2024 · 33 comments
Assignees

Comments

@RF2023
Copy link

RF2023 commented May 1, 2024

Describe the bug
Enabling qase on our build.yml file and running on MS playwright service is throwing an error.

To Reproduce
Steps to reproduce the behavior:

  1. Step 1 'Running a test suite with a specific tag on gha'
  2. Step 2 'send_to_qase: true in build.yml file'
  3. Step 3 'suite doesnt run and throws the following error'
  4. See error

Expected behavior
Test results should be sent to qase

Screenshots
If applicable, add screenshots to help explain your problem.

Setup information (please complete the following information):
Azure Subscription Id:798f112d-0c47-431d-9b57-eeae586eb023
Workspace Name: Playwright
Service trace Id (if available):
Log snippet (if available):
Browser on which tests are executed (if applicable): chromium
Time range (in PST) when the issue was experienced: Multiple times
Playwright Version: 1.42.1
Client machine OS: MAC
OS requested on Service: linux

Additional context
We have verified this after upgrading our node version as well, based on another ticket but the error still persists. Due to this we are unable to enable qase for MS Playwright service.

ErrorCaptureStackTrace(err); ^ TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined at new ReadStream (node:internal/fs/streams:174:5) at Object.createReadStream (node:fs:2916:10) at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:388:30) at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23) at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71 at new Promise (<anonymous>) at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:387:57 at Array.map (<anonymous>) at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:387:36) at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23) at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71 at new Promise (<anonymous>) at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12) at PlaywrightReporter.uploadAttachments (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:443:16) at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:231:43) at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23) at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53) at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71 at new Promise (<anonymous>) at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12) at PlaywrightReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:245:16) at ReporterV2Wrapper.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/reporterV2.js:87:158) at /runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:65 at wrap (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:86:5) at Multiplexer.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:45) at InternalReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/internalReporter.js:62:20) at JobDispatcher._reportTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:516:20) at JobDispatcher._onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:288:10) at WorkerHost.emit (node:events:513:28) at ChildProcess.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/processHost.js:105:16) at ChildProcess.emit (node:events:513:28) at emit (node:internal/child_process:946:14) at processTicksAndRejections (node:internal/process/task_queues:84:21) { code: 'ERR_INVALID_ARG_TYPE' }

@RF2023 RF2023 changed the title [ERROR]: TypeError [ERR_INVALID_ARG_TYPE] when running playwright with node 20.10.0 and set qase=ture [ERROR]: TypeError [ERR_INVALID_ARG_TYPE] when running playwright with node 20.10.0 and set qase=true May 1, 2024
@RF2023 RF2023 changed the title [ERROR]: TypeError [ERR_INVALID_ARG_TYPE] when running playwright with node 20.10.0 and set qase=true [ERROR]: TypeError [ERR_INVALID_ARG_TYPE] when running playwright service with node v20.10.0 and set qase=true May 1, 2024
@puagarwa
Copy link
Member

puagarwa commented May 1, 2024

Hi @RF2023
The stacktrace suggest error is coming from qase module and it doesnt seems like coming from playwright service. I would suggest you to check with QASE team.
If you still think this is related to MS playwright service, let me know why.

@RF2023
Copy link
Author

RF2023 commented May 7, 2024

I opened a ticket with QASE, i will keep you posted.

@puagarwa
Copy link
Member

Closing the issue for now, feel free to reopen it if you still think its related to playwright service.

@RF2023
Copy link
Author

RF2023 commented May 16, 2024

So we are clearly seeing an issue when we use playwright.config.ts vs playwright.service.config.ts

latest error, it starts of hitting the qase API we get a notification saying qase run is created but , can you still confirm if this is not relating to the service and is a QASE issue? I have a open ticket with them as well.

`qase: Current PID: 2695

Running 32 tests using 10 workers

qase: Project ***** exists
qase: Using run 142 to publish test results`

`node:internal/fs/utils:699
    throw new ERR_INVALID_ARG_TYPE(propName, ['string', 'Buffer', 'URL'], path);
          ^

TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined
    at new ReadStream (node:internal/fs/streams:185:5)
    at Object.createReadStream (node:fs:3076:10)
    at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:390:30)
    at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23)
    at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53)
    at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71
    at new Promise (<anonymous>)
    at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12)
    at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:389:57
    at Array.map (<anonymous>)
    at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:389:36)
    at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23)
    at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53)
    at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71
    at new Promise (<anonymous>)
    at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12)
    at PlaywrightReporter.uploadAttachments (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:445:16)
    at PlaywrightReporter.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/src/index.ts:233:43)
    at step (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:64:23)
    at Object.next (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:45:53)
    at /runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:39:71
    at new Promise (<anonymous>)
    at __awaiter (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:35:12)
    at PlaywrightReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright-qase-reporter/dist/index.js:247:16)
    at ReporterV2Wrapper.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/reporterV2.js:87:150)
    at /runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:65
    at wrap (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:86:5)
    at Multiplexer.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/multiplexer.js:47:45)
    at InternalReporter.onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/reporters/internalReporter.js:62:20)
    at JobDispatcher._reportTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:518:20)
    at JobDispatcher._onTestEnd (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/dispatcher.js:284:10)
    at WorkerHost.emit (node:events:514:28)
    at ChildProcess.<anonymous> (/runner/_work/lively-automation/lively-automation/node_modules/playwright/lib/runner/processHost.js:105:16)
    at ChildProcess.emit (node:events:514:28)
    at emit (node:internal/child_process:951:14)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  code: 'ERR_INVALID_ARG_TYPE'
}

Node.js v20.10.0
Playwright tests failed with exit code 1`

@puagarwa
Copy link
Member

HI @RF2023 , Even though the above log clearly pointing at issue in Qase package but lets do below to further understanding if MPT service has any role here.

  • playwright.service.config inherit playwright.config and override few fields where one is them is essential for service integration which is connectOption, others are optional like ignoreSnapshots, snapshotPathTemplate, please comment these two and try once, if works then all good.
  • We could also try to disable service via this change
    • if issue persist then service.config has issue and we can work it out but service itself doesnt have any issue.
    • if issue goes away, then we need to deep dive to understand qase dependency.

Basically we are trying to find of we have issue in service.config or service+Qase use case.

@puagarwa puagarwa reopened this May 17, 2024
@puagarwa puagarwa self-assigned this May 17, 2024
@RF2023
Copy link
Author

RF2023 commented May 17, 2024 via email

@RF2023
Copy link
Author

RF2023 commented May 17, 2024 via email

@puagarwa
Copy link
Member

HI @RF2023, can you please try my other suggestion as well
We could also try to disable service via this change

  • if issue persist then service.config has issue and we can work it out but service itself doesnt have any issue.
  • if issue goes away, then we need to deep dive to understand qase dependency.

@RF2023
Copy link
Author

RF2023 commented May 17, 2024 via email

@Sid200026
Copy link
Member

Hi @RF2023

So when you say disable service, the tests are not running on service
unless we specify the service file. Did i understand that correctly?
Do you want me to still make the changes suggested and use the service
config file?

Since running with no service using playwright.config.ts file is already working, we want you to run with service disabled but using the playwright.service.config.ts file. This can be achieved by not passing the connectOptions parameter - Reference PR - env variable to toggle MPT service #108

@RF2023
Copy link
Author

RF2023 commented May 20, 2024 via email

@Sid200026
Copy link
Member

Hi @RF2023,

Could you share the playwright-qase-reporter version that is being used in your project?

I integrated qase to the sample playwright project as per qase playwright integration example and was able to publish tests results to qase while using service config with MPT enabled.

I encountered a similar error when used v1.2.2 and had to upgrade to v2.0.0. There's a fix from the qase team regarding your issue - TypeError: The "path" argument must be of type string or an instance of Buffer or URL. Received undefined at new ReadStream which checks if the path is undefined before creating the read stream.

// v1.2.2
const data = createReadStream(attachment?.path as string);
// v2.0.0
if (attachment.file_path) {
    data = { name: attachment.file_name, value: createReadStream(attachment.file_path) };
}

@RF2023
Copy link
Author

RF2023 commented May 22, 2024 via email

@Sid200026
Copy link
Member

does this need to be added??

No, it's a part of the playwright-qase-reporter package. I was just highlighting the line of code (in the playwright-qase-reporter package) which is causing the error you faced.

The only way forward I see is you upgrading playwright-qase-reporter to v2.0.0 which contains the fix.

upgrade was breaking our changes the other day before i reopened this ticket.

When I was upgrading the qase reporter package from v1.2.2 to v2.0.0 locally, I faced upgrade issues as well. I resolved it by following the new Reporter options provided by qase as per - playwright config for Playwright - Qase Integration

previous change looks like this

You can revert the debug changes you did earlier as part of Reference PR - env variable to toggle MPT service #108
since they are no longer required.

@RF2023
Copy link
Author

RF2023 commented May 22, 2024 via email

@RF2023
Copy link
Author

RF2023 commented May 22, 2024 via email

@RF2023
Copy link
Author

RF2023 commented May 22, 2024

[email protected] play
playwright test --project=chromium -g @testtag -c playwright.config.ts
qase: Current PID: ****
Running 32 tests using 4 workers
qase: Project automation exists
qase: Using run 154 to publish test results

works fine when i use the normal file, can you please share the service file that worked for you along with changes made as part of new qase reporter upgrade?

also tried it with this change to service file, doesnt error out but doesnt publish as well

Also on the service file , following is commented out as part of previous suggestion.
// Enable screenshot testing and configure directory with expectations.
// https://learn.microsoft.com/azure/playwright-testing/how-to-configure-visual-comparisons
//ignoreSnapshots: false,
//snapshotPathTemplate: {testDir}/__screenshots__/{testFilePath}/${os}/{arg}{ext},

@Sid200026
Copy link
Member

works fine when i use the normal file, can you please share the service file that worked for you along with changes made as part of new qase reporter upgrade?

I used the sample MPT config and just added the qase reporter as per below.

reporter: [
    ['list'],
    [
      'playwright-qase-reporter',
      {
        debug: true,

        testops: {
          api: {
            token: '<>',
          },

          project: 'DEMO',
          uploadAttachments: true,

          run: {
            complete: true,
          },
        },
      },
    ],
  ],

I also had to set an environment variable QASE_MODE = testops as per test script in package.json

@RF2023
Copy link
Author

RF2023 commented Jun 4, 2024 via email

@puagarwa
Copy link
Member

puagarwa commented Jun 5, 2024

HI @RF2023
We dont have any knowledge about qase reporter, but we could try to bifurcate the issue with playwright service so that you can easily work with qase team on the issue.

Here is what i would suggest, you can make the qase work in your non-service config (playwright.config.ts) and work with qase team to resolve if any issue. Once it works fine then you can share the config with us and we can help you with changes to integrate service, and if you find any issue then we would work with you on fix. if require we can schedule a call for faster resolution, please send mail to [email protected].

Ideally, our service changes shouldn't interfere with reporter working because both are different worker thread which works independently. so most likely if you are facing issue with qase then issue is either in qase integration or in their package.

@vvs11
Copy link
Contributor

vvs11 commented Jun 5, 2024

Hi @RF2023

We have recently introduced the reporting feature for Microsoft Playwright Testing service. With this feature, you can publish your test results and artifacts to the service and view them in the MPT portal. You'll have all the information needed to diagnose failed tests. The portal also hosts all collected artifacts, including the trace viewer. Integrating this feature with your existing test suite is straightforward and easy.

You can read more about it here: https://aka.ms/mpt/reporting-announce

Currently, the feature is invite-only preview. You can sign up using this link: https://aka.ms/mpt/reporting-signup

Thanks

@RF2023
Copy link
Author

RF2023 commented Jun 5, 2024 via email

@RF2023
Copy link
Author

RF2023 commented Jun 5, 2024 via email

@puagarwa
Copy link
Member

puagarwa commented Jun 5, 2024

Puneet, thank you for the response. This is what i will be doing going forward 1. Downgrade playwright-qase-reporter back to 1.2.2 from 2.0.0 2. Run tests with config file and make sure its working as expected. 3. Run it with service file as well (wont work) Let me know what exact information you need from me once i have it back working on the config file. Thank you for your time !

On Wed, Jun 5, 2024 at 12:49 AM Puneet Agarwal @.> wrote: HI @RF2023 https://github.com/RF2023 We dont have any knowledge about qase reporter, but we could try to bifurcate the issue with playwright service so that you can easily work with qase team on the issue. Here is what i would suggest, you can make the qase work in your non-service config (playwright.config.ts) and work with qase team to resolve if any issue. Once it works fine then you can share the config with us and we can help you with changes to integrate service, and if you find any issue then we would work with you on fix. if require we can schedule a call for faster resolution, please send mail to @. Ideally, our service changes shouldn't interfere with reporter working because both are different worker thread which works independently. so most likely if you are facing issue with qase then issue is either in qase integration or in their package. — Reply to this email directly, view it on GitHub <#106 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZ6GIWA5N5NFVRDR5UY5LG3ZF2KFZAVCNFSM6AAAAABHCJATDWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCNBYHA2DQNBTGA . You are receiving this because you were mentioned.Message ID: @.***>
-- Regards, Shravan Kumar Eetha Sr QA Engineer Lively, Inc. | www.livelyme.com https://livelyme.com/

Hi @RF2023, I actually think you can make qase latest version work without our service and then share your config file with us and we will integrate service. As my team mate earlier mentioned there was known issue in thier old package #106 (comment)

@RF2023
Copy link
Author

RF2023 commented Jun 5, 2024

Problem is the latest version of playwright qase reporter is not showing the QASE ID in the report both with the service and normal config file. Thats why i was planning to downgrade to 1.2.2. I can still share what we have currently, what is the exact piece you guys are looking for , so that i can share the masked code snippet?

@puagarwa
Copy link
Member

puagarwa commented Jun 5, 2024

i see, i would advise to work with them to resolve issue with latest version.

But if you still wants to work with old qase version, then we could try integrating service in your default config file via this kind of change #110 ,

  • first get your suite working with non service config
  • then modify non service config like this PR
  • Run it and report the result

We can try to find some time for faster resolution over call , please send mail at [email protected]

@RF2023
Copy link
Author

RF2023 commented Jun 5, 2024

Currently in the config file, this is what we have which is working for both after upgrading playwright qase reporter to 2.0
This is the new suggested format which is working with both files, but at the same time QASE ID at the end is missing for service and normal file. QASE ID is displayed for both

if (process.env.QASE_REPORT) { reporters.push([ 'playwright-qase-reporter', { basePath: 'https://api.qase.io/v1', runId: process.env.QASE_TESTOPS_RUN_ID, testops: { api: { token: process.env.QASE_API_TOKEN, }, project: process.env.QASE_TESTOPS_PROJECT, uploadAttachments: true, run: { complete: true, }, }, }, ]); }

Old Format: Works with both files, but send_to_qase = true is selected keeps failing on service and normal file.
if (process.env.QASE_REPORT) { reporters.push([ 'playwright-qase-reporter', { basePath: 'https://api.qase.io/v1', apiToken: process.env.QASE_API_TOKEN, projectCode: process.env.QASE_PROJECT_CODE, runId: process.env.QASE_RUN_ID, runComplete: false, logging: true, uploadAttachments: true, }, ]); }

@RF2023
Copy link
Author

RF2023 commented Jun 5, 2024

i see, i would advise to work with them to resolve issue with latest version.

But if you still wants to work with old qase version, then we could try integrating service in your default config file via this kind of change #110 ,

  • first get your suite working with non service config
  • then modify non service config like this PR
  • Run it and report the result

We can try to find some time for faster resolution over call , please send mail at [email protected]

i will send you an invite shortly.

@RF2023
Copy link
Author

RF2023 commented Jun 6, 2024 via email

@puagarwa
Copy link
Member

puagarwa commented Jun 6, 2024

@RF2023 Great to hear things worked this way. You could still switch between service and non service mode, if you could add condition env variable like done in this PR https://github.com/microsoft/playwright-testing-service/pull/108/files , Let me know if any doubt here.

@RF2023
Copy link
Author

RF2023 commented Jun 11, 2024

So we are good with QASE ID being displayed in the report, but

  1. when we add send_to_qase=true on the old version both normal config and service the run fails.
  2. Upgraded reporter version with all changes suggested above will actually remove the QASE ID in the report.

@puagarwa
Copy link
Member

HI @RF2023, Both these questions are related to qase and we dont have any knowledge about the qase hence it would be better if you ask them with qase team .
I would be happy to help you with any issues/questions in MPT service.

@puagarwa
Copy link
Member

closing this issue as its qase issue. feel free to reopen if needed help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants