Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add consistency for reporting TestNG results for data providers #787

Open
jlg7 opened this issue Aug 23, 2019 · 1 comment
Open

Add consistency for reporting TestNG results for data providers #787

jlg7 opened this issue Aug 23, 2019 · 1 comment
Labels

Comments

@jlg7
Copy link

jlg7 commented Aug 23, 2019

Summary

Overall, great experience working with vscode and this extension thus far. Adding this feature/fixing the bug (eye of the beholder) will save a bit of time running commands in the terminal to get reliable results.

See below for a depiction of the disagreement between the user interface and console reports as well as a simple reproduce case.

I'd imagine there are many ways to approach this for full flexibility but I do think having the default test listener support this out of the box and at least have agreement in counts would be a desirable feature. I have used other IDEs and here is one way it is being approached: https://github.com/JetBrains/intellij-community/blob/2aa816fb814bb07c697a17d6a27e427e142a3633/plugins/testng_rt/src/com/intellij/rt/testng/IDEATestNGRemoteListener.java.

Note: there are ways to work around this by implementing the ITest interface per test and overriding the getTestName method but this is cumbersome as well!

System and application versions

Version: 1.37.1 (user setup)
Commit: f06011ac164ae4dc8e753a3fe7f9549844d15e35
Date: 2019-08-15T16:17:55.855Z
Electron: 4.2.7
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Windows_NT x64 10.0.18362

TestNG version

  <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8</version>
            <scope>test</scope>
  </dependency>

Repro case

image

@jdneo jdneo added the bug label Aug 24, 2019
@itaygoz
Copy link

itaygoz commented Apr 11, 2021

Hi, I have the same issue.
Seems like the testng is not configured correctly, also it doesn't make a report on test-output as usual.
I've tried to look at testng docs and seems like it needs to be configured, but I've failed to configure it.

Version: 1.55.0 (user setup)
Commit: c185983a683d14c396952dd432459097bc7f757f
Date: 2021-03-30T16:01:55.261Z
Electron: 11.3.0
Chrome: 87.0.4280.141
Node.js: 12.18.3
V8: 8.7.220.31-electron.0
OS: Windows_NT x64 10.0.18363

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants