ci: LSDV-4734: Setup Cypress integration tests (#1236)

* Introducing Cypress * Remove outputs * Remove old outputs * Add logs * fix: LSDV-4676: Region corruption MIG (#1230) * Fix regions corruption by utilizing existing result * Cover data export with tests * Only define _rawResult for newly created areas * Fix tests + Fix segmentation lookup * Update src/stores/Annotation/Annotation.js Co-authored-by: bmartel <[email protected]> * Fix lint issue * Remove excess trace * Wrap `_rawValue` with feature flag * Fix incorrect export data + Assertion --------- Co-authored-by: bmartel <[email protected]> * docs: LSDV-4707: Remove Repeater from docs (#1235) * fix: LSDV-4715: Start, end and offset of paragraph selection should not include empty or newline (#1233) * fix: LSDV-4715: Start and offset of paragraph selection should not include empty or newline * fix empty lines * fix empty lines when selecting across collapsed text * adding tests for paragraph selection fixes * Update Paragraphs.js * adding tests for paragraph selection end offsets * Add test case for multi-line selection with wrong offsets * Update Paragraphs.js * linting * remove console logs * changing the implementation to base off indices for clarity * putting back the comment, mistakenly removed * Update Paragraphs.js * Fix offset when we move one phrase back * Fix test to have phrases with different length --------- Co-authored-by: Sergey <[email protected]> Co-authored-by: hlomzik <[email protected]> * fix: LSDV-4649: Regrouping regions after switching annotations (#1223) * fix: LSDV-4649: Regrouping regions after switching annotations * test: Add plugin to monitoring errors from the browser in e2e * test: LSDV-4649: Add test for checking drag and drop at regions tree in outliner * Fix creating paragraph regions according to failed tests * Add plugin for modifying e2e step logs to filter unnecessary information * Fix tests * Fix function naming in logs * Clear code * Add comment for test fix --------- Co-authored-by: bmartel <[email protected]> * fix: LSDV-4662: Use empty range GET requests instead of HEAD to avoid CORS issue on video presigned urls (#1225) fix: LSDV-4662: Use empty range GET requests instead of HEAD to avoid CORS issue * fix: LSDV-4593: CORS error requesting audio (#1234) * fix: LSDV-4593: CORS error requesting audio * use a less likely query param, ensure all relative urls are handled so 404 errors are thrown correctly * feat: LSDV-3023: Annotation Tab Carousel (#1188) * feat: dev-3879: Control bar * lint clean up * visual tweaks * control bar visual improvements * cleanup * visual tweak for view all button * visual tweak for auto annotation * feat: DEV-3879: Control bar (#1075) * applying changes based on feedback * adding reset button * feat: LSDV-3023: Annotation Tab Carousel * prepping button for usage * tweaking the icons * more icon tweaks * no using mobx observer * now using mobx observer * adding some icons + clean up * annotation state icons added and working on carousel * carousel working 🎉 * we can now update button capability when window is resized * resolving an error picked up in e2e * minor tweak for icon when dropdown is open * have a working test for AnnotationCarousel and AnnotationButton * making sure bottombar height is correct when FF is off * check for pk value before ground truth button display * cannot duplicate drafts * sort buttons by pk * viewall vertical alignment * side-pannels do not pad when hidden in view all * tab bar recalculates carosel on add/remove entity * update use effect with entities.length * space after set and unset * check history for that action type and entity for can undo states * create new button added to top bar * remove avatar icons * removing unused icon imports --------- Co-authored-by: yyassi-heartex <[email protected]> Co-authored-by: Travis1282 <[email protected]> Co-authored-by: bmartel <[email protected]> * fix: LSDV-4752: Support relations in MIG scenario (#1240) * Support relations in MIG scenario * Remove excess import * Fix relations checks * Render relation if above checks were skipped * Fix formatting * Remove excess checks * Fix check * ci: Number of tests is increasing, need more time (#1256) * fix: LSDV-4740: Video Rectangles are displaying while drawing (#1258) * docs: LSDV-4754: Update Audio documentation to point to v3 (#1242) * docs: LSDV-4574: Update Audio documentation to point to v3 * this apparently shouldn't be committed * Update Paragraphs.js * Add some comment about return type JSDoc can't parse tuple type, so let it be just `Array` with comment * Escape tags in Labels and Choices docs * "publicly" spelling --------- Co-authored-by: hlomzik <[email protected]> * ci: update autolabeler.yml * fix: LSDV-4801: Better file type playable detection for video (#1259) * fix: LSDV-4801: Better file type playable detection for video * adding unit test for VirtualVideo filetype detection * fixing cache invalidation of urls, to not operate on signed urls as it will invalidate the signature * fix preview loading * fix audio resolving in tests * fix: LSDV-4747: Selecting the end character on a Paragraph phrase to the very start of other phrases only includes the first phrase selection (#1257) * adding another test case to paragraphs for selection issues * Look for correct text node when selection ends on div But range stored in region should also be fixed and not have 0 as the endOffset * changing the test to select a more easily reproducible case manually * fix highlighting of spans * adding test for loading paragraph ranges correctly, still need to get this passing but now closer to a solution * making test for initializing a paragraph region pass * for some reason using Dialogue as the default dev config causes timeout errors * fix smoke test for audio paragraphs * making the result text function simplified to use existing phraseElements * removing unused function * adding another test case, and making a slight adjustment to fix * updating codept timeout as more tests are pushing the threshold --------- Co-authored-by: hlomzik <[email protected]> * fix: LSDV-4811: Impossible to select and drag a hidden region (#1270) * fix: LSDV-4708: Scroll page to canvas only if needed (#1268) * fix: LSDV-4708: Scroll page to canvas only if needed If canvas is already in the viewport, don't scroll the page! * Scroll only if image region is not visible scrollIntoViewIfNeeded scolls even if it's not really needed :( * Add comments; fix zoomed image; add smooth - hard to calculate region position on zoomed image and it can be hidden, so don't scroll if image is zoomed in - add behavior: smooth for better UX * Improve scroll to region * Huge improvement for zoomed and huge images Zoomed in images makes it hard to calculate region bboxes, so we are trying to make the whole image visible enough. * Rename method for better readability --------- Co-authored-by: yyassi-heartex <[email protected]> * ci: SRE-471: Publish tags from LSF repo to LS (#1276) * fix: LSDV-4692: Brush segmentation is not supported (#1247) * Fix tools in MIG * Remove excess FF * Make RLE Export stageless * Remove feature flag * Fix comment * Remove unused helper * Remove unused code * fix: LSDV-4774: Drawing on the hidden image in MIG scenario (#1254) Fixing Brush and Polygon can draw on hiddent images * Fix inability to edit brush regions * Remove RLE cache * Fix helper * Rename test suite * Remove ESLint debug info * Fix audio assertions * Fix audio error matching --------- Co-authored-by: Sergey <[email protected]> * Separating cypress into own repo * Cypress detached * Remove output from git * Enable Cypress in CI * Rename workflow * Enable parallel running * Parallel running * Change ls-test source * Setup SSH to install private packages * Update workflow * Run SSH agent before setting up * Manual ssh setup * Update secret name * Properly pass secrets to the workflow * Use SSH action * Properly init SSH * Fix variables * Default feature flags for tests * Set default feature falgs before every test * Fix feature flags * Allow setting feature flags after navigation * Allow setting feature flags on page load * Canvas comparison test * Fix tests * Rename test suite * Move test, enable test coverage collection * Enable code coverage upload * Fix feature falgs * Run tests in chrome * Rename the suite, update coverage toggle * Change Cypress run CI * Change coverage dir * Install specific version of the test plugin * Wrong package * Change coverage directory * Enable testing environment * Use different version of coverage uploader * Change coverage uploader name * Run Cypress in parallel * Update coverage settings * Fix coverage preparation command * Playing with coverage * Fix coverage command * Just a commit * Compare images * Enable source checkout for coverage upload * Move cypress tests to tests/functional * Improve feature flag testing * Remove unnecessary mapping * Remove unnecessary feature flags stuff * Rename external feature flags, update ls-test * Rename init preventer variable * Switch CI/CD to master --------- Co-authored-by: bmartel <[email protected]> Co-authored-by: hlomzik <[email protected]> Co-authored-by: Sergey <[email protected]> Co-authored-by: yyassi-heartex <[email protected]> Co-authored-by: yyassi-heartex <[email protected]> Co-authored-by: Travis1282 <[email protected]> Co-authored-by: Julio Sgarbi <[email protected]> Co-authored-by: Sergey Zhuk <[email protected]> Co-authored-by: Nikita Belonogov <[email protected]>
HumanSignal · Apr 14, 2023 · a087162 · a087162
1 parent 965ad3e
commit a087162
Show file tree

Hide file tree

Showing 24 changed files with 4,587 additions and 36 deletions.
diff --git a/.eslintrc.js b/.eslintrc.js
@@ -3,6 +3,7 @@ module.exports = {
     process: true,
     module: true,
     require: true,
+    DISABLE_DEFAULT_LSF_INIT: true,
     __dirname: true,
   },
   extends: ['plugin:@heartexlabs/frontend/recommended'],

diff --git a/.github/workflows/cicd_pipeline.yml b/.github/workflows/cicd_pipeline.yml
@@ -46,6 +46,16 @@ jobs:
     with:
       sha: ${{ github.event.pull_request.head.sha || github.event.after }}
 
+  run_functional:
+    name: "Tests"
+    if: github.event_name == 'push' || github.event.pull_request.draft == false
+    uses: heartexlabs/label-studio-frontend/.github/workflows/fun_tests.yml@master
+    needs:
+      - build_for_coverage
+    with:
+      sha: ${{ github.event.pull_request.head.sha || github.event.after }}
+    secrets: inherit
+
   run_unit:
     name: "Tests"
     if: github.event_name == 'push' || github.event.pull_request.draft == false

diff --git a/.github/workflows/e2e_tests.yml b/.github/workflows/e2e_tests.yml
@@ -115,15 +115,3 @@ jobs:
           name: e2e-tests-coverage
           path: coverage/
 
-      # - name: Upload coverage to Codecov
-      #   uses: codecov/[email protected]
-      #   if: ${{ !github.event.pull_request.head.repo.fork }}
-      #   with:
-      #     fail_ci_if_error: true
-      #     token: ${{ secrets.CODECOV_TOKEN }}
-
-      #- name: Upload coverage to artifact
-        #uses: actions/upload-artifact@v3
-        #with:
-          #name: e2e-tests-coverage
-          #path: coverage/
diff --git a/.github/workflows/fun_tests.yml b/.github/workflows/fun_tests.yml
@@ -0,0 +1,133 @@
+name: "Run Cypress"
+
+on:
+  workflow_call:
+    inputs:
+      sha:
+        required: true
+        type: string
+
+env:
+  # increment it in case if you need to reset cache
+  CACHE_NAME_PREFIX: v3
+  NODE: '14'
+
+jobs:
+  run:
+    name: "Cypress Tests"
+    runs-on: ubuntu-latest
+
+    # ci can be skipped with `[skip ci]` prefix in message
+    if: "!contains(github.event.head_commit.message, 'skip ci')"
+    steps:
+      - uses: hmarr/[email protected]
+
+      - name: "Checkout codebase"
+        uses: actions/checkout@v3
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+          ref: ${{ inputs.sha }}
+
+      - uses: actions/setup-node@v3
+        with:
+          node-version: "${{ env.NODE }}"
+
+      - name: Get CPU info
+        id: "cpu-info"
+        run: echo "cores-count=$(cat /proc/cpuinfo  | grep processor | wc -l)" >> $GITHUB_OUTPUT
+
+      - name: Setup SSH agent
+        env:
+          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
+          SSH_DIR: /home/runner/.ssh
+        run: |
+          mkdir $SSH_DIR
+          ssh-keyscan github.com >> $SSH_DIR/known_hosts
+          echo "${{ secrets.SSH_PRIVATE_KEY }}" > $SSH_DIR/package_rsa
+          chmod 600 $SSH_DIR/package_rsa
+          ssh-agent -a $SSH_AUTH_SOCK > /dev/null
+          ssh-add $SSH_DIR/package_rsa
+
+      - name: Upgrade Yarn
+        run: npm install -g [email protected]
+
+      - name: Get yarn cache directory path
+        id: yarn-cache-dir-path
+        run: echo "dir=$(yarn cache dir)" >> $GITHUB_OUTPUT
+
+      - name: Configure yarn cache
+        uses: actions/cache@v3
+        with:
+          path: ${{ steps.yarn-cache-dir-path.outputs.dir }}
+          key: ${{ env.CACHE_NAME_PREFIX }}-${{ runner.os }}-node-${{ env.NODE }}-${{ hashFiles('**/yarn.lock') }}
+          restore-keys: |
+            yarn-${{ env.CACHE_NAME_PREFIX }}-${{ runner.os }}-node-${{ env.NODE }}-
+
+      - name: Print Yarn cache size
+        run: du -d 0 -h ${{ steps.yarn-cache-dir-path.outputs.dir }}
+
+      - name: "Download bundle"
+        uses: actions/download-artifact@v3
+        with:
+          name: LSF-coverage-${{ inputs.sha }}
+          path: build/
+
+      # run http-server with build in background (will be killed after job ends)
+      # do this only for master branch (so only for push event)
+      # because pr can contain unfinished job
+      - name: "Run server"
+        run: |
+          cp -r public/files build/ &&
+          cp -r public/images build/ &&
+          cp -r public/styles build/ &&
+          npx serve -l tcp://localhost:3000 build &
+
+      - id: wait_for_npx_server
+        name: "Wait for server"
+        timeout-minutes: 1
+        run: |
+          while [ "$(curl -s -o /dev/null -L -w ''%{http_code}'' "http://localhost:3000/")" != "200" ]; do
+            echo "=> Waiting for service to become available" && sleep 2s
+          done
+
+      - name: "Setup Cypress"
+        timeout-minutes: 1
+        env: 
+          SSH_AUTH_SOCK: /tmp/ssh_agent.sock
+        run: |
+          set -euo pipefail
+          cd ./tests/functional
+          yarn install --frozen-lockfile
+
+      - name: Run Cypress test suite
+        timeout-minutes: 40
+        env:
+          NODE_ENV: 'production'
+          COLLECT_COVERAGE: 'true'
+          CPU_NUMBER: ${{ steps.cpu-info.outputs.cores-count }}
+        run: |
+          set -euo pipefail
+          cd ./tests/functional
+          yarn run test:parallel
+
+      - name: "Upload suite output" 
+        uses: actions/upload-artifact@v3
+        if: ${{ failure() }}
+        with:
+          name: failure-result
+          path: ./tests/functional
+
+      - name: Prepare coverage report
+        if: ${{ success() }}
+        run: |
+          cd ./tests/functional
+          rm -rf coverage
+          yarn cvg:report
+
+      - name: Upload coverage to Artifact
+        uses: actions/upload-artifact@v3
+        if: ${{ success() }}
+        with:
+          name: cypress-tests-coverage
+          path: ./tests/functional/coverage/
+
diff --git a/.github/workflows/tests_coverage.yml b/.github/workflows/tests_coverage.yml
@@ -55,3 +55,25 @@ jobs:
           fail_ci_if_error: true
           token: ${{ secrets.codecov_token }}
 
+  run_cypress_coverage_upload:
+    name: Upload Cypress coverage
+    runs-on: ubuntu-latest
+
+    if: "!contains(github.event.head_commit.message, 'skip ci')"
+
+    steps:
+      - name: "Checkout codebase"
+        uses: actions/checkout@v3
+
+      - name: "Download Cypress coverage from Artifact"
+        uses: actions/download-artifact@v3
+        with:
+          name: cypress-tests-coverage
+          path: coverage/
+
+      - name: Upload coverage to Codecov
+        uses: codecov/[email protected]
+        if: ${{ !github.event.pull_request.head.repo.fork }}
+        with:
+          fail_ci_if_error: true
+          token: ${{ secrets.codecov_token }}