Add tests for default-python template on different Python versions #2025

denik · 2024-12-17T11:34:45Z

Changes

Add new type of test helpers that run the command and compare full output (golden files approach).

In case of JSON, there is also an option to ignore certain paths.

Add test for different versions of Python to go through bundle init default-python / validate / deploy / summary.

Tests

New integration tests.

go.mod

…path

I've also considered https://pkg.go.dev/github.com/stretchr/testify/require#TestingT to avoid dependency on /internal/ but that one does not have Logf method that I use

Instead, store original WD on env var.

pietern

Few remaining questions/suggestions.

internal/testutil/env.go

pietern · 2024-12-20T14:07:00Z

libs/testdiff/testdiff.go

+}
+
+func AssertEqualTexts(t testutil.TestingT, filename1, filename2, expected, out string) {
+	if len(out) < 1000 && len(expected) < 1000 {


Constants (used in multiple spots) can be a top level const, e.g. textDiffCharacterThreshold or smth.

pietern · 2024-12-20T14:08:18Z

libs/testdiff/testdiff.go

+}
+
+// compare strings and ignore forward vs backward slashes
+func almostSameStrings(v1, v2 any) bool {


Where is this needed? Didn't see it before.

b972bcd

This is to fix slashes on windows. See below "whl" for example. https://github.com/databricks-eng/eng-dev-ecosystem/actions/runs/12432516778/job/34712095471

testdiff.go:49: Allowed differences between C:\a\eng-dev-ecosystem\eng-dev-ecosystem\ext\cli\integration\bundle\testdata\default_python\bundle_summary.txt and Output from [bundle summary --output json]: ==> replace /bundle/terraform/exec_path C:\Users\RUNNER~1\AppData\Local\Temp\TestDefaultPython3.<NUMID>\<NUMID>\project_name_$UNIQUE_PRJ\.databricks\bundle\dev\bin\terraform.exe old=/tmp/.../terraform ==> replace /include/0 resources\project_name_$UNIQUE_PRJ.job.yml old=resources/project_name_$UNIQUE_PRJ.job.yml ==> replace /include/1 resources\project_name_$UNIQUE_PRJ.pipeline.yml old=resources/project_name_$UNIQUE_PRJ.pipeline.yml ==> remove /resources/jobs/project_name_$UNIQUE_PRJ_job/email_notifications <nil> old=map[on_failure:[$USERNAME]] ==> replace /resources/jobs/project_name_$UNIQUE_PRJ_job/job_clusters/0/new_cluster/node_type_id Standard_D3_v2 old=i3.xlarge ==> replace /resources/jobs/project_name_$UNIQUE_PRJ_job/tasks/2/libraries/0/whl dist\*.whl old=dist/*.whl ==> replace /resources/jobs/project_name_$UNIQUE_PRJ_job/url https://$DATABRICKS_HOST/jobs/<NUMID> old=https://$DATABRICKS_HOST/jobs/<NUMID>?o=<NUMID> ==> remove /resources/pipelines/project_name_$UNIQUE_PRJ_pipeline/catalog <nil> old=main ==> replace /resources/pipelines/project_name_$UNIQUE_PRJ_pipeline/url https://$DATABRICKS_HOST/pipelines/<UUID> old=https://$DATABRICKS_HOST/pipelines/<UUID>?o=<NUMID> ==> add /workspace/current_user/externalId <UUID> old=<nil> ==> replace /workspace/current_user/groups/0/display admins old=team.engineering ==> remove /workspace/current_user/name/familyName <nil> old=$USERNAME

github-actions · 2024-12-20T14:19:53Z

If integration tests don't run automatically, an authorized user can run them manually by following the instructions below:

Trigger:
go/deco-tests-run/cli

Inputs:

PR number: 2025
Commit SHA: 74345897c2b043417cdce3fbc2a2b876c83a82d6

Checks will be approved automatically on success.

denik temporarily deployed to test-trigger-is December 17, 2024 11:34 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 17, 2024 11:35 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 17, 2024 13:51 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 17, 2024 16:28 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 17, 2024 19:12 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 17, 2024 20:41 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 17, 2024 21:32 — with GitHub Actions Inactive

pietern reviewed Dec 18, 2024

View reviewed changes

go.mod Outdated Show resolved Hide resolved

denik force-pushed the DECO-24194--denik--python-tests branch from 930057b to 1f8896c Compare December 18, 2024 13:09

denik temporarily deployed to test-trigger-is December 18, 2024 13:09 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 13:44 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 14:22 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 15:00 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 15:20 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 15:33 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 16:12 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 16:13 — with GitHub Actions Inactive

denik temporarily deployed to test-trigger-is December 18, 2024 17:13 — with GitHub Actions Inactive

denik added 11 commits December 20, 2024 14:42

fix test - add missing Chdir

004d0df

fix comparison on Windows: ignore slash difference, ignore terraform …

b972bcd

…path

clean ups; update NOTICE

c8beeb2

Remove orderedmap; use slice wrapped in struct

f6e8f26

switch to testutil.TestingT interface

8945ee3

I've also considered https://pkg.go.dev/github.com/stretchr/testify/require#TestingT to avoid dependency on /internal/ but that one does not have Logf method that I use

re-order replacement

b91e7fb

disable $DATABRICKS_AUTH_TYPE and add a comment

b645c47

update NOTICE; fix formatting

e685482

restore "go 1.23" line

90b7413

rename RequireOutput to AssertOutput

1e242cb

get rid of "_, filename, _, _ := runtime.Caller(1)"

418564a

Instead, store original WD on env var.

denik force-pushed the DECO-24194--denik--python-tests branch from abcf45d to 418564a Compare December 20, 2024 13:44

denik temporarily deployed to test-trigger-is December 20, 2024 13:44 — with GitHub Actions Inactive

denik requested review from andrewnester and pietern December 20, 2024 13:48

denik enabled auto-merge December 20, 2024 13:48

pietern approved these changes Dec 20, 2024

View reviewed changes

denik added this pull request to the merge queue Dec 20, 2024

pietern removed this pull request from the merge queue due to a manual request Dec 20, 2024

Comment on TestData

7434589

denik temporarily deployed to test-trigger-is December 20, 2024 14:19 — with GitHub Actions Inactive

denik enabled auto-merge December 20, 2024 14:22

denik added this pull request to the merge queue Dec 20, 2024

andrewnester approved these changes Dec 20, 2024

View reviewed changes

Merged via the queue into main with commit e095249 Dec 20, 2024
9 checks passed

denik deleted the DECO-24194--denik--python-tests branch December 20, 2024 14:47

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add tests for default-python template on different Python versions #2025

Add tests for default-python template on different Python versions #2025

denik commented Dec 17, 2024 •

edited

Loading

pietern left a comment

pietern Dec 20, 2024

pietern Dec 20, 2024

denik Dec 20, 2024

github-actions bot commented Dec 20, 2024

Add tests for default-python template on different Python versions #2025

Add tests for default-python template on different Python versions #2025

Conversation

denik commented Dec 17, 2024 • edited Loading

Changes

Tests

pietern left a comment

Choose a reason for hiding this comment

pietern Dec 20, 2024

Choose a reason for hiding this comment

pietern Dec 20, 2024

Choose a reason for hiding this comment

denik Dec 20, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 20, 2024

denik commented Dec 17, 2024 •

edited

Loading