Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: DBTP-1700 Deprecate cross_enviroment_service_access application property #780

Merged
merged 3 commits into from
Feb 20, 2025
Merged

chore: DBTP-1700 Deprecate cross_enviroment_service_access application property #780

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6c2a16c
DBTP-1700 Deprecatecross_enviroment_service_access application property
WillGibson Feb 20, 2025
d4a7a93
Merge branch 'main' into DBTP-1700-deprecate-cross_environment-servic…
WillGibson Feb 20, 2025
f206f3b
Merge branch 'main' into DBTP-1700-deprecate-cross_environment-servic…
WillGibson Feb 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion dbt_platform_helper/providers/platform_config_schema.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -506,7 +506,9 @@ def _valid_s3_bucket_arn(key):
},
Optional("cross_environment_service_access"): {
PlatformConfigSchema.__valid_schema_key(): {
"application": str,
# Deprecated: We didn't implement cross application access, no service teams are asking for it.
# application should be removed once we can confirm that no-one is using it.
Optional("application"): str,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed with @antroy-madetech and @ksugden.

"environment": PlatformConfigSchema.__valid_environment_name(),
"account": str,
"service": str,
Expand Down
6 changes: 0 additions & 6 deletions tests/platform_helper/domain/test_copilot_environment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,6 @@ def s3_xenv_extensions(self):
"bucket_name": "x-acc-bucket",
"cross_environment_service_access": {
"test_access": {
"application": "app2",
"environment": "staging",
"account": "123456789010",
"service": "test_svc",
Expand All @@ -137,7 +136,6 @@ def s3_xenv_multiple_extensions(self):
"bucket_name": "x-acc-bucket-1",
"cross_environment_service_access": {
"test_access_1": {
"application": "app1",
"environment": "staging",
"account": "123456789010",
"service": "other_svc_1",
Expand All @@ -146,7 +144,6 @@ def s3_xenv_multiple_extensions(self):
"cyber_sign_off_by": "[email protected]",
},
"test_access_2": {
"application": "app2",
"environment": "dev",
"account": "123456789010",
"service": "other_svc_2",
Expand All @@ -166,7 +163,6 @@ def s3_xenv_multiple_extensions(self):
"bucket_name": "x-acc-bucket-2",
"cross_environment_service_access": {
"test_access_3": {
"application": "app2",
"environment": "hotfix",
"account": "987654321010",
"service": "other_svc_2",
Expand All @@ -180,7 +176,6 @@ def s3_xenv_multiple_extensions(self):
"bucket_name": "x-acc-bucket-3",
"cross_environment_service_access": {
"test_access_4": {
"application": "app2",
"environment": "staging",
"account": "123456789010",
"service": "other_svc_3",
Expand All @@ -194,7 +189,6 @@ def s3_xenv_multiple_extensions(self):
"bucket_name": "x-acc-bucket-4",
"cross_environment_service_access": {
"test_access_5": {
"application": "app2",
"environment": "staging",
"account": "123456789010",
"service": "other_svc_4",
Expand Down
5 changes: 2 additions & 3 deletions tests/platform_helper/utils/fixtures/addons_files/s3_addons.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,11 +55,11 @@ my-s3-bucket-with-data-migration:
bucket_name: s3-data-migration
versioning: false
data_migration:
import:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role

my-s3-bucket-with-data-migration-import-sources:
type: s3
environments:
Expand Down Expand Up @@ -91,7 +91,6 @@ my-s3-cross-environment-service-access-bucket:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: prod
account: test-account
service: web
Expand Down
46 changes: 12 additions & 34 deletions tests/platform_helper/utils/fixtures/addons_files/s3_addons_bad_data.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ my-s3-bucket-data-migration-source-bucket-invalid-arn:
import:
source_bucket_arn: 1234abc
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role

my-s3-bucket-data-migration-source-kms-key-invalid-arn:
type: s3
Expand All @@ -141,7 +141,7 @@ my-s3-bucket-data-migration-source-kms-key-invalid-arn:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: 1234abc
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role

my-s3-bucket-data-migration-worker-role-invalid-arn:
type: s3
Expand All @@ -153,7 +153,7 @@ my-s3-bucket-data-migration-worker-role-invalid-arn:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: 1234abc

my-s3-bucket-data-migration-import-sources-source-bucket-2-invalid-arn:
type: s3
environments:
Expand All @@ -167,7 +167,7 @@ my-s3-bucket-data-migration-import-sources-source-bucket-2-invalid-arn:
- source_bucket_arn: arn:aws:s3:::valid-source-bucket-2
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key-3
worker_role_arn: arn:aws:iam::123456789012:role/test-role-3

my-s3-bucket-data-migration-import-cannot-have-both-import-and-import-sources:
type: s3
environments:
Expand Down Expand Up @@ -195,7 +195,7 @@ my-s3-bucket-data-migration-import-sources-source-bucket-3-invalid-arn:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role
import_sources:
- source_bucket_arn: arn:aws:s3:::valid-source-bucket-2
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key-2
Expand All @@ -213,7 +213,7 @@ my-s3-bucket-data-migration-import-sources-kms-key-invalid-arn:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role
import_sources:
- source_bucket_arn: arn:aws:s3:::test-app-2
source_kms_key_arn: 1234abc
Expand All @@ -228,7 +228,7 @@ my-s3-bucket-data-migration-import-sources-worker-role-invalid-arn:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role
import_sources:
- source_bucket_arn: arn:aws:s3:::test-app-2
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key-2
Expand All @@ -243,7 +243,7 @@ my-s3-bucket-data-migration-import-sources-empty:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role
import_sources: []

my-s3-bucket-data-migration-import-sources-missing-bucket-arn:
Expand All @@ -255,11 +255,11 @@ my-s3-bucket-data-migration-import-sources-missing-bucket-arn:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role
import_sources:
- source_kms_key_arn: arn:aws:kms::123456789012:key/test-key-2
worker_role_arn: arn:aws:iam::123456789012:role/test-role-2

my-s3-bucket-data-migration-import-sources-missing-worker-role-arn:
type: s3
environments:
Expand All @@ -269,10 +269,10 @@ my-s3-bucket-data-migration-import-sources-missing-worker-role-arn:
import:
source_bucket_arn: arn:aws:s3:::test-app
source_kms_key_arn: arn:aws:kms::123456789012:key/test-key
worker_role_arn: arn:aws:iam::123456789012:role/test-role
worker_role_arn: arn:aws:iam::123456789012:role/test-role
import_sources:
- source_kms_key_arn: arn:aws:kms::123456789012:key/test-key-2
source_bucket_arn: arn:aws:s3:::test-app-2
source_bucket_arn: arn:aws:s3:::test-app-2

my-s3-external-access-bucket-invalid-arn:
type: s3
Expand Down Expand Up @@ -305,7 +305,6 @@ my-s3-cross-environment-service-access-bucket-invalid-environment:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: hyphen-not-allowed-in-environment
account: test-account
service: web
Expand All @@ -320,36 +319,20 @@ my-s3-cross-environment-service-access-bucket-invalid-email:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: anotherenvironment
account: test-account
service: web
write: True
read: True
cyber_sign_off_by: noone-signed-this-off

my-s3-cross-environment-service-access-bucket-missing-application:
type: s3
environments:
dev:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
environment: anotherenvironment
account: test-account
service: web
write: True
read: True
cyber_sign_off_by: [email protected]

my-s3-cross-environment-service-access-bucket-missing-environment:
type: s3
environments:
dev:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
account: test-account
service: web
write: True
Expand All @@ -363,7 +346,6 @@ my-s3-cross-environment-service-access-bucket-missing-account:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: anotherenvironment
service: web
write: True
Expand All @@ -377,7 +359,6 @@ my-s3-cross-environment-service-access-bucket-missing-service:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: anotherenvironment
account: test-account
write: True
Expand All @@ -391,7 +372,6 @@ my-s3-cross-environment-service-access-bucket-invalid-write:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: anotherenvironment
account: test-account
service: web
Expand All @@ -406,7 +386,6 @@ my-s3-cross-environment-service-access-bucket-invalid-read:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: anotherenvironment
account: test-account
service: web
Expand All @@ -421,7 +400,6 @@ my-s3-cross-environment-service-access-bucket-missing-cyber-sign-off:
bucket_name: mandatory
cross_environment_service_access:
demodjango-hotfix:
application: test-app
environment: anotherenvironment
account: test-account
service: web
Expand Down
1 change: 0 additions & 1 deletion tests/platform_helper/utils/test_validation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,6 @@ def test_validate_addons_success(addons_file):
"my-s3-external-access-bucket-invalid-email": r"cyber_sign_off_by must contain a valid DBT email address",
"my-s3-cross-environment-service-access-bucket-invalid-environment": r"Environment name hyphen-not-allowed-in-environment is invalid",
"my-s3-cross-environment-service-access-bucket-invalid-email": r"cyber_sign_off_by must contain a valid DBT email address",
"my-s3-cross-environment-service-access-bucket-missing-application": r"Missing key: 'application'",
"my-s3-cross-environment-service-access-bucket-missing-environment": r"Missing key: 'environment'",
"my-s3-cross-environment-service-access-bucket-missing-account": r"Missing key: 'account'",
"my-s3-cross-environment-service-access-bucket-missing-service": r"Missing key: 'service'",
Expand Down