Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: deploy test resources w/ new test packages #433

Merged
merged 61 commits into from
Aug 26, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
9c38627
chore: deploy test resources
stijnmoreels Jun 4, 2024
a9e1018
pr-fix: missing assignment keyword
stijnmoreels Jun 4, 2024
3571cb0
pr-fix: correct rg assignment
stijnmoreels Jun 4, 2024
be1babb
pr-fix: key vault name assignment
stijnmoreels Jun 4, 2024
2d60c6f
pr-fix: add location param + correct existing service principal condi…
stijnmoreels Jun 4, 2024
da52745
pr-fix: add depends on
stijnmoreels Jun 4, 2024
3af2fc5
pr-fix: use correct way to retrieve secret version
stijnmoreels Jun 4, 2024
6f4014b
pr-fix: add logging for sp-related commands
stijnmoreels Jun 6, 2024
9a4a67a
pr-fix: use solely security-related resources
stijnmoreels Jun 6, 2024
ea31397
pr-fix: add logging for secret version
stijnmoreels Jun 6, 2024
2b92d6f
pr-fix: add secret version other way
stijnmoreels Jun 6, 2024
93505cf
pr-fix: use property outputs
stijnmoreels Jun 6, 2024
b3c3885
pr-fix: add logging for deploy outputs
stijnmoreels Jun 6, 2024
32ce4e2
pr-fix: correct new output variable assignment
stijnmoreels Jun 6, 2024
98635df
pr-fix: mark as string
stijnmoreels Jun 6, 2024
e26f620
pr-add: integrate tests and key vault
stijnmoreels Jun 6, 2024
a3dd37f
pr-fix: hashicorp template path
stijnmoreels Jun 6, 2024
2a22d9b
pr-fix: add devops resources
stijnmoreels Jun 6, 2024
3cc7f5d
pr-fix: add test variables
stijnmoreels Jun 6, 2024
23bbaa2
pr-fix: az identity vulnerability
stijnmoreels Jun 6, 2024
4a81c54
pr-fix: correct secret value
stijnmoreels Jun 6, 2024
d3d7223
pr-fix: run secret retrieval as a pre-job
stijnmoreels Jun 6, 2024
c3c3f7a
pr-fix: get own key vault secrets
stijnmoreels Jun 7, 2024
d8338dd
pr-fix: correct parameters
stijnmoreels Jun 7, 2024
df7ca09
pr-fix: install module az.keyvault
stijnmoreels Jun 7, 2024
8a8954b
pr-fix: use az keyvault
stijnmoreels Jun 7, 2024
18aa43f
pr-fix: use client id as var
stijnmoreels Jun 7, 2024
d89314a
pr-fix: remove app insights reference
stijnmoreels Jun 7, 2024
cd4b3e1
pr-fix: add infra smoke tests
stijnmoreels Jun 10, 2024
2e28460
pr-fix: use az cli task
stijnmoreels Jun 10, 2024
cc7b1cd
pr-fix: argument syntax
stijnmoreels Jun 10, 2024
8c8d837
pr-fix: add enabled assertion
stijnmoreels Jun 10, 2024
bb33679
pr-fix: correct running
stijnmoreels Jun 10, 2024
d564e91
pr-fix: add az module
stijnmoreels Jun 10, 2024
129ef28
pr-fix: use new arguments syntax
stijnmoreels Jun 10, 2024
d8694af
pr-fix: use higher version of pester
stijnmoreels Jun 10, 2024
6368d88
pr-fix: use at least 5.3.0
stijnmoreels Jun 10, 2024
737e3a1
pr-fix: use env variables
stijnmoreels Jun 10, 2024
686b930
pr-fix: correct test result
stijnmoreels Jun 10, 2024
1faf523
pr-fix: import module
stijnmoreels Jun 10, 2024
d923837
pr-fix: remove param
stijnmoreels Jun 10, 2024
0bedb01
pr-fix: enable test result
stijnmoreels Jun 10, 2024
1cf0915
pr-fix: correct env vars
stijnmoreels Jun 10, 2024
31a2256
pr-fix: use other env vars
stijnmoreels Jun 10, 2024
2bf6eb2
pr-fix: use correct secret version extraction
stijnmoreels Jun 11, 2024
83e9de9
pr-fix: use pester container for external data
stijnmoreels Jun 11, 2024
350490b
pr-fix: broaden test assertion + trim secret version setup
stijnmoreels Jun 11, 2024
c04b2a3
pr-fix: simplify config value retrieval
stijnmoreels Jun 11, 2024
32cb930
pr-fix: remove any spaces from version and secret
stijnmoreels Jun 11, 2024
606afae
pr-fix: use direct setting of variable
stijnmoreels Jun 11, 2024
ef68b2c
pr-fix: clean tests
stijnmoreels Jun 11, 2024
8c9d941
pr-fix: remove tried smoke tests
stijnmoreels Jun 11, 2024
af93499
Update Arcus.Security.Providers.AzureKeyVault.csproj
stijnmoreels Jun 18, 2024
2c19040
Update Arcus.Security.Providers.AzureKeyVault.csproj
stijnmoreels Jun 18, 2024
da97464
pr-fix: use most recent test fixtures
stijnmoreels Aug 6, 2024
14760cd
Merge branch 'chore/deploy-test-resources' of https://github.com/stij…
stijnmoreels Aug 6, 2024
51ab492
pr-fix: use correct unauthorized secret names
stijnmoreels Aug 6, 2024
3294b2e
pr-fix: remove remote resource group
stijnmoreels Aug 6, 2024
4910f93
Update deploy-test-resources.yml
stijnmoreels Aug 6, 2024
aef7f87
Update deploy-test-resources.yml
stijnmoreels Aug 6, 2024
6ba344e
Update deploy-test-resources.yml
stijnmoreels Aug 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions build/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ parameters:
- name: 'Package.Version.ManualTrigger'
type: string
default: 'preview'
- name: azureServiceConnection
displayName: 'Azure service connection'
type: string
default: 'Azure Codit-Arcus Service Principal'

resources:
repositories:
Expand All @@ -27,9 +31,6 @@ resources:
endpoint: arcus-azure

variables:
# 'Arcus_ServicePrincipal_AccessKey' is added as secret on build in Azure DevOps
- group: 'Arcus Security - Integration Testing'
- group: 'Arcus - GitHub Package Registry'
- group: 'Build Configuration'
- template: ./variables/build.yml
- template: ./variables/test.yml
Expand Down Expand Up @@ -106,14 +107,10 @@ stages:
inputs:
artifact: 'Build'
path: '$(Build.SourcesDirectory)'
- template: 'templates/download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- template: templates/run-integration-tests.yml
parameters:
dockerProjectName: '$(Project).Tests.Runtimes.AzureFunctions'
azureServiceConnection: '${{ parameters.azureServiceConnection }}'

- stage: ReleaseToMyget
displayName: 'Release to MyGet'
Expand Down
54 changes: 54 additions & 0 deletions build/deploy-test-resources.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
name: Arcus Security - Deploy test resources

trigger: none
pr: none

parameters:
- name: azureServiceConnection
displayName: 'Azure service connection'
type: string
default: 'Azure Codit-Arcus Service Principal'
- name: resourceGroupName
displayName: 'Resource group name'
default: arcus-security-dev-we-rg

variables:
- template: ./variables/build.yml
- template: ./variables/test.yml

resources:
repositories:
- repository: templates
type: github
name: arcus-azure/azure-devops-templates
endpoint: arcus-azure

stages:
- stage: Deploy
jobs:
- job: DeployBicep
displayName: 'Deploy test resources'
pool:
vmImage: '$(Vm.Image)'
steps:
- task: AzureCLI@2
inputs:
azureSubscription: '${{ parameters.azureServiceConnection }}'
addSpnToEnvironment: true
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
$secretName = $env:ARCUS_SECURITY_KEYVAULT_TESTSECRETNAME
$secretValue = [System.Guid]::NewGuid().ToString()
$objectId = (az ad sp show --id $env:servicePrincipalId | ConvertFrom-Json).id

az deployment sub create `
--location westeurope `
--template-file ./build/templates/deploy-test-resources.bicep `
--parameters location=westeurope `
--parameters resourceGroupName=${{ parameters.resourceGroupName }} `
--parameters keyVaultName=$env:ARCUS_SECURITY_KEYVAULT_NAME `
--parameters secretName=$secretName `
--parameters secretValue=$secretValue `
--parameters servicePrincipal_objectId=$objectId `
| ConvertFrom-Json
7 changes: 0 additions & 7 deletions build/nuget-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,6 @@ resources:
endpoint: arcus-azure

variables:
- group: 'Arcus Security - Integration Testing'
- group: 'Arcus - GitHub Package Registry'
- group: 'Build Configuration'
- template: ./variables/build.yml
- template: ./variables/test.yml
Expand Down Expand Up @@ -92,11 +90,6 @@ stages:
inputs:
artifact: 'Build'
path: '$(Build.SourcesDirectory)'
- template: 'templates/download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- template: templates/run-integration-tests.yml
parameters:
dockerProjectName: '$(Project).Tests.Runtimes.AzureFunctions'
Expand Down
56 changes: 56 additions & 0 deletions build/templates/deploy-test-resources.bicep
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
// Define the location for the deployment of the components.
param location string

// Define the name of the resource group where the components will be deployed.
param resourceGroupName string

// Define the name of the Key vault.
param keyVaultName string

// Define the name of the secret that will be added to the Key vault.
param secretName string

// Define the secret value that will be by default added to the Key vault.
@secure()
param secretValue string

// Define the Service Principal ID that needs access full access to the deployed resource group.
param servicePrincipal_objectId string

targetScope='subscription'

module resourceGroup 'br/public:avm/res/resources/resource-group:0.2.3' = {
name: 'resourceGroupDeployment'
params: {
name: resourceGroupName
location: location
}
}

resource rg 'Microsoft.Resources/resourceGroups@2021-04-01' existing = {
name: resourceGroupName
}

module vault 'br/public:avm/res/key-vault/vault:0.6.1' = {
name: 'vaultDeployment'
dependsOn: [
resourceGroup
]
scope: rg
params: {
name: keyVaultName
location: location
roleAssignments: [
{
principalId: servicePrincipal_objectId
roleDefinitionIdOrName: 'Key Vault Secrets officer'
}
]
secrets: [
{
name: secretName
value: secretValue
}
]
}
}
33 changes: 33 additions & 0 deletions build/templates/run-integration-tests.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
parameters:
dockerProjectName: ''
azureServiceConnection: ''

steps:
- bash: |
Expand All @@ -9,6 +10,38 @@ steps:
fi
env:
PROJECT_NAME: ${{ parameters.dockerProjectName }}
- task: AzureCLI@2
displayName: 'Import secrets from Azure Key Vault'
inputs:
azureSubscription: '${{ parameters.azureServiceConnection }}'
addSpnToEnvironment: true
scriptType: 'pscore'
scriptLocation: 'inlineScript'
inlineScript: |
Set-PSRepository -Name PSGallery -InstallationPolicy Trusted
Install-Module -Name Arcus.Scripting.DevOps -AllowClobber

Set-AzDevOpsVariable 'Arcus.Security.TenantId' -Value $env:tenantId -AsSecret
Set-AzDevOpsVariable 'Arcus.Security.ServicePrincipal.ClientId' -Value $env:servicePrincipalId -AsSecret
Set-AzDevOpsVariable 'Arcus.Security.ServicePrincipal.ClientSecret' -Value $env:servicePrincipalKey -AsSecret

$unauthorizedClientId = az keyvault secret show --name $env:ARCUS_GENERAL_UNAUTHORIZED_SERVICEPRINCIPAL_CLIENTID_SECRETNAME --vault-name $env:ARCUS_GENERAL_KEYVAULT_NAME | ConvertFrom-Json
$unauthorizedClientSecret = az keyvault secret show --name $env:ARCUS_GENERAL_UNAUTHORIZED_SERVICEPRINCIPAL_CLIENTSECRET_SECRETNAME --vault-name $env:ARCUS_GENERAL_KEYVAULT_NAME | ConvertFrom-Json
Set-AzDevOpsVariable 'Arcus.Security.Unauthorized.ServicePrincipal.ClientId' -Value $unauthorizedClientId.value -AsSecret
Set-AzDevOpsVariable 'Arcus.Security.Unauthorized.ServicePrincipal.ClientSecret' -Value $unauthorizedClientSecret.value -AsSecret

$testSecret = az keyvault secret show --name $env:ARCUS_SECURITY_KEYVAULT_TESTSECRETNAME --vault-name $env:ARCUS_SECURITY_KEYVAULT_NAME | ConvertFrom-Json
$testSecretVersion = $testSecret.id.Split('/') | Select-Object -Last 1
Write-Host "Test secret '$($testSecret.name)' version is '$testSecretVersion'"
$testSecretValue = $testSecret.value

Set-AzDevOpsVariable -AsSecret -Name 'Arcus.Security.KeyVault.TestSecretValue' -Value $testSecretValue
Set-AzDevOpsVariable -AsSecret -Name 'Arcus.Security.KeyVault.TestSecretVersion' -Value $testSecretVersion
- template: 'download-hashicorp-vault.yml'
parameters:
targetFolder: '$(Build.SourcesDirectory)'
version: $(HashiCorp.Vault.Version)
vaultBinVariableName: 'Arcus.HashiCorp.VaultBin'
- task: UseDotNet@2
displayName: 'Import .NET Core SDK ($(DotNet.Sdk.VersionBC))'
inputs:
Expand Down
11 changes: 7 additions & 4 deletions build/variables/test.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
variables:
Arcus.KeyVault.TestKeyName: "ArcusTestSecret"
Arcus.KeyVault.TestKeyVersion: "8bde7a16366849e28b7abe26732e12e3"
HashiCorp.Vault.Version: 1.5.0
Arcus.AzureFunctions.HttpPort: "5000"
Arcus.Security.KeyVault.Name: 'arcus-security-kv'
Arcus.Security.KeyVault.TestSecretName: 'ArcusTestSecret'
Arcus.General.KeyVault.Name: 'arcus-kv'
Arcus.General.Unauthorized.ServicePrincipal.ClientId.SecretName: 'Arcus-Unauthorized-ServicePrincipal-ClientId'
Arcus.General.Unauthorized.ServicePrincipal.ClientSecret.SecretName: 'Arcus-Unauthorized-ServicePrincipal-ClientSecret'
Arcus.AzureFunctions.HttpPort: '5000'
HashiCorp.Vault.Version: 1.5.0
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
</ItemGroup>

<ItemGroup>
<PackageReference Include="Azure.Identity" Version="1.10.2" />
<PackageReference Include="Azure.Identity" Version="1.11.4" />
<PackageReference Include="Azure.Security.KeyVault.Secrets" Version="4.5.0" />
<PackageReference Include="Microsoft.Extensions.Configuration" Version="6.0.0" />
<PackageReference Include="Polly" Version="7.2.1" />
Expand All @@ -34,4 +34,4 @@
<ProjectReference Include="..\Arcus.Security.Core\Arcus.Security.Core.csproj" />
</ItemGroup>

</Project>
</Project>
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<Project Sdk="Microsoft.NET.Sdk">

<PropertyGroup>
<TargetFramework>netstandard2.1</TargetFramework>
<TargetFrameworks>net6.0;net8.0</TargetFrameworks>
</PropertyGroup>

<ItemGroup>
<PackageReference Include="Guard.Net" Version="1.2.0" />
<PackageReference Include="Arcus.Testing.Logging" Version="0.5.0" />
<PackageReference Include="Arcus.Testing.Security.Providers.InMemory" Version="0.5.0" />
<PackageReference Include="Guard.Net" Version="3.0.0" />
<PackageReference Include="Arcus.Testing.Logging.Xunit" Version="1.0.3" />
<PackageReference Include="Arcus.Testing.Security.Providers.InMemory" Version="1.0.3" />
<PackageReference Include="Serilog" Version="2.10.0" />
</ItemGroup>

Expand Down
45 changes: 0 additions & 45 deletions src/Arcus.Security.Tests.Core/Stubs/SpyLogger.cs

This file was deleted.

36 changes: 0 additions & 36 deletions src/Arcus.Security.Tests.Core/Stubs/TestLoggerProvider.cs

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

<ItemGroup>
<FrameworkReference Include="Microsoft.AspNetCore.App" />
<PackageReference Include="Arcus.Testing.Core" Version="1.0.3" />
<PackageReference Include="Arcus.Observability.Telemetry.Serilog.Sinks.ApplicationInsights" Version="3.0.0" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="16.7.1" />
<PackageReference Include="Serilog.Extensions.Hosting" Version="3.1.0" />
Expand Down
Loading
Loading