feat(core): ARC-1906 fetch env from ssm

[Tyagi-Sunny]

fetch env from ssm

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context. List any dependencies that are required for this change.

Fixes # (issue)

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Intermediate change (work in progress)

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

• Test A
• Test B

Checklist:

• Performed a self-review of my own code
• npm test passes on your machine
• New tests added or existing tests modified to cover all changes
• Code conforms with the style guide
• API Documentation in code was updated
• Any dependent changes have been merged and published in downstream modules

[sonarqubecloud]

Quality Gate failed

Failed conditions

1 New Bugs (required ≤ 0)
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarCloud

Catch issues before they fail your Quality Gate with our IDE extension SonarLint

[Yogi-Vishwas]

This misses the case when there are comments in the env.
There can be both inline and a whole line comments.
I think the regex is already splitting the string on space, new line and = so you just need to filter values that begin with #.
readFileSync(path, "utf8").split(/[\n =]/).filter(Boolean).filter((elem) => !elem.startsWith("#"));

also I think it would be better to split it into lines by splitting it first into "\n" and then take the left side of "=" (i.e. line[0]), because this code assumes that the values will not be there and the values will be filtered using filter(Boolean) so in case the user has a value with a property in the .env.schema this code will treat that (value) as property also. You don't need to deal with values anyway, you just need properties.

you can use this
const lines = readFileSync(path, "utf-8").split("\n").filter((line) => line.length);
const properties = lines.map((line) => line.trim().split("=")[0].trim()).filter((line) => !line.startsWith("#"));

[Tyagi-Sunny]

hmm ,
better

[jamescrowley321]

Why would you always assume a .env file? We should just be checking for environment variables. Assuming an ENV assumes a CI/CD process creates a file when it does

[jamescrowley321]

We can use dotenv. Let's not hand write anything to read an env file or environment variables.

[Yogi-Vishwas]

Why do we need to add properties here also?
This would be duplication of effort for the user whenever user adds a new property. User will have to add that property here also.
We can just pick the properties from the .env. Whatever properties are there we can fetch from ssm and feed it into lambda env variables.
cc: @jamescrowley321

[jamescrowley321]

These are all params that should be loaded from SSM.

The idea is that the env file is a list of SSM param keys, we query SSM for those keys at runtime, and then we pass the values from SSM to the CDK where needed.

[Tyagi-Sunny]

we will be maintainig all our env variables here only. not in .env.
we can bring our values from .env , but in thatcase we can not use process.env for that. either we have to use fs methods or something else. because process.env brings in many other variables too. And using fs methods with .env, it does not seems usual, i also went through some workaround but didn't get the funcitonal way. that's why i followed this approach.

storing all our env inside an object makes them accessible easily, we don't have to add any further logic to separate our required env from the system env variables.

this is one approcah, and may be its possible to do in some better way

[Yogi-Vishwas]

Let's discuss it tomorrow @Tyagi-Sunny

[Yogi-Vishwas]

@jamescrowley321 so we can easily assume that the properties that user wants to fetch from SSM. User will have to specify those beforehand.
So we can create an array propertiesToFetch and user have to add properties there
whatever properties user will add we'll fetch it from ssm.
and all other non-sensitive values can be picked from .env

[jamescrowley321]

Here is what I recommend

• ENV file is a list of values like DB_HOST_SSM_PARAM_KEY=whatever/the/key/value/is
• Create a configuration type, i.e. AuthServiceConfiguration
• Create a type such as AuthServiceConfigurationLoader (or w/e you want to name it) and a concrete implementation that encapsulates reading the ENV variables and looking up the values from SSM to return an instance of AuthServiceConfiguration.
• Only reference AuthServiceConfiguration
• Inject AuthServiceConfigurationLoader where you can
• Then the devs can also use that interface to create a Mock/Stub/Local version of the AuthServiceConfigurationLoader to read from other param stores
• We may not always use SSM to load the variables, so decouple things where you can.

[samarpan-b]

Why cant we do all of this in Code Pipeline ? Ideally we do these things in code pipeline or jenkins or whatever tool we use for CI/CD. So that application code remains unaware of tool to use. It just accesses everything from env. CI/CD pipeline scripts read values from SSM or Vault and put into the env while deploying the service.

[jamescrowley321]

These are all params that should be loaded from SSM.

The idea is that the env file is a list of SSM param keys, we query SSM for those keys at runtime, and then we pass the values from SSM to the CDK where needed.

[jamescrowley321]

Why would you always assume a .env file? We should just be checking for environment variables. Assuming an ENV assumes a CI/CD process creates a file when it does

[jamescrowley321]

This implementation is still incorrect.

[jamescrowley321]

We can use dotenv. Let's not hand write anything to read an env file or environment variables.

[jamescrowley321]

Here is what I recommend

• ENV file is a list of values like DB_HOST_SSM_PARAM_KEY=whatever/the/key/value/is
• Create a configuration type, i.e. AuthServiceConfiguration
• Create a type such as AuthServiceConfigurationLoader (or w/e you want to name it) and a concrete implementation that encapsulates reading the ENV variables and looking up the values from SSM to return an instance of AuthServiceConfiguration.
• Only reference AuthServiceConfiguration
• Inject AuthServiceConfigurationLoader where you can
• Then the devs can also use that interface to create a Mock/Stub/Local version of the AuthServiceConfigurationLoader to read from other param stores
• We may not always use SSM to load the variables, so decouple things where you can.