Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework Axiom Cloudwatch Forwarder #73

Closed
wants to merge 80 commits into from

Conversation

dasfmi
Copy link
Collaborator

@dasfmi dasfmi commented Jul 26, 2024

This PR revises the naming of CloudFormation stack elements to make their purpose clearer, and allows greater flexibility in filtering log groups that are applied as subscription filters to the (newly named) Lambda Forwarder, now using a combination of names, prefix and regular expression. It also introduces versioning to the CloudFormation stacks so breaking changes do not impact users, and introduces a new Unsubscriber stack for convenience.

TODO:

  • Renames the Ingester to Forwarder, Backfiller to Subscriber and the LogsSubscriber to LogGroupsListener.
  • Moves the stack templates to their own folder to improve readability with shorter file names
  • Moves out the responsibility of creating log group subscription filters and gives it to the Subscriber component
  • Subscriber component to handle different inputs for log groups selection (names list, prefix or regex pattern)
  • Create Unsubscriber component
  • Versioning the released stack templates
  • Prefix the created log groups names with /aws/axiom to easily discard them from being forwarded

dasfmi and others added 30 commits July 16, 2024 18:52
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
Co-authored-by: Dominic Chapman <[email protected]>
dasfmi and others added 27 commits July 19, 2024 18:27
…r-log-group

Create permission statement for each log group
Co-authored-by: a-khaledf <[email protected]>
Migrate to using `*` instead of log group names in Forwarder permissions
as the individual permissions can hit the AWS lambda permission size of
20KB. Trying to solve this will lead to complicate the setup and hurt
the DX badly.
@dasfmi dasfmi mentioned this pull request Jul 26, 2024
9 tasks
@dasfmi dasfmi changed the title Islam/axiom 4385 revert to astrek permissions Rework Axiom Cloudwatch Forwarder Jul 26, 2024
@dasfmi dasfmi closed this Jul 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants