Skip to content

Commit

Permalink
feat: initial setup for pillarbox-monitoring data transfer service
Browse files Browse the repository at this point in the history
This project sets up a Kotlin-based Spring Boot service that connects to the Pillarbox Monitoring
Event Dispatcher SSE (Server-Sent Events) endpoint and publishes the received events to an
OpenSearch storage.

Key components include:

- **SSE Client**: Listens to the Server-Sent Events (SSE) endpoint, receiving event data in
  real-time, with retry logic in case of connection failures.
- **LockManager**: Ensures concurrency control, locking based on session ID to allow thread-safe
  operations during event processing.
- **OpenSearch Integration**: Handles index creation and retry logic, connecting to an OpenSearch
  cluster for event storage.
- **Setup Service**: Initializes the OpenSearch index and applies Index State Management (ISM)
  policies before starting the SSE client.
- **Custom Health Monitoring**: Uses Spring Boot Actuator with to monitor and report on performance,
  including method execution times.
- **GitHub Actions**: for quality checks on PRs, automated releases on pushes to main using
  semantic-release, and Docker image deployments to ECR on tag creation.

Co-authored-by: Gaëtan Muller <[email protected]>
  • Loading branch information
jboix and MGaetan89 committed Oct 8, 2024
1 parent 6a3e556 commit e66267e
Show file tree
Hide file tree
Showing 45 changed files with 2,036 additions and 0 deletions.
24 changes: 24 additions & 0 deletions .githooks/commit-msg
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
#!/bin/sh

# Define color codes
RED='\033[0;31m'
GREEN='\033[0;32m'
NC='\033[0m' # No Color

# Create a regex for a conventional commit
conventional_commit_regex="^(build|chore|ci|docs|feat|fix|perf|refactor|revert|style|test)(\([a-z \-]+\))?!?: .+$"

# Get the commit message (the parameter we're given is just the path to the temporary file which holds the message)
commit_message=$(cat "$1")

# Check if the commit message doesn't match the conventional commit regex
if ! echo "$commit_message" | grep -Eq "$conventional_commit_regex"; then
echo "${RED}The commit message does not meet the Conventional Commit standard${NC}"
echo "An example of a valid message is: "
echo " feat(login): add the 'remember me' button"
echo "More details at: https://www.conventionalcommits.org/en/v1.0.0/#summary"
exit 1
fi

# If the commit message is correct
echo "${GREEN}Commit message meets Conventional Commit standards...${NC}"
29 changes: 29 additions & 0 deletions .githooks/pre-commit
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
#!/bin/sh

# Define color codes
RED='\033[0;31m'
YELLOW='\033[0;33m'
GREEN='\033[0;32m'
NC='\033[0m' # No Color


# Script to run code quality checks
echo "${YELLOW}Running ktlintCheck...${NC}"
if ! ./gradlew ktlintCheck; then
echo "${RED}ktlintCheck failed ✘${NC}"
echo "${RED}Commit aborted.${NC}"
exit 1
fi

echo "${GREEN}ktlintCheck passed ✔${NC}"

echo "${YELLOW}Running detekt...${NC}"
if ! ./gradlew detekt; then
echo "${RED}detekt failed ✘${NC}"
echo "${RED}Commit aborted.${NC}"
exit 1
fi

echo "${GREEN}detekt passed ✔${NC}"

exit 0
16 changes: 16 additions & 0 deletions .githooks/pre-push
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/bin/sh
# Pre-push Git hook script to run code quality checks

# Define color codes
RED='\033[0;31m'
YELLOW='\033[0;33m'
NC='\033[0m' # No Color

echo "${YELLOW}Running tests...${NC}"

# Check if tests was successful
if ! ./gradlew test; then
echo "${RED}There are failing tests ✘${NC}"
echo "${RED}Push aborted.${NC}"
exit 1
fi
21 changes: 21 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
## Description

<!--
Please provide a brief summary of the changes made. Please explain why
this change was necessary. Was there a problem or an issue this change
will address? What will be improved with this change?
-->

## Changes Made

<!--
Please detail the modifications made. This could include areas such as
code, documentation, structure, or formatting.
-->

## Checklist

- [ ] I have followed the project's style and contribution guidelines.
- [ ] I have performed a self-review of my own changes.
- [ ] I have made corresponding changes to the documentation.
- [ ] I have added tests that prove my fix is effective or that my feature works.
32 changes: 32 additions & 0 deletions .github/workflows/deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
name: Publish Production Image

on:
push:
tags:
- '*'

jobs:
deploy-dev:
runs-on: ubuntu-latest
env:
ECR_REGISTRY: ${{ secrets.AWS_ACCOUNT_ID }}.dkr.ecr.${{ secrets.AWS_REGION }}.amazonaws.com

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: configure AWS credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.GH_ROLE }}
aws-region: ${{ secrets.AWS_REGION }}

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v2

- name: Build and Push Image
uses: docker/build-push-action@v4
with:
push: true
tags: ${{ env.ECR_REGISTRY }}/pillarbox-monitoring-dispatch:${{ github.ref_name }}
25 changes: 25 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
name: Quality

on:
merge_group:
pull_request:
branches: [ main ]

jobs:
quality:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up JDK
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '22'

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4

- name: Run Build
run: ./gradlew build
47 changes: 47 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Release

on:
push:
branches: [ main ]

jobs:
release:
runs-on: ubuntu-latest

permissions:
contents: write

steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0 # Needed for semantic-release

- name: Set up JDK
uses: actions/setup-java@v4
with:
distribution: 'temurin'
java-version: '22'

- name: Setup Gradle
uses: gradle/actions/setup-gradle@v4

- name: Set up Node.js
uses: actions/setup-node@v4
with:
node-version: '22'

- name: Install semantic-release
run: npm install -g @semantic-release/[email protected] \
@semantic-release/[email protected] \
@semantic-release/[email protected] \
@semantic-release/[email protected] \
@semantic-release/[email protected] \
@semantic-release/[email protected] \
[email protected] \
[email protected]

- name: Release
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: npx semantic-release
110 changes: 110 additions & 0 deletions .releaserc
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
{
"branches": ["main"],
"plugins": [
"@semantic-release/commit-analyzer",
[
"@semantic-release/release-notes-generator",
{
"preset": "conventionalcommits",
"parserOpts": {
"noteKeywords": [
"BREAKING CHANGE",
"BREAKING CHANGES",
"BREAKING"
]
},
"presetConfig": {
"types": [
{
"type": "breaking",
"section": "Breaking Changes ❗",
"hidden": false
},
{
"type": "feat",
"section": "New Features 🚀",
"hidden": false
},
{
"type": "fix",
"section": "Enhancements and Bug Fixes 🐛",
"hidden": false
},
{
"type": "docs",
"section": "Docs 📖",
"hidden": false
},
{
"type": "style",
"section": "Styles 🎨",
"hidden": false
},
{
"type": "refactor",
"section": "Refactor 🔩",
"hidden": false
},
{
"type": "perf",
"section": "Performances ⚡️",
"hidden": false
},
{
"type": "test",
"section": "Tests ✅",
"hidden": false
},
{
"type": "ci",
"section": "CI 🔁",
"hidden": false
},
{
"type": "chore",
"section": "Chore 🧹",
"hidden": false
}
]
},
"writerOpts": {
"groupBy": "type",
"commitGroupsSort": [
"breaking",
"feat",
"fix"
]
}
}
],
[
"@semantic-release/changelog",
{
"changelogFile": "CHANGELOG.md"
}
],
[
"@semantic-release/exec",
{
"prepareCmd": "./gradlew release -Pversion=${nextRelease.version}"
}
],
[
"@semantic-release/git",
{
"assets": ["build.gradle.kts", "gradle.properties", "CHANGELOG.md"],
"message": "chore(release): ${nextRelease.version} [skip ci]\n\n${nextRelease.notes}"
}
],
[
"@semantic-release/github",
{
"assets": [
{
"path": "build/libs/pillarbox-monitoring-transfer.jar"
}
]
}
]
]
}
21 changes: 21 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
# First stage: Build native image with GraalVM
FROM gradle:8.10-jdk22-alpine AS build

WORKDIR /app
COPY build.properties .
COPY build.gradle.kts .
COPY src ./src
RUN gradle clean build -x test

# Second stage: Slim runtime image with optimized settings
FROM eclipse-temurin:22-jre-alpine
VOLUME /tmp

# Copy the built JAR file from the build stage
COPY --from=build /app/build/libs/pillarbox-monitoring-transfer.jar app.jar

# Extract layered JAR content for better image caching and performance
RUN java -Djarmode=layertools -jar app.jar extract

# JVM memory and garbage collection optimizations for containers
ENTRYPOINT ["java", "-Dsun.net.inetaddr.ttl=60", "-Dsun.net.inetaddr.negative.ttl=10", "-Xms512m", "-Xmx1024m", "-XX:+UseG1GC", "-XX:MaxGCPauseMillis=200", "-XX:+UseContainerSupport", "-jar", "/app.jar"]
21 changes: 21 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
MIT License

Copyright (c) 2024 SRG SSR

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Loading

0 comments on commit e66267e

Please sign in to comment.