Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add deploy logic for Thanos receive ingest component #13

Merged
merged 13 commits into from
Jun 17, 2024

Conversation

philipgough
Copy link
Collaborator

@philipgough philipgough commented Jun 17, 2024

The following PR adds code for the ingester components only in the split router/ingester model.
To complete the Thanos receive work, I will split the work into two separate PRs after this one merges to make things easier to review.

  1. Ingest components
  2. Hashring management
  3. Router components

@philipgough philipgough force-pushed the receive-dev branch 7 times, most recently from d76e930 to b55c4a1 Compare June 17, 2024 13:13
@philipgough philipgough force-pushed the receive-dev branch 2 times, most recently from 6bf64fb to 820fe4c Compare June 17, 2024 16:57
@philipgough philipgough marked this pull request as ready for review June 17, 2024 16:57
@philipgough philipgough changed the title draft Thanos receive ingest components Jun 17, 2024
Copy link
Member

@saswatamcode saswatamcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks a lot! Generally LGTM, added a few small nits.

I'll try to review a bit deeper again later

@@ -29,13 +29,13 @@ type RouterSpec struct {
// Replicas is the number of router replicas.
// +kubebuilder:validation:Minimum=1
// +kubebuilder:default=1
// +kubebuilder:validation:Optional
Replicas *int32 `json:"replicas,omitempty"`
// +kubebuilder:validation:Required
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need both default and required?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it works in terms of the openapi spec tbh

}

// buildController sets up the controller with the Manager.
func (r *ThanosReceiveReconciler) buildController(bld builder.Builder) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this need to be a separate method? I don't think we can reuse/test this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as of now no, but will be adding watch logic etc to endpointslices here so the function will grow.


objs = append(objs, r.buildHashrings(receive)...)

var errCount int32
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be better to wrap errors here instead of just count? Or use merrors?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think it might become unwieldy - we already log on each of the failed requests but we will log the count at the top level. if anything we could drop it entirely but i think its a useful summary of the reconciliation loop?

}

func (r *ThanosReceiveReconciler) handleDeletionTimestamp(log logr.Logger, receiveHashring *monitoringthanosiov1alpha1.ThanosReceive) (ctrl.Result, error) {
if controllerutil.ContainsFinalizer(receiveHashring, receiveFinalizer) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When do we add the finalizer? I don't think this would fire without adding as per https://book.kubebuilder.io/reference/using-finalizers

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't add it yet so this is a no-op right now indeed - will need to add logic to clean up orphaned resources etc

internal/controller/thanosreceive_controller.go Outdated Show resolved Hide resolved
internal/pkg/manifests/options.go Show resolved Hide resolved
// Endpoint represents a single logical member of a hashring.
type Endpoint struct {
// Address is the address of the endpoint.
Address string `json:"address"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wonder if we can add some validation to these address fields

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, will take care of that in the follow up when we do the hashring work. just wanted to port over the thanos types here for visibility

internal/pkg/manifests/receive/builder.go Outdated Show resolved Hide resolved
test/utils/utils.go Show resolved Hide resolved
internal/pkg/k8s/check.go Outdated Show resolved Hide resolved
@philipgough philipgough changed the title Thanos receive ingest components Add deploy logic for Thanos receive ingest component Jun 17, 2024
Copy link
Member

@saswatamcode saswatamcode left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Let's merge this and iterate! 🚀

@saswatamcode saswatamcode merged commit 96f263c into thanos-community:main Jun 17, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants