Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: VM Creation with publicIP are not well retrived during cloud-init boot #419

Open
sebglon opened this issue Jan 29, 2025 · 4 comments · May be fixed by #427
Open

[Bug]: VM Creation with publicIP are not well retrived during cloud-init boot #419

sebglon opened this issue Jan 29, 2025 · 4 comments · May be fixed by #427
Labels
bug Something isn't working

Comments

@sebglon
Copy link
Contributor

sebglon commented Jan 29, 2025

What happened

During creation of our VM with public IP, we need to use the publicIP available from the metadata server to inject data during the kubeAdm join process.
But sometime the publicIp returned by the metadata server during the boot is set to ok

Step to reproduce

Step to reproduce:

  1. create the KubeAdmTemplate
---
apiVersion: bootstrap.cluster.x-k8s.io/v1beta1
kind: KubeadmConfigTemplate
metadata:
  name: worker-data
  namespace: test
  labels:
    cluster.x-k8s.io/watch-filter: test
spec:
  template:
    spec:
      joinConfiguration:
        nodeRegistration:
          name: '{{ ds.meta_data.local_hostname }}'
          kubeletExtraArgs:
            config: /etc/kubernetes/kubelet-config.yaml
            cloud-provider: external
            provider-id: aws:///'{{ ds.meta_data.placement.availability_zone }}'/'{{ ds.meta_data.instance_id }}'
            node-labels: "worker-type=load-balancer-internal,public-ip={{ ds.meta_data.public_ipv4 }}"
            register-with-taints: "dedicated=data:NoExecute"
...
  1. Deploy the OscMachineTemplate with spec.template.spec.node.vm.publicIp=true and MachineDeployment associated to the previous KubeAdmTemplate

Sometime the VM is well created and the node-labels.public-ip is well provisioned with the public IP.
But sometime the node-labels.public-ip is set to OK.

Expected to happen

The publicIP is available at the VM creation and the metadata server return it.

Add anything

After some investigation this issue is related to publicIP link with the VM that is coming to late.

To fix this we can create a NIC and attach them the publicIp, and provide this NIC at the VM creation.

cluster-api output

Environment

- Kubernetes version: (use `kubectl version`): 
- OS (e.g. from `/etc/os-release`):
- Kernel (e.g. `uname -a`):
- cluster-api-provider-outscale version:
- cluster-api version: 
- Install tools:
- Kubernetes Distribution:
- Kubernetes Diestribution version:
@sebglon sebglon added the bug Something isn't working label Jan 29, 2025
@jfbus
Copy link
Contributor

jfbus commented Feb 3, 2025

Another way to ensure that a public IP is attached to a VM at boot is to use the osc.fcu.eip.auto-attach tag (https://docs.outscale.com/en/userguide/About-Public-IPs.html#_in_the_public_cloud).

It is probably simpler to add the tag rather than creating nics (unless you have a specific requirement I missed, please provide more details if that is the case)... I'll look into it.

@pierreozoux
Copy link
Contributor

I think this relates to #377

@jfbus the way I understand osc.fcu.eip.auto-attach is that you specify the nic you want.

So if you want 5 VMs, with 5 different IPs, then you need to create 5 different machines with the different tag.

I think osc.fcu.eip.auto-attach is more of a hack than a proper solution. For instance, we'll never be able to get autoscaling with osc.fcu.eip.auto-attach.

@jfbus
Copy link
Contributor

jfbus commented Feb 4, 2025

There are two issues:

  • how do we manage autoscaling with public ips ?
  • how do we ensure that publics ips are static and linked at boot ?

The tag definitively only solves the second issue, and we can discuss autoscaling requirements in #377

@sebglon
Copy link
Contributor Author

sebglon commented Feb 4, 2025

@pierreozoux we can dynamicaly add the osc.fcu.eip.auto-attach tag at VM creation. i create an new PR to test this scenario

@sebglon sebglon linked a pull request Feb 4, 2025 that will close this issue
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Development

Successfully merging a pull request may close this issue.

3 participants