Skip to content

Latest commit

 

History

History
285 lines (189 loc) · 10.8 KB

README.md

File metadata and controls

285 lines (189 loc) · 10.8 KB

warning This document applies to the HEAD of the calico-containers source tree.

View the calico-containers documentation for the latest release here.

Calico Networking with rkt

This tutorial describes how to set up a Calico cluster in a pure rkt environment. rkt is used for running both the Calico components and the workloads.

1. Environment setup

This tutorial walks through getting a cluster set up with Vagrant.

1.1 Install dependencies

1.2 Clone this project

git clone https://github.com/projectcalico/calico-containers.git

1.3 Startup and SSH

Change into the directory for this guide:

cd calico-containers/docs/cni/rkt

Run

vagrant up

To connect to your servers

1.4 Verify environment

You should now have two CoreOS servers. The servers are named calico-01 and calico-02 and have IP addresses 172.18.18.101 and 172.18.18.102.

At this point, it's worth checking that your servers can ping each other.

From calico-01

ping 172.18.18.102

From calico-02

ping 172.18.18.101

If you see ping failures, the likely culprit is a problem with the VirtualBox network between the VMs. You should check that each host is connected to the same virtual network adapter in VirtualBox and rebooting the host may also help. Remember to shut down the VMs with vagrant halt before you reboot.

You should also verify each host can access etcd. The following will return an error if etcd is not available.

curl -L http://localhost:2379/version

2. Starting Calico services

Once you have your cluster up and running, start Calico on both hosts

sudo calicoctl node  --runtime=rkt

This will create a systemd unit called calico-node which runs the Calico components under rkt

You can check that it's running

$ sudo rkt list
UUID            APP     IMAGE NAME                              STATE   NETWORKS
bc13af40        node    registry-1.docker.io/calico/node:latest running

You can check the status and logs using normal systemd commands e.g. systemctl status calico-node and journalctl -u calico-node

3. Create the networks

You can configure multiple networks when using rkt. Each network is represented by a configuration file in /etc/rkt/net.d/. By default, connections to a given container are only allowed from containers on the same network. This can be changed by applying additional Calico policy.

Containers on multiple networks can be accessed by containers on each network that it is connected to.

  • The container only gets a single Calico IP address and single ethernet interface.
  • The container is associated with the Calico profiles for each of the networks.

To define a rkt network for Calico, create a configuration file in /etc/rkt/net.d/.

  • Each network should be given a unique "name". This corresponds to a "profile" in Calico.
  • To use Calico networking, specify "type": "calico"
  • To use Calico IPAM, specify "type": "calico-ipam" in the "ipam" section.

This worked example creates two rkt networks. Run these commands on both calico-01 and calico-02

cat >/etc/rkt/net.d/10-calico-backend.conf <<EOF
{
    "name": "backend",
    "type": "calico",
    "ipam": {
        "type": "calico-ipam"
    }
}
EOF

cat >/etc/rkt/net.d/10-calico-frontend.conf <<EOF
{
    "name": "frontend",
    "type": "calico",
    "ipam": {
        "type": "calico-ipam"
    }
}
EOF

4. Create services

With the networks created, let's start some services. We'll create a "frontend" and a "backend" service, one on calico-01 and the other on calico-02 Both "services" will just be a httpd running in a busybox container. The service just serves up the filesystem in the container over HTTP.

On calico-01

Create the "frontend" service using systemd-run.

sudo systemd-run --unit=frontend sudo rkt run --net=frontend registry-1.docker.io/library/busybox --exec httpd -- -f -h /

Normal systemd commands can then be used to get the status of the container and view its logs (e.g. sudo journalctl -u frontend). Use rkt list to see the IP.

$ sudo rkt list
UUID		APP	IMAGE NAME					STATE	CREATED		STARTEDNETWORKS
05f8779a	node	registry-1.docker.io/calico/node:latest		running	1 hour ago	1 hour ago
c89f2f35	busybox	registry-1.docker.io/library/busybox:latest	running	2 seconds ago	1 second ago	frontend:ip4=192.168.0.0, default-restricted:ip4=172.16.28.2

We now have a busybox container running on the network frontend with an IP address of 192.168.0.0 rkt also creates a second network called default-restricted. This is used for communication with the rkt metadata service running on the host and is covered in the rkt documentation

On calico-02

Repeat for a "backend" service on calico-02

sudo systemd-run --unit=backend sudo rkt run --net=backend registry-1.docker.io/library/busybox --exec httpd -- -f -h /

$ sudo rkt list
UUID		APP	IMAGE NAME					STATE	CREATED		STARTEDNETWORKS
2cc27ce1	node	registry-1.docker.io/calico/node:latest		running	1 hour ago	1 hour ago
407208a5	busybox	registry-1.docker.io/library/busybox:latest	running	11 seconds ago	10 seconds ago	backend:ip4=192.168.0.64, default-restricted:ip4=172.16.28.2

We now have a busybox container running on the network backend with an IP address of 192.168.0.1

5. Validate access to services

Now that we've created the services and we know their IP addresses, we can access them using wget from containers running on either host, as long as they are created on the same network.

e.g. On calico-02 use wget to access the frontend service which is running on calico-01

sudo rkt run --net=frontend registry-1.docker.io/library/busybox --exec=/bin/wget -- -T 10 192.168.0.0/etc/passwd 2>/dev/null

Expected output

[62032.807862] wget[4]: Connecting to 192.168.0.0 (192.168.0.0:80)
[62032.813662] wget[4]: passwd               100% |*******************************|   334   0:00:00 ETA

This command runs the wget command in a busybox container to fetch the passwd file from our host. '-T 1' tells wget to only wait a second for a response. Stderr is redirected to /dev/null as we're not interested in the logs from rkt for this command.

You can repeat this command on calico-01 and check that access works the same from any server in your cluster.

5.1 Checking network isolation

Repeat the above command but try to access the backend from the frontend. Because we've not allowed access between these networks, the command will fail.

sudo rkt run --net=backend registry-1.docker.io/library/busybox --exec=/bin/wget -- -T 2 192.168.0.0/etc/passwd 2>/dev/null

Expected output

[62128.109283] wget[4]: Connecting to 192.168.0.0 (192.168.0.0:80)
[62129.109472] wget[4]: wget: download timed out

Calico always allows access to the containers running on a host from that host only. The rules are bypassed in this case only. This means that the backend service can be accessed directly from calico-02 (the host it's running on), but not from calico-01

6. Add network policy

To view the existing network policy, use the calicoctl command.

On either host, run

calicoctl profile backend rule show

Expected output

Inbound rules:
	 1 allow from tag backend
Outbound rules:
	 1 allow

The "frontend" profile produces a similar result.

Open access to backends

We want the backends to allow inbound traffic from the frontends, but only on port 80.

Run

calicoctl profile backend rule add inbound allow tcp from tag frontend to ports 80
calicoctl profile backend rule show

To produce the following output

Inbound rules:
	 1 allow from tag backend
	 2 allow tcp from tag frontend to ports 80
Outbound rules:
	 1 allow

And we can now access our backend service from the frontend containers.

On either host, run

sudo rkt run --net=frontend registry-1.docker.io/library/busybox --exec=/bin/wget -- -T 10 192.168.0.64/etc/passwd 2>/dev/null

Open access to frontends

We want to allow everyone to access our frontends, but only on port 80.

calicoctl profile frontend rule add inbound allow tcp to ports 80
calicoctl profile frontend rule show

To produce the following output

calicoctl profile frontend rule show
Inbound rules:
	 1 allow from tag frontend
	 2 allow tcp to ports 80
Outbound rules:
	 1 allow

Now on either host, we can access the container directly

wget -T 10 192.168.0.0/etc/passwd

7. Resetting/Cleanup up

If you want to start again from the beginning, then run the following commands on both hosts to ensure that all the rkt containers and systemd jobs are removed.

# Stop the frontend (in case the job failed, manually remove the service too)
sudo systemctl stop frontend; sudo rm -rf /run/systemd/system/frontend.service /run/systemd/system/frontend.service.d; sudo systemctl daemon-reload

# Stop the backend (in case the job failed, manually remove the service too)
sudo systemctl stop backend; sudo rm -rf /run/systemd/system/backend.service /run/systemd/system/backend.service.d; sudo systemctl daemon-reload

# Stop the calico-node (in case the job failed, manually remove the service too)
sudo systemctl stop calico-node; sudo rm -rf /run/systemd/system/calico-node.service /run/systemd/system/calico-node.service.d; sudo systemctl daemon-reload

# Remove any stopped busybox containers
sudo rkt list --no-legend | cut -f1 |sudo xargs rkt rm

# Wipe Calico data from etcd
etcdctl rm --recursive /calico

Further reading

For more information on managing the calico/node container please refer to the rkt and systemd documentation.

For details on configuring Calico for different network topologies and to learn more about Calico under-the-covers please refer to the Further Reading section on the main documentation README.

Analytics