localhost vs 127.0.0.1 in docker-compose

[jinserk]

Hi,

I have a project with fastapi and fief altogether in docker-compose as different containers (named api and fief respectively), serving them in the exposed ports 8001 and 8002 respectively. I followed the instruction described in here and checked the auth successful in http://localhost:8001/docs . However, when I try /user GET api, I've got an error that it have tried to connect 127.0.0.1:8002 -- which is fief api calls. I gave the args of FiefAsyc as:

fief = FiefAsync(
    "http://localhost:8002",
    "YOUR_CLIENT_ID",
    "YOUR_CLIENT_SECRET",
)

but it looks like fastapi or auth = FiefAuth(fief, scheme) changed the localhost to 127.0.0.1. This prevents the url to go to the container fief via host, but redirect back to the api container itself inside the docker compose env. Of course I set the env vars listed in here. Do you have any idea how to resolve this?

[frankie567]

Hi @jinserk 👋

Could you share your docker-compose configuration and the error you're getting? That would help debug the issue :)

I think the problems comes from the fact that api container tries to call the fief container through localhost. But in a Docker context, you should probably call it through the fief host:

fief = FiefAsync(
    "http://fief:8002",
    "YOUR_CLIENT_ID",
    "YOUR_CLIENT_SECRET",
)

But I fear that'll break the redirection when trying to authenticate through your local machine 🤔

[jinserk]

Thank you again, @frankie567! I had to look into the code in fief-python. Now I clearly understand the dependencies. The web service works as expected after modifying the code as you explained. However, I repeatedly get the 401 Error: unauthroized error when I logged in using the Authorize button in docs, and try the api /auth/user. This doesn't look llike just related to the dependencies.

[frankie567]

Hmm, that's weird 🤔 Could you copy/paste one of those access token you're getting?

[jinserk]

This is the access token I got from the "Authorize" button login:

{"access_token":"eyJhbGciOiJSUzI1NiJ9.eyJhdWQiOlsiVE1DdllUTXVzWmhUSjd6UFlMMVF0dUtCWkhlZ05hWU1hNENhWHktVWtYZyJdLCJhenAiOiJUTUN2WVRNdXNaaFRKN3pQWUwxUXR1S0JaSGVnTmFZTWE0Q2FYeS1Va1hnIiwiZXhwIjoxNjU1OTE3NTIyLCJpYXQiOjE2NTU5MTM5MjIsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0OjgwMDIiLCJzY29wZSI6Im9wZW5pZCBvZmZsaW5lX2FjY2VzcyIsInN1YiI6Ijc0ZTg4MjdiLTFhMDctNGI4Ny04ZGUxLTZmMmYxY2UzZDYwNCJ9.VEMitAepXzSE0ZnzoYX_myy-d1KTRuxnuN0fs2f5troxSS-9U6Xlm-gyNkmY3sOvJjbsnPEA8PAhY-Ojy43Mvkc11KlEHK8ULs9Y_yzqNcX86I4xNVw_mNCLH9asn7RorreiHYEBANX4CnArSUdDEPk8BXdLO1CWbSSzb5DKItz8DuszQ_-sRoP42kvsXpaDHnFH1J7DqsaZZjRz71EMI_CVYSFjtiCirai1nPEfCgGwGDAaIQQECmfQTVYiyOPkWQxnJq-DjrNJsqelfbgYEP23w6lNRgnIqt8ERySjqLeH8JBny1CeWOqM-aJ-mD5H2mdIRq4HZdGwj6yjz8abgi5ky-OGgc-vB6Djqo5v8BWMocvaD_BjDNEJ4eVW6DmYeGMf62xiq9FTsPvHDwR7lh_yDvfxBoysQ3EkeAb4FthU0x6IOWzN2GLRmd9VUo8ihnuj7SNZoGFUryvEB7roG9iKF3xirX8i4eEgtx9iUPO9uKlm7OC90ME9b5BMANdynJlu1gJ4pKowTSMx1wxmnvUvXJs-qeSGRq99MJ7VjV0Ul_emldHzom0WqAz0Giv5e0OEFP2COOLs1dMD2NoWan-4eL9eOaXh8xW9Vub6DCsqf7yFP787KFpA-Zd-CkJQP3RqksV5e53e_E4bYTWv0GFPabP7oDnOFMMJxKfd9zI","id_token":"eyJhbGciOiJSUzI1NiJ9.eyJhdF9oYXNoIjoiT3pITEZhVGxDa0FweHh6TElxN2lvZyIsImF1ZCI6WyJUTUN2WVRNdXNaaFRKN3pQWUwxUXR1S0JaSGVnTmFZTWE0Q2FYeS1Va1hnIl0sImF1dGhfdGltZSI6MTY1NTQxMDE3NCwiYXpwIjoiVE1DdllUTXVzWmhUSjd6UFlMMVF0dUtCWkhlZ05hWU1hNENhWHktVWtYZyIsImNfaGFzaCI6Iko1Vjl1bmlOa0ozVWd4bkZwQlVmTHciLCJlbWFpbCI6ImppbnNlcmsuYmFpa0BnbWFpbC5jb20iLCJleHAiOjE2NTU5MTc1MjIsImZpZWxkcyI6e30sImlhdCI6MTY1NTkxMzkyMiwiaXNfYWN0aXZlIjp0cnVlLCJpc19zdXBlcnVzZXIiOmZhbHNlLCJpc192ZXJpZmllZCI6ZmFsc2UsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0OjgwMDIiLCJzdWIiOiI3NGU4ODI3Yi0xYTA3LTRiODctOGRlMS02ZjJmMWNlM2Q2MDQiLCJ0ZW5hbnRfaWQiOiIyZDI1OTIyYS0yNDdlLTQyMGMtOGNhYy1kNDBkOTk3NDZiOGQifQ.RQlfpyqG3BtyYqBo0KrDSklDssupweuBvysLDcUJ4hOdcHKowYpSO_Uu-QZzObgolTY2WxBWtihUozyemST8r4AItYAa-nUFJD3SzPGCN9oSBTRVnKw3vRJcOT3sNH3GixNX5QWl1f8zPJgS0uW2MEJHC_Ix8_dlfXh4uA9acunWGeHQ3qUJY5G1O27I7FEoOQExRyste-G_DjmLf7vM4ZvIy0lD_hwEbpmy7zQPxjF77xRwa7JOfMrHKzMHvRDDdS5908jAv1nQkv36FF9Nf3Sk6TJ6GA8BvpXQSmOysaimwLCRq0j-T1KFBL5Bzp8p51f4Brii2yzMCaEhL16-uJ6XBMUQ6ZhW3f_tgoe4_ttDYFzA9-RFYGC-0N-UJkBDY5Bt4bvFHXiS141e93t_RBFEwdNQzHIu4Z6FsOlf-W2hiu8xM1TUS5Te-SYHf3JFkniIXRDS38kMudi0n7LNdsrtvfHhwuKdrav6k7vh1-ICZ7BTeoUmAlAaULWCNPEIUm35L2vdPimV4NdyTg3WZJX6VjaQwtgvG-cUnOBr2w9XrCRsybHuMU6jdLT6m5Jmb-s-asnaSWW4HP3hx9ir20QU4NlXfSVxsy1QLS-Ro_j0wc_czeKPTXDKenYn1iixD-jaQRAH2SpYfUOzoS5jzcTjdnaYGS_fAgyUyyrhVqY","token_type":"bearer","expires_in":3600,"refresh_token":"WNF_bQSfwBazQDpajJB9y3jLVX8aR8Y7LSMXirW1NEg"}

This is the curl cmd to try the /auth/user api:

curl -X 'GET' \
  'http://localhost:8001/v1/auth/user' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer eyJhbGciOiJSUzI1NiJ9.eyJhdWQiOlsiVE1DdllUTXVzWmhUSjd6UFlMMVF0dUtCWkhlZ05hWU1hNENhWHktVWtYZyJdLCJhenAiOiJUTUN2WVRNdXNaaFRKN3pQWUwxUXR1S0JaSGVnTmFZTWE0Q2FYeS1Va1hnIiwiZXhwIjoxNjU1OTE3NTIyLCJpYXQiOjE2NTU5MTM5MjIsImlzcyI6Imh0dHBzOi8vbG9jYWxob3N0OjgwMDIiLCJzY29wZSI6Im9wZW5pZCBvZmZsaW5lX2FjY2VzcyIsInN1YiI6Ijc0ZTg4MjdiLTFhMDctNGI4Ny04ZGUxLTZmMmYxY2UzZDYwNCJ9.VEMitAepXzSE0ZnzoYX_myy-d1KTRuxnuN0fs2f5troxSS-9U6Xlm-gyNkmY3sOvJjbsnPEA8PAhY-Ojy43Mvkc11KlEHK8ULs9Y_yzqNcX86I4xNVw_mNCLH9asn7RorreiHYEBANX4CnArSUdDEPk8BXdLO1CWbSSzb5DKItz8DuszQ_-sRoP42kvsXpaDHnFH1J7DqsaZZjRz71EMI_CVYSFjtiCirai1nPEfCgGwGDAaIQQECmfQTVYiyOPkWQxnJq-DjrNJsqelfbgYEP23w6lNRgnIqt8ERySjqLeH8JBny1CeWOqM-aJ-mD5H2mdIRq4HZdGwj6yjz8abgi5ky-OGgc-vB6Djqo5v8BWMocvaD_BjDNEJ4eVW6DmYeGMf62xiq9FTsPvHDwR7lh_yDvfxBoysQ3EkeAb4FthU0x6IOWzN2GLRmd9VUo8ihnuj7SNZoGFUryvEB7roG9iKF3xirX8i4eEgtx9iUPO9uKlm7OC90ME9b5BMANdynJlu1gJ4pKowTSMx1wxmnvUvXJs-qeSGRq99MJ7VjV0Ul_emldHzom0WqAz0Giv5e0OEFP2COOLs1dMD2NoWan-4eL9eOaXh8xW9Vub6DCsqf7yFP787KFpA-Zd-CkJQP3RqksV5e53e_E4bYTWv0GFPabP7oDnOFMMJxKfd9zI'

and the result:

{"detail":"Unauthorized"}

[frankie567]

Ok, I think I understand what's going on 😄 You probably use a previous version of the Fief server which didn't set the permissions field in the access token. The Fief client is designed to reject access token which don't expose that field.

To solve this, you just need to update the Fief image and restart your containers:

docker pull ghcr.io/fief-dev/fief:latest

[jinserk]

Finally I made it work! Thank you so much @frankie567, I really appreciate it.
In short, we need

• maintain the latest fief-dev/fief image to support `permissions field
• network_mode: host for Docker compose env

It was a long discussion. I'm so sorry for bothering you a lot. I will try to contribute to the fief community too!

[frankie567]

Hey @jinserk!

I've had a second thought about this routing problem between containers, and following a suggestion from the great Samuel Colvin, I may have come with a good solution.

You're maybe familiar with ngrok, it's a great tool to quickly "put localhost on the internet". Said another way, you get an internet-enabled URL that gets routed to your machine. So we'll use this to enable communication between fief and backend.

For this to work, you'll need to pull Fief latest version, 0.16.2.

Then, start a ngrok instance to route the Fief container, i.e., the port 8002:

ngrok http --host-header localhost:8002 8002

Notice that we add the --host-header option so that Fief still receives the right host to identify your workspace.

Ngrok will output you an URL in the form https://2e44-2a02-8429-80b2-1401-14f1-2b34-8bb-df37.eu.ngrok.io. Use it as the base URL for you Fief client:

fief = FiefAsync(
    "https://2e44-2a02-8429-80b2-1401-14f1-2b34-8bb-df37.eu.ngrok.io",
    "YOUR_CLIENT_ID",
    "YOUR_CLIENT_SECRET",
)

And everything should work! If you restart Ngrok, you'll get a new address. Don't forget to update it afterwards 😉

I would really like to have your feedback on this approach and if you find it viable and practical. If so, this is something I would like to add to the docs.

PS: I'll also answer you on your access token problem above.

[jinserk]

Thank you again for the nice suggestion. I met the keyword ngrok during googling localhost related search, bud didn't know what it is. It looks like a charm reverse proxy service worth to know. Thank you for letting me know.
By the way, I really wonder the fundamental reason why fief_client cannot make a proper connection to the localhost. At first I thought that it is originated from the internal forwarding to 127.0.0.1, which makes the query loop back to my API server not to the Fief server on the other container. But I found that FastAPI sometimes forwards it to ::1 of IPv6 loopback. I just wonder if there is a strict way to use localhost without change in FastAPI or Fief client. I've tried to set host args in FiefAsync client constructor but it doesn't change anything. I also checked some docs from Internet mentioned TLD (Top level Domain) allowance. What's the origin of this issue -- FastAPI, httpx, or others?

[frankie567]

I'm not sure I follow you completely there. The reason why fief_client is not able to reach localhost is because of the nature of containers. You can consider containers as two separate virtual machines, each one with their own OS and network (— it's an overly simplification, but you get the idea —).

If you are inside backend container, localhost is itself, not the other container. On the backend container, nothing runs on the port 8002; so the request fails. Actually, backend container could even start a process on port 8002, it would not conflict with the Fief container; they are isolated.

A good exercice to feel this is to run a bash command line inside your container, e.g. docker compose run backend sh: you'll have a complete command line in a virtual system, isolated from your computer and other containers, with its own processes and files. A bit like an SSH connection on a remote server.

With Docker Compose, by default, every containers you define can communicate with each other by their alias. So, from backend, you can make a request to http://fief:8002 and it'll work. Think of it as private local network, where each container is a distinct machine.

However, if you define the network_mode to host, containers won't be part of this private network; they will be directly exposed to the local machine. Therefore, they can communicate with each other on localhost. Note that in this case, ports could collide.

99% of the time, you can develop micro-services with Docker Compose and make them communicate using the private Docker network without any issue.

The problem with Fief is that the client automatically discovers the URL it has to call to perform authentication (that's part of the OpenID protocol). So we have two possibilities:

• If we generate the Fief URL with the internal host, i.e. http://fief:8002, backend container will be able to call them, but the host machine (you and your browser) won't.
• If we generate the Fief URL with http://localhost:8002, you will be able to open the auth pages but the backend container won't be able to call the routes (and first and foremost, the route to discover the URL).

That's why the most straightforward solution is to have a proper "internet" hostname that's accessible both from the inside and the outside: ngrok is the perfect tool for that.

Hope it clarifies things a little 😃

[jinserk]

Yeah.. I know the inside of docker compose env. I thougtht that the relationship between the FastAPI and the Fief is exactly the same with the FastAPI and db container -- but the problem of fief hostname is that it is used for the external url and the internal url. If I use the host fief instead of localhost, it will be the problem when I click Authorize button to log in. The url is beginning with http://fief:8002 and cannot be understood in the host browser.
Is there any way to define the internal host and the external host separately?

[frankie567]

Well, actually that's more or less the purpose of the host keyword argument:

fief = FiefAsync(
    "http://fief:8002",
    "YOUR_CLIENT_ID",
    "YOUR_CLIENT_SECRET",
    host="localhost:8002"
)

Still, the problem will be the result of the /.well-known/openid-configuration route which will generate absolute URL respecting the Host header. So you'll get things like http://localhost:8000/token. Then, the Fief client won't do anything clever and just try to call this URL (and fail).

[jinserk]

Then how about wrapping FiefAuth in my FastAPI app to make every calls to the internal? In other words, can I make the urls

• Authorization URL: http://localhost:8001/authorize
• Token URL: http://localhost:8001/api/token

in my FastAPI app and call to the counterparts beginning with http://fief:8002 internally?

EDIT: It was a stupid question that /authorize needs input form for username and password. Please ignore the above question. :(