Point rest client at docker compose dev service

[LarsSven]

Description

We have a setup where we use the Quarkus compose dev service in dev/test mode, which, among other services, spins up a service that has an HTTP frontend, which our Quarkus application talks to. Currently, we have our configuration set up as following:

quarkus:
  rest-client:
    assignments-api:
      url: http://localhost:8081

However this obviously starts failing once the docker container that is spun up by the compose service no longer has localhost as its host (like in a CI for example).

How can we make the URL host that the rest client points to when using docker compose dev services change dynamically based on where the docker compose container is spawned? Just like how it currently works for Quarkus extensions where the host of things like kafka/mysql dynamically are configured depending on the host and port of the dev service container.

Implementation ideas

It would be nice if there was some integration such that we can link a rest client endpoint to a dev service (for us specifically a docker compose one), such that the host is correct when in test/dev mode.

[quarkus-bot]

/cc @cescoffier (rest-client), @edeandrea (container-image), @geoand (container-image,devservices,rest-client), @holly-cummins (devservices), @ozangunalp (devservices)

[geoand]

I like the idea, but I have the following questions:

• How would we know which started services need to have the host and port "captured" for use by other parts in Quarkus?
• How would we know which REST client is supposed to use the "captured" information?

[ozangunalp]

So you'd need the container runtime host and the mapped port of the compose service?

For the first, you can expose the container runtime host as an env var, say DOCKER_HOST.
For the second, you can use this compose dev service feature to expose the mapped port of a service to the application: https://quarkus.io/guides/compose-dev-services#mapping-exposed-ports

[ozangunalp]

If you defined the service you want to access with the label :

labels:
      io.quarkus.devservices.compose.config_map.port.8080: assignments-api.port

You can configure the app with the test profile

%test.quarkus.rest-client.assignments-api.url: http://${docker.host}:${assignments-api.port}

[LarsSven]

How would we know which started services need to have the host and port "captured" for use by other parts in Quarkus?
How would we know which REST client is supposed to use the "captured" information?

I feel like one way you can see this is that the rest client extension has a dev service, just like kafka has a kafka dev service. Because we often run into the issue where our rest clients need to point at a specific service in our tests/dev mode. I think the best way to deal with this is not necessarily just look at docker compose, but look at it from the perspective of a normal dev service, and then just like other dev services for docker compose it looks at the name/port of the compose service.

We also don't mind just giving the name of the service within compose.

For the first, you can expose the container runtime host as an env var, say DOCKER_HOST.

Sure, but how would I then integrate that with the quarkus.rest-client.*.url property? In the end I can't programmatically set that at runtime, so this would need to be somehow integrated with the Quarkus config right?

%test.quarkus.rest-client.assignments-api.url: http://${docker.host}:${assignments-api.port}

Ah I didn't know I can use DOCKER_HOST within the Quarkus config, that may actually be very useful, though I'd need to experiment with it to see if that would solve the problem.

[LarsSven]

I will have a look on monday about whether I can oslve my specific usecase with the docker.host approach. Thanks for the information.

[geoand]

I think the best way to deal with this is not necessarily just look at docker compose, but look at it from the perspective of a normal dev service, and then just like other dev services for docker compose it looks at the name/port of the compose service.

Agreed, and that has been brought up in the past, but we (as in everyone interested) have never come up with an adequate model for this.

[LarsSven]

@ozangunalp I tried to do the following, but this does not seem to work:

quarkus:
  rest-client:
    assignments-api:
      url: http://${docker.host}:8081

It doesn't seem like docker.host is available in the Quarkus config?

[ozangunalp]

It doesn't seem like docker.host is available in the Quarkus config?

You still need to set it in the environment you are running the tests.

[LarsSven]

But that makes it very user-unfriendly. My goal is to have the same behaviour as extension dev services, where it figures out the container host without the user having to deal with it. DOCKER_HOST is indeed set in the CI, but it's not set locally, where we also want to be able to run this, ideally without the user having to manually set DOCKER_HOST.

This is the functionality I am missing, where I can get the Docker host that Quarkus is able to figure out for its own dev services within the rest config.

[LarsSven]

Also another thing that makes that setup less ideal is that even in our CI DOCKER_HOST is not set, but somehow quarkus is still able to find the address of our dev service containers (172.17.0.1, which is the default docker bridge) for its own dev services. So for the rest client, I have to manually set localhost for local development and the docker bridge for the CI, which is a lot more brittle than being able to reuse the logic Quarkus is already using to find the other docker compose dev services, since I am for example also connecting to Kafka, which I don't have to do anything for as the extension can already itself find the host of the container.