Airbyte Source Connector support with Meltano

[tayloramurphy]

A fantastic community member added support for Airbyte connectors within Meltano. This dicussion can be used to answer any questions or provide any feedback on the integration.

• tap-airbyte-wrapper on GitHub
• FAQ
• MeltanoHub
• Blog Post

[lideke]

Hi,
Did someone succed to use tap-airbyte-wrapper with docker-compose ?
I tried to with custom build with meltano and docker.io
I was able to run docker ps in the container, but when i tried with tap-sonar-cloud i've got : No such file or directory: '/tmp/config.json'

[tayloramurphy]

@lideke you shouldn't need to directly use tap-airbyte-wrapper. Just reference the individual connector you're trying to use in your Meltano configuration. If you have more trouble can you open an issue in the main meltano project and I can bring more folks there (or ping us on Slack!)

[alexmaras]

I had this issue using meltano in docker and trying to use docker-in-docker for the airbyte wrapper - the trick is to mount the /tmp directory on your host to the /tmp directory on the meltano image. Meltano then spins up a docker container, which runs the container on the host, so the mappings need to be passed through so that /tmp is in the same location for all three - the host, the meltano container, and the airbyte container.

The wrapper puts the config.json generated from the meltano.yml into a /tmp directory created dynamically, so this ensures the airbyte container knows where to find /tmp/config.json

[liornaimalon]

@alexmaras can you provide more details how can this be achieved in github actions ? when i try to add a docker_mount configuration with /tmp target i receive :
docker: Error response from daemon: Duplicate mount point: /tmp.

[tayloramurphy]

@edgarrmondragon can you chime in here when you get a moment?