Skip to main content

Dremio

Availability
Available CoreAvailable StandardAvailable PlusAvailable ProAvailable Enterprise FlexAvailable PyAirbyte
Connector type (beta)
Available Data replicationNot available Agent(learn more)
Support Level
Marketplace
Connector Version
0.2.25 (last updated 9 days ago)
Data workers used
0.2 per sync (Learn more)
Definition ID
d99e9ace-8621-46c2-9144-76ae4751d64b

Overview​

The Dremio source supports Full Refresh sync of catalog metadata from a self-hosted Dremio instance. It does not replicate the rows of the tables listed in the catalog.

Output schema​

The following output stream is available from this source:

If there are more endpoints you'd like Airbyte to support, please create an issue.

Features​

FeatureSupported?
Full Refresh SyncYes
Incremental - Append SyncNo
SSL connectionYes
NamespacesNo

Getting started​

Requirements​

  • A valid Dremio authentication token
  • The base URL of a self-hosted Dremio instance reachable from Airbyte

Setup guide​

Connector needs a self-hosted instance of Dremio, this way you can access the Dremio REST API on which this source is based. Please refer to Dremio Deployment Models document, or take a look at Dremio OSS for reference.

  1. Set Base URL to the URL of your self-hosted Dremio instance, including the scheme and port if needed, such as https://dremio.example.com:9047. Do not include /api/v3/catalog; the connector appends that path.
  2. Generate an authentication token following Dremio's authentication instructions.
  3. Set API Key to the returned token value, without an Authorization, Bearer, or _dremio prefix. The connector adds the _dremio prefix itself.
Self-hosted Dremio only

Replace the prefilled https://app.dremio.cloud URL with your self-hosted instance URL. The prefilled URL is the Dremio Cloud web application, not the self-hosted REST API. Dremio Cloud uses a different, project-scoped API and is not supported by this connector.

Troubleshooting unauthorized errors​

Dremio authentication tokens generated by /apiv2/login expire after 30 hours. If the connection check reports that the catalogs stream is unauthorized, generate a new token and update API Key. Ensure the token belongs to the instance in Base URL and does not already contain the _dremio prefix. See Dremio API authentication for token details.

IP allow list​

If you use Airbyte Cloud and your organization restricts access to specific IPs, add the Airbyte Cloud IP addresses to your allow list.

Reference​

Config fields reference

Field
Type
Property name
string
api_key
string
base_url

Changelog​

Expand to review
VersionDatePull RequestSubject
0.2.252026-10-0788209Send catalog Content-Type as a header and clarify self-hosted authentication setup
0.2.242025-05-2460358Update dependencies
0.2.232025-05-1059965Update dependencies
0.2.222025-05-0359375Update dependencies
0.2.212025-04-2658837Update dependencies
0.2.202025-04-1958323Update dependencies
0.2.192025-04-1257840Update dependencies
0.2.182025-04-0557243Update dependencies
0.2.172025-03-2956491Update dependencies
0.2.162025-03-2255976Update dependencies
0.2.152025-03-0855339Update dependencies
0.2.142025-03-0154456Update dependencies
0.2.132025-02-1553777Update dependencies
0.2.122025-02-0853335Update dependencies
0.2.112025-02-0152837Update dependencies
0.2.102025-01-2552330Update dependencies
0.2.92025-01-1851690Update dependencies
0.2.82025-01-1151124Update dependencies
0.2.72024-12-2850558Update dependencies
0.2.62024-12-2150029Update dependencies
0.2.52024-12-1449529Update dependencies
0.2.42024-12-1249194Update dependencies
0.2.32024-12-1148904Starting with this version, the Docker image is now rootless. Please note that this and future versions will not be compatible with Airbyte versions earlier than 0.64
0.2.22024-10-2947886Update dependencies
0.2.12024-10-2847633Update dependencies
0.2.02024-08-1944415Refactor connector to manifest-only format
0.1.152024-08-1744311Update dependencies
0.1.142024-08-1243785Update dependencies
0.1.132024-08-1043608Update dependencies
0.1.122024-08-0343201Update dependencies
0.1.112024-07-2742670Update dependencies
0.1.102024-07-2042186Update dependencies
0.1.92024-07-1738692Make compatible with builder
0.1.82024-07-1341812Update dependencies
0.1.72024-07-1041578Update dependencies
0.1.62024-07-0941127Update dependencies
0.1.52024-07-0641008Update dependencies
0.1.42024-06-2540471Update dependencies
0.1.32024-06-2240045Update dependencies
0.1.22024-06-0639235[autopull] Upgrade base image to v1.2.2
0.1.12024-05-2138497[autopull] base image + poetry + up_to_date
0.1.02022-12-0119912New Source: Dremio
Was this page helpful?