dltHub
Blog /

The future's Re-Composable: Converting Connectors Between Solutions with LLMs

  • Adrian Brudaru,
    Co-Founder & CDO

Our struggle discussed today is the needless duplication of work because connector frameworks refuse to talk to each other. Every community is spending valuable time reinventing API integrations, when we should be building shared knowledge instead.

The Real Problem
Across our field, engineers create API connectors in open source, filled with insights that are often not reflected in documentation but rather the result of hard work. Unfortunately, these scripts, wrappers or connectors live in separate languages and frameworks that reduce their composability. This forces us to redo the same work, wasting effort on converting authentication schemes, handling rate limits, and managing pagination all over again.

Re-Composable

The Way Forward: Re-composition of Information LLM Conversion of Connectors
Large language models (LLMs) offer a solution. By converting connectors between different solutions, LLMs can bridge the gaps between isolated frameworks.

This conversion can look at any rich information sources like existing implementations of API calls and reimplement them in the technology you need.

Examples of possible conversions:

  • Legacy code we wrote into modern, self maintaining best practice code (janky python to dlt)
  • Connectors from other frameworks, from new to old such as Singer, Talend, Airbyte
  • Docs or non-code sources of info, along with the API responses (during development)

Are LLM-generated connectors too good to be true?
Let's be real, generation has many limits. But when starting from a comprehensive piece of information, such as code, the generation works very well. When starting with a weaker source of information, such as docs, we will have to work with the LLMs to collect the remaining necesssary pieces of info - like response paths, incremental strategies, or dependent requests.

What about IP implications?

First, let me start by saying I am not qualified to give legal advice. Based on my research, if you discard all original source code and rewrite independently, you do not need to worry about the original work's license.

Reusing API parameters, endpoint names, or calling conventions from API documentation doesn't constitute derivative work. APIs and their documented usage patterns (such as parameter names, endpoints, or structures) are considered functional specifications of interfaces, which aren't copyrightable.

However, if you retain significant parts of the original code, original license may apply.

Try it yourself.
We've published clear docs, hands-on demos, and conversion walkthroughs so you can immediately test this out. See exactly how your connectors evolve, and let us know what you think.

For this end, we prepared materials to help you:

Are you already coding with LLMs? We encourage you to share any tips you have to the broader community.