Skip to main content
Version: 1.4.0 (latest)

Freshdesk

Need help deploying these sources or figuring out how to run them in your data stack?
Join our Slack community or Book a call with a dltHub Solutions Engineer.

Freshdesk is a cloud-based customer service software that provides businesses with tools for managing customer support via multiple channels including email, phone, websites, and social media.

This Freshdesk dlt verified source and pipeline example loads data using the "Freshdesk API" to the destination of your choice.

Resources that can be loaded using this verified source are:

S.No.NameDescription
1.agentsUsers responsible for managing and resolving customer inquiries and support tickets.
2.companiesCustomer organizations or groups that agents support.
3.contactsIndividuals or customers who reach out for support.
4.groupsAgents organized based on specific criteria.
5.rolesPredefined sets of permissions that determine what actions an agent can perform.
6.ticketsCustomer inquiries or issues submitted via various channels like email, chat, phone, etc.

Setup guide​

Grab credentials​

To obtain your Freshdesk credentials, follow these steps:

  1. Log in to your Freshdesk account.
  2. Click on the profile icon to open "Profile Settings".
  3. Copy the API key displayed on the right side.

Note: The Freshdesk UI, which is described here, might change. The full guide is available at this link.

Initialize the verified source​

To get started with your data pipeline, follow these steps:

  1. Enter the following command:

    dlt init freshdesk duckdb

    This command will initialize the pipeline example with Freshdesk as the source and duckdb as the destination.

  2. If you'd like to use a different destination, simply replace duckdb with the name of your preferred destination.

  3. After running this command, a new directory will be created with the necessary files and configuration settings to get started.

For more information, read the guide on how to add a verified source.

Add credentials​

  1. In the .dlt folder, there's a file called secrets.toml. It's where you store sensitive information securely, like access tokens. Keep this file safe. Here's what the file looks like:

    # Put your secret values and credentials here
    # Github access token (must be classic for reactions source)
    [sources.freshdesk]
    domain = "please set me up!" # Enter the Freshdesk domain here
    api_secret_key = "please set me up!" # Enter the Freshdesk API key here

  2. In the domain, enter the domain of your Freshdesk account.

  3. In api_secret_key, enter the API key you copied above.

Run the pipeline​

  1. Before running the pipeline, ensure that you have installed all the necessary dependencies by running the command:
    pip install -r requirements.txt
  2. You're now ready to run the pipeline! To get started, run the following command:
    python freshdesk_pipeline.py
  3. Once the pipeline has finished running, you can verify that everything loaded correctly by using the following command:
    dlt pipeline <pipeline_name> show
    For example, the pipeline_name for the above pipeline example is freshdesk_pipeline. You may also use any custom name instead.

For more information, read the guide on how to run a pipeline.

Sources and resources​

dlt works on the principle of sources and resources.

Source freshdesk_source​

This function retrieves the data from specified Freshdesk API endpoints.

@dlt.source()
def freshdesk_source(
    endpoints: Optional[List[str]] = None,
    per_page: int = 100,
    domain: str = dlt.secrets.value,
    api_secret_key: str = dlt.secrets.value,
) -> Iterable[DltResource]:
    ...

This source supports pagination and incremental data loading. It fetches data from a list of specified endpoints, or defaults to predefined endpoints in "settings.py".

endpoints: A list of Freshdesk API endpoints to fetch. Defaults to "settings.py".

per_page: The number of items to fetch per page, with a maximum of 100.

domain: The Freshdesk domain from which to fetch the data. Defaults to "config.toml".

api_secret_key: Freshdesk API key. Defaults to "secrets.toml".

Resource endpoints​

This function creates and yields a dlt resource for each endpoint in "settings.py".

@dlt.source()
def freshdesk_source(
    #args as defined above
) -> Iterable[DltResource]:
    for endpoint in ENDPOINTS:
        yield dlt.resource(
            incremental_resource,
            name=endpoint,
            write_disposition="merge",
            primary_key="id",
        )(endpoint=endpoint)

incremental_resource: A function that fetches and yields paginated data from a specified API endpoint.

name: Specifies the name of the endpoint.

write_disposition: Specifies the write disposition to load data.

primary_key: Specifies "id" as the primary key of the resource.

Customization​

Create your own pipeline​

If you wish to create your own pipelines, you can leverage source and resource methods from this verified source.

  1. Configure the pipeline by specifying the pipeline name, destination, and dataset as follows:

    pipeline = dlt.pipeline(
        pipeline_name="freshdesk_pipeline",  # Use a custom name if desired
        destination="duckdb",  # Choose the appropriate destination (e.g., duckdb, redshift, post)
        dataset_name="freshdesk_data"  # Use a custom name if desired
    )

    To read more about pipeline configuration, please refer to our documentation.

  2. To load data from all the endpoints, specified in "settings.py".

    load_data = freshdesk_source()
    # Run the pipeline
    load_info = pipeline.run(load_data)
    # Print the pipeline run information
    print(load_info)

  3. To load the data from "agents", "contacts", and "tickets":

    load_data = freshdesk_source().with_resources("agents", "contacts", "tickets")
    # Run the pipeline
    load_info = pipeline.run(load_data)
    # Print the pipeline run information
    print(load_info)

Additional Setup guides​

This demo works on codespaces. Codespaces is a development environment available for free to anyone with a Github account. You'll be asked to fork the demo repository and from there the README guides you with further steps.
The demo uses the Continue VSCode extension.

Off to codespaces!

DHelp

Ask a question

Welcome to "Codex Central", your next-gen help center, driven by OpenAI's GPT-4 model. It's more than just a forum or a FAQ hub – it's a dynamic knowledge base where coders can find AI-assisted solutions to their pressing problems. With GPT-4's powerful comprehension and predictive abilities, Codex Central provides instantaneous issue resolution, insightful debugging, and personalized guidance. Get your code running smoothly with the unparalleled support at Codex Central - coding help reimagined with AI prowess.