Disqus Python API Docs | dltHub

Disqus is a commenting platform that provides a REST API for managing comments, threads, and users. The REST API base URL is https://disqus.com/api/3.0/ and All requests that modify data or access private user information require OAuth 2.0 authentication; some public GET endpoints may be unauthenticated but benefit from API keys..

dlt is an open-source Python library that handles authentication, pagination, and schema evolution automatically. dlthub provides AI context files that enable code assistants to generate production-ready pipelines. Install with uv pip install "dlt[workspace]" and start loading Disqus data in under 10 minutes.

---

What data can I load from Disqus?

Here are some of the endpoints you can load from Disqus:

---

How do I authenticate with the Disqus API?

Disqus uses OAuth 2.0; applications redirect users to the Disqus authorize endpoint to obtain a code, exchange it for an access_token, and then pass the access_token (and optionally api_key/api_secret) as query parameters on each request.

1. Get your credentials

1. Go to the Disqus Applications page: https://disqus.com/api/applications/
2. Create a new application or select an existing one.
3. Record the Public Key and Secret Key shown for the application.
4. For a non‑expiring access token (account owner), open your account details page and copy the Access Token.
5. For user‑specific tokens, implement the OAuth 2.0 flow: redirect users to https://disqus.com/api/oauth/2.0/authorize/ with client_id, scope, response_type=code, and redirect_uri; then exchange the returned code at https://disqus.com/api/oauth/2.0/access_token/ using your client_id and client_secret.

2. Add them to .dlt/secrets.toml

Copy
[sources.disqus_source]
access_token = "your_access_token_here"
api_key = "your_public_key_here"
api_secret = "your_secret_key_here"

dlt reads this automatically at runtime — never hardcode tokens in your pipeline script. For production environments, see setting up credentials with dlt for environment variable and vault-based options.

---

How do I set up and run the pipeline?

Set up a virtual environment and install dlt:

Copy
uv venv && source .venv/bin/activate
uv pip install "dlt[workspace]"

1. Install the dlt AI Workbench:

Copy
dlt ai init --agent <your-agent> # <agent>: claude | cursor | codex

This installs project rules, a secrets management skill, appropriate ignore files, and configures the dlt MCP server for your agent. Learn more →

2. Install the rest-api-pipeline toolkit:

Copy
dlt ai toolkit rest-api-pipeline install

This loads the skills and context about dlt the agent uses to build the pipeline iteratively, efficiently, and safely. The agent uses MCP tools to inspect credentials — it never needs to read your secrets.toml directly. Learn more →

3. Start LLM-assisted coding:

Use /find-source to load data from the Disqus API into DuckDB.

The rest-api-pipeline toolkit takes over from here — it reads relevant API documentation, presents you with options for which endpoints to load, and follows a structured workflow to scaffold, debug, and validate the pipeline step by step.

4. Run the pipeline:

Copy
python disqus_pipeline.py

If everything is configured correctly, you'll see output like this:

Copy
Pipeline disqus_pipeline load step completed in 0.26 seconds
1 load package(s) were loaded to destination duckdb and into dataset disqus_data
The duckdb destination used duckdb:/disqus.duckdb location to store data
Load package 1749667187.541553 is LOADED and contains no failed jobs

Inspect your pipeline and data:

Copy
dlt pipeline disqus_pipeline show

This opens the Pipeline Dashboard where you can verify pipeline state, load metrics, schema (tables, columns, types), and query the loaded data directly.

---

Python pipeline example

This example loads users_list_posts and categories_list_posts from the Disqus API into DuckDB. It mirrors the endpoint and data selector configuration from the table above:

Copy
import dlt
from dlt.sources.rest_api import RESTAPIConfig, rest_api_resources

@dlt.source
def disqus_source(access_token=dlt.secrets.value):
    config: RESTAPIConfig = {
        "client": {
            "base_url": "https://disqus.com/api/3.0/",
            "auth": {
                "type": "oauth2",
                "access_token": access_token,
            },
        },
        "resources": [
            {"name": "users_list_posts", "endpoint": {"path": "users/listPosts.json", "data_selector": "response"}},
            {"name": "categories_list_posts", "endpoint": {"path": "categories/listPosts.json", "data_selector": "response"}}
        ],
    }
    yield from rest_api_resources(config)


def get_data() -> None:
    pipeline = dlt.pipeline(
        pipeline_name="disqus_pipeline",
        destination="duckdb",
        dataset_name="disqus_data",
    )
    load_info = pipeline.run(disqus_source())
    print(load_info)

To add more endpoints, append entries from the resource table to the "resources" list using the same name, path, and data_selector pattern.

---

How do I query the loaded data?

Once the pipeline runs, dlt creates one table per resource. You can query with Python or SQL.

Python (pandas DataFrame):

Copy
import dlt

data = dlt.pipeline("disqus_pipeline").dataset()
sessions_df = data.users_list_posts.df()
print(sessions_df.head())

SQL (DuckDB example):

Copy
SELECT * FROM disqus_data.users_list_posts LIMIT 10;

In a marimo or Jupyter notebook:

Copy
import dlt

data = dlt.pipeline("disqus_pipeline").dataset()
data.users_list_posts.df().head()

See how to explore your data in marimo Notebooks and how to query your data in Python with dataset.

---

What destinations can I load Disqus data to?

dlt supports loading into any of these destinations — only the destination parameter changes:

Change the destination in dlt.pipeline(destination="snowflake") and add credentials in .dlt/secrets.toml. See the full destinations list.

---

Troubleshooting

Authentication Errors

Disqus API requests can fail due to incorrect or missing authentication credentials. Ensure that your access_token, api_key, and api_secret are correctly provided in your requests. For OAuth 2.0, verify that the redirect_uri is correctly configured in your Disqus application settings and matches the one used in the authorization flow. Expired access_tokens will require a refresh using the refresh_token if available, or re‑initiating the OAuth flow.

Rate Limiting

The Disqus API implements rate limiting. If you exceed the allowed number of requests within a given timeframe, your requests may be temporarily blocked, returning an error. The documentation states that endpoints inherit a global rate limit. It is recommended to implement exponential backoff and retry mechanisms in your application to handle rate limit errors gracefully. Monitor response headers for rate limit information if provided by the API.

Invalid Parameters

Many API endpoints have required and optional parameters. Providing incorrect parameter types, out‑of‑range values (e.g., limit exceeding 100), or missing required parameters will result in an error. Always refer to the specific endpoint documentation for valid parameter values and types. For example, thread or category IDs must be valid integers.

Data Not Found (Code 13)

If you request a resource that does not exist (e.g., a user ID, thread ID, or category ID that is not found), the API may return an error indicating that the resource could not be found. This often corresponds to an error code like 13 (Not Found) in the API response. Verify the IDs and parameters used in your request.

Permissions Errors

Accessing certain data or performing specific actions may require particular permissions (scopes) that were not granted during the OAuth authorization process. For instance, write scope is needed for modifying data. If your application attempts an action without the necessary permissions, the API will return a permission denied error. Ensure your application requests the appropriate scopes during the OAuth flow.

Ensure that the API key is valid to avoid 401 Unauthorized errors. Also, verify endpoint paths and parameters to avoid 404 Not Found errors.

---

Next steps

Continue your data engineering journey with the other toolkits of the dltHub AI Workbench:

• data-exploration — Build custom notebooks, charts, and dashboards for deeper analysis with marimo notebooks.
• dlthub-runtime — Deploy, schedule, and monitor your pipeline in production.

Copy
dlt ai toolkit data-exploration install
dlt ai toolkit dlthub-runtime install

• How to deploy a pipeline
• How to explore your data in marimo Notebooks
• How to query your data in Python with dataset
• Join the dlt community on Slack