In this tutorial, we'll build a command-line tool that accepts a prompt, selects a model, optionally streams the answer, and reports token usage for non-streaming calls. We'll also handle common network and API errors.

Prerequisites

You'll need Python 3.9 or newer, an API key for an OpenAI-compatible chat endpoint, and the openai Python package:

python -m pip install --upgrade openai

Set your API key and base URL as environment variables. The base URL should end at the API version prefix; the SDK adds /chat/completions itself.

On macOS or Linux:

export AI_API_KEY="your-api-key"
export AI_BASE_URL="https://hefu.hk/api/v1"

On PowerShell:

$env:AI_API_KEY = "your-api-key"
$env:AI_BASE_URL = "https://hefu.hk/api/v1"

Build the CLI

Save this as model_cli.py:

import argparse
import os
import sys

from openai import APIConnectionError, APIStatusError, APITimeoutError, OpenAI


def parse_args() -> argparse.Namespace:
    parser = argparse.ArgumentParser(
        description="Send a prompt to an OpenAI-compatible chat API."
    )
    parser.add_argument("prompt", nargs="+", help="Text to send to the model")
    parser.add_argument(
        "--model",
        default="deepseek-v4-flash",
        help="Model ID supported by your API endpoint",
    )
    parser.add_argument(
        "--system",
        default="You are a helpful assistant.",
        help="System instruction",
    )
    parser.add_argument(
        "--stream",
        action="store_true",
        help="Print response chunks as they arrive",
    )
    return parser.parse_args()


def main() -> int:
    args = parse_args()
    api_key = os.getenv("AI_API_KEY")
    base_url = os.getenv("AI_BASE_URL")

    if not api_key or not base_url:
        print("Set AI_API_KEY and AI_BASE_URL first.", file=sys.stderr)
        return 2

    client = OpenAI(
        api_key=api_key,
        base_url=base_url,
        timeout=30.0,
        max_retries=2,
    )
    messages = [
        {"role": "system", "content": args.system},
        {"role": "user", "content": " ".join(args.prompt)},
    ]

    try:
        if args.stream:
            stream = client.chat.completions.create(
                model=args.model,
                messages=messages,
                stream=True,
            )
            for chunk in stream:
                if not chunk.choices:
                    continue
                text = chunk.choices[0].delta.content
                if text:
                    print(text, end="", flush=True)
            print()
        else:
            response = client.chat.completions.create(
                model=args.model,
                messages=messages,
            )
            print(response.choices[0].message.content or "")
            if response.usage:
                print(
                    f"\nTokens: input={response.usage.prompt_tokens}, "
                    f"output={response.usage.completion_tokens}",
                    file=sys.stderr,
                )
    except APITimeoutError:
        print("Request timed out.", file=sys.stderr)
        return 1
    except APIConnectionError as exc:
        print(f"Connection error: {exc}", file=sys.stderr)
        return 1
    except APIStatusError as exc:
        print(f"API error (HTTP {exc.status_code}): {exc.message}", file=sys.stderr)
        return 1

    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Run it

The default model ID in this example matches the sample endpoint above:

python model_cli.py "Explain what an API gateway does in two sentences."

Choose another model ID that your endpoint supports without editing the Python file:

python model_cli.py --model YOUR_OTHER_MODEL_ID "Summarize the purpose of unit tests."

Stream a longer answer:

python model_cli.py --stream "Explain the difference between retries and timeouts."

The second command contains a placeholder. Replace YOUR_OTHER_MODEL_ID with an actual ID listed by your API provider.

Why the two response paths differ

Without --stream, create() returns one completed response. We can read response.choices[0].message.content and, when provided, response.usage.

With --stream, the call yields chunks. Each chunk can contain new text in choices[0].delta.content. Some chunks contain no text, so we check before printing. This basic streaming example does not request usage data; if you need streaming usage, check whether your provider supports the relevant stream_options behavior.

Common failure cases

Keep the API key in an environment variable and avoid printing it in logs. For production use, add application-specific retry policies, structured logs, and request tracing.

If you adapted this code for your own endpoint, which part would you change first: model selection, output formatting, or error handling?

Disclosure: This article was drafted with AI assistance.

We’re the team behind HeFu. In this first post, we’ll introduce the four parts of our platform and show how they fit into practical workflows for developers and businesses, including teams in Malaysia.

1. Model aggregation: access multiple models through one API

Managing separate integrations can make experimentation more complicated. Each additional provider may introduce another API key, billing account and integration to maintain.

HeFu brings multiple AI models together through an OpenAI-compatible API, with a single API key and centralized usage and billing.

For developers, this provides a common starting point for comparing models and choosing one for a specific task. API usage follows a prepaid model: users top up their balance, and charges are deducted according to usage.

A useful way to evaluate models is to test the same small set of representative tasks across your candidates. Compare factual accuracy, instruction following, response time and usage cost. API compatibility simplifies integration, but individual models can still differ in supported features and parameters.

2. Office tools: turn business inputs into useful drafts

Not every task needs a custom integration. HeFu’s office tools include a Seller Assistant for common e-commerce workflows:

• Amazon listing drafts, including titles, bullet points and backend search terms.

• A+ content copy for brand stories and product presentation modules.

• Shopify product page copy, including SEO titles and meta descriptions.

• Customer support reply drafts based on buyer messages and order context.

• Buyer review analysis to identify recurring complaints and product strengths.

For example, a seller preparing a new product page can start with verified specifications, the intended audience and the product’s main benefits. These inputs provide a stronger foundation than a short instruction such as “write a good listing.”

Before publishing, review generated copy for accuracy, unsupported claims and the requirements of the marketplace you use.

3. Enterprise services: organize AI use across teams

As AI adoption expands, businesses may need to coordinate access across teams or support different customer groups.

HeFu’s enterprise offering includes options for team access, centralized management and multi-tenant use. The appropriate setup depends on how the business plans to organize users and deliver its services.

A useful starting point is to define who needs access, which workflows they will use and whether different teams or customers require separate environments.

4. Developer platform: connect AI to your own workflows

The developer platform provides APIs and resources for integrating AI into applications, websites and business processes.

One example is a product content workflow: collect verified product information, request a draft, send it for human review, and publish the approved version. This keeps AI generation connected to an actual business process.

Start with one task and a clear success criterion before expanding the integration.

What we’ll share next

This blog will cover API integration tutorials, model evaluation methods and practical AI workflows for sellers and business teams.

What are you working on: a multi-model application, an e-commerce workflow or an internal AI tool? We’d welcome your questions and feedback.