distill::teacher::TeacherClient¶

Public Functions¶

	Name
	init(self self, Optional config_path[Path] =None, Optional project_root[Path] =None)
	reset_budget(self self)
	generate(self self, Optional model_name[str] =None, messages messages =None, ** kwargs)
	generate_with_logprobs(self self, Optional model_name[str] =None, messages messages =None, ** kwargs)
	generate_with_cascade(self self, messages messages, domain domain ="encyclopedic", ** kwargs)
	total_cost(self self)
	budget_cap(self self)
	circuit_open(self self)
	call_count(self self)

Protected Functions¶

	Name
str	_resolve_api_key(str endpoint_name, str api_type)
	_load_config(self self, config_path config_path)
	_get_or_create_backend(self self, str endpoint_name)
	_resolve_backend(self self, str model_name)
float	_estimate_cost(self self, int prompt_tokens, int completion_tokens)
	_log_cost(self self, str model_name, int prompt_tokens, int completion_tokens, float cost)
	_log_error(self self, str error_type, Optional status_code[int], str detail)
	_load_budget_state(self self)
	_save_budget_state(self self)
	_check_circuit(self self)
	_check_budget(self self)
bool	_is_retryable(self self, Exception exception)
	_call_api(self self, str model_name, messages messages, ** kwargs)

Protected Attributes¶

	Name
	_project_root
	_config
	_models
	_default_max_tokens
	_default_temperature
	_max_retries
	_backoff_base
	_budget_cap
	_max_consecutive_failures
	_circuit_recovery_timeout
	_endpoint_registry
	_backends
	_cascade
	_total_cost
	_budget_version
	_call_count
	_consecutive_failures
	_circuit_open
	_circuit_opened_at
	_circuit_half_open
	_cost_log_path
	_error_log_path
	_budget_state_path

Detailed Description¶

class distill::teacher::TeacherClient;

Multi-backend teacher API client.

Builds a backend registry from ``config/pipeline.yaml`` endpoints and
dispatches each ``generate()`` call to the correct backend based on the
model's endpoint ``apiType``.

Backends are constructed lazily on first use so that test code can inject
mock backends via ``client._backends`` without triggering real SDK imports.

Public Functions Documentation¶

function init¶

__init__(
    self self,
    Optional config_path[Path] =None,
    Optional project_root[Path] =None
)

function reset_budget¶

reset_budget(
    self self
)

Reset cumulative spend to zero and persist the change.

Used by the pipeline runner when the ``--reset-budget`` CLI flag
is passed.

function generate¶

generate(
    self self,
    Optional model_name[str] =None,
    messages messages =None,
    ** kwargs
)

Generate a completion through the appropriate backend.

Args:
    model_name: Model key from the ``models`` config block.  If ``None``,
        defaults to ``teacher.level1`` from pipeline.yaml.
    messages: List of message dicts (OpenAI format).
    **kwargs: Extra parameters forwarded to the backend.

Returns:
    ``_ResponseWrapper`` with ``.choices[0].message.content`` and ``.usage``.

function generate_with_logprobs¶

generate_with_logprobs(
    self self,
    Optional model_name[str] =None,
    messages messages =None,
    ** kwargs
)

Generate with log-probabilities (OpenAI-compatible endpoints only).

Args:
    model_name: Model key (defaults to ``teacher.level1``).
    messages: List of message dicts.
    **kwargs: Extra parameters.

Returns:
    ``_ResponseWrapper`` with logprobs data.

function generate_with_cascade¶

generate_with_cascade(
    self self,
    messages messages,
    domain domain ="encyclopedic",
    ** kwargs
)

Generate a completion using the multi-teacher cascade.

Routes through ``TeacherCascade.execute()``: Level 1 always runs;
Level 2 is invoked only when Level 1 confidence is below threshold
and the best Level 2 teacher is selected from the benchmark table.

Args:
    messages: List of message dicts (OpenAI format).
    domain: Specialist niche name (e.g. ``"code"``, ``"medical"``).
        Defaults to ``"encyclopedic"``.
    **kwargs: Extra parameters forwarded to each teacher call.

Returns:
    ``_ResponseWrapper`` with ``.choices[0].message.content`` set to
    the cascade's final content.  The raw response payload is the
    cascade result dict (for logging / inspection).

function total_cost¶

total_cost(
    self self
)

function budget_cap¶

budget_cap(
    self self
)

function circuit_open¶

circuit_open(
    self self
)

function call_count¶

call_count(
    self self
)

Protected Functions Documentation¶

function _resolve_api_key¶

static str _resolve_api_key(
    str endpoint_name,
    str api_type
)

Resolve the API key for an endpoint.

Priority:
1. ``LITELLM_API_KEY`` env var (for LiteLLM proxy endpoints)
2. ``{ENDPOINT_NAME_UPPER}_API_KEY`` env var
3. ``{API_TYPE_UPPER}_API_KEY`` env var (e.g. ``ANTHROPIC_API_KEY``)

Raises:
    TeacherConfigError: If no API key is found.

function _load_config¶

_load_config(
    self self,
    config_path config_path
)

function _get_or_create_backend¶

_get_or_create_backend(
    self self,
    str endpoint_name
)

Return (possibly creating) the backend instance for an endpoint.

Backends are created lazily so that tests may inject mocks into
``self._backends`` before any real SDK client is constructed.

function _resolve_backend¶

_resolve_backend(
    self self,
    str model_name
)

Look up the backend instance for a model name.

Args:
    model_name: Key in the ``models`` config block (e.g. ``"deepseek-v4-fast"``).

Returns:
    A ``TeacherBackend`` instance.

Raises:
    TeacherConfigError: If the model or its endpoint is unknown.

function _estimate_cost¶

float _estimate_cost(
    self self,
    int prompt_tokens,
    int completion_tokens
)

function _log_cost¶

_log_cost(
    self self,
    str model_name,
    int prompt_tokens,
    int completion_tokens,
    float cost
)

function _log_error¶

_log_error(
    self self,
    str error_type,
    Optional status_code[int],
    str detail
)

function _load_budget_state¶

_load_budget_state(
    self self
)

Load cumulative spend from ``artifacts/.budget_state.json``.

Budget state file format::

    {
        "cumulative_cost_usd": 1.234,
        "budget_cap_usd": 5.0,
        "last_updated": "2026-06-19T12:00:00+00:00",
        "version": 1
    }

If the file does not exist the budget starts at ``0.0``.
The budget state file can be edited manually — it is a soft
cost-control limit, not a security boundary (see T-04-01).

function _save_budget_state¶

_save_budget_state(
    self self
)

Persist current cumulative spend to ``artifacts/.budget_state.json``.

Called after every successful API call that adds cost.  Creates
parent directories if they do not exist.

function _check_circuit¶

_check_circuit(
    self self
)

Gate API calls through a half-open circuit breaker.

**Closed:**  calls proceed normally.
**Open:**    calls are blocked for ``recovery_timeout`` seconds.
             After the timeout elapses the circuit transitions to
             *half-open* — the next call is allowed as a probe.
**Half-open:** a single probe call is permitted.  If it succeeds
             the circuit closes.  If it fails the circuit re-opens
             with a fresh recovery timer.

Raises:
    CircuitBreakerOpenError: When the circuit is open and the
        recovery timeout has not elapsed.

function _check_budget¶

_check_budget(
    self self
)

Raise ``BudgetExceededError`` when cumulative spend hits the cap.

Budget enforcement reads the persisted total from disk on startup
(see ``_load_budget_state``), so the cap applies across runs.

function _is_retryable¶

bool _is_retryable(
    self self,
    Exception exception
)

function _call_api¶

_call_api(
    self self,
    str model_name,
    messages messages,
    ** kwargs
)

Execute an API call through the correct backend with retry + circuit breaker.

Circuit breaker state machine:

* **Closed** → calls proceed; after ``failure_threshold`` consecutive
  failures the circuit **opens** with a timestamp.
* **Open** → calls are blocked for ``recovery_timeout`` seconds.
* **Half-open** → one probe call is allowed.  Success **closes** the
  circuit.  Failure **re-opens** it with a fresh recovery timer.

Args:
    model_name: Key from the ``models`` config block.
    messages: List of message dicts (OpenAI format).
    **kwargs: Passed to ``backend.generate()`` (max_tokens, temperature, etc.).

Returns:
    ``_ResponseWrapper`` with ``.choices[0].message.content`` and ``.usage``.

Protected Attributes Documentation¶

variable _project_root¶

_project_root;

variable _config¶

_config;

variable _models¶

_models;

variable _default_max_tokens¶

_default_max_tokens;

variable _default_temperature¶

_default_temperature;

variable _max_retries¶

_max_retries;

variable _backoff_base¶

_backoff_base;

variable _budget_cap¶

_budget_cap;

variable _max_consecutive_failures¶

_max_consecutive_failures;

variable _circuit_recovery_timeout¶

_circuit_recovery_timeout;

variable _endpoint_registry¶

_endpoint_registry;

variable _backends¶

_backends;

variable _cascade¶

_cascade;

variable _total_cost¶

_total_cost;

variable _budget_version¶

_budget_version;

variable _call_count¶

_call_count;

variable _consecutive_failures¶

_consecutive_failures;

variable _circuit_open¶

_circuit_open;

variable _circuit_opened_at¶

_circuit_opened_at;

variable _circuit_half_open¶

_circuit_half_open;

variable _cost_log_path¶

_cost_log_path;

variable _error_log_path¶

_error_log_path;

variable _budget_state_path¶

_budget_state_path;

Updated on 2026-07-25 at 22:56:57 +0000