pipeline::PipelineRunner¶

Public Functions¶

	Name
	init(self self, Optional project_root[Path] =None, Optional config_path[Path] =None)
None	run(self self, Optional niche[str] =None, Optional from_stage[str] =None, bool force =False)

Protected Functions¶

	Name
None	_load_config(self self)
StageResult	_run_stage(self self, str niche, str stage)
List[str]	_build_command(self self, str niche, str stage)
List[str]	_load_niches(self self)
int	_stage_index(self self, str stage_name)
bool	_is_complete(self self, str niche, str stage)
None	_mark_complete(self self, str niche, str stage)
None	_print_success_output(str stage, StageResult result)
None	_print_failure_output(str stage, StageResult result)

Public Attributes¶

	Name
list	STAGES
	STAGES

Protected Attributes¶

	Name
int	_kDefaultRetryCount
float	_kDefaultBackoffSeconds
int	_kDefaultStageTimeout
	_checkpoint
	_config
	_config_path
	_stage_retry_count
	_stage_backoff_seconds

Detailed Description¶

class pipeline::runner::PipelineRunner;

Orchestrates the 7-stage pipeline for all specialist niches.

Loads configuration from YAML, executes each stage via subprocess with
stdout/stderr capture, validates outputs with CheckpointValidator, and
supports --force and --from-stage flags for checkpoint control.

Public Functions Documentation¶

function init¶

__init__(
    self self,
    Optional project_root[Path] =None,
    Optional config_path[Path] =None
)

Initialize the pipeline runner.

Args:
    project_root: Root directory of the gnus-poc project.
        Defaults to the parent of this file's directory.
    config_path: Path to ``pipeline.yaml``. Defaults to
        ``{project_root}/config/pipeline.yaml``.

function run¶

None run(
    self self,
    Optional niche[str] =None,
    Optional from_stage[str] =None,
    bool force =False
)

Run the pipeline for all niches (or a single niche).

Args:
    niche: Run for a single specialist niche. If ``None``, runs for
        all niches listed in ``pipeline.yaml``.
    from_stage: Stage name to resume from (inclusive). Earlier stages
        are skipped if their checkpoints exist.
    force: If ``True``, clear all checkpoints and re-run every stage.

Protected Functions Documentation¶

function _load_config¶

None _load_config(
    self self
)

Load pipeline configuration from YAML file.```


### function _run_stage

```python
StageResult _run_stage(
    self self,
    str niche,
    str stage
)

Execute a single pipeline stage for the given niche via subprocess.

Handles retry, timeout, and per-D-10 error-type classification.

function _build_command¶

List[str] _build_command(
    self self,
    str niche,
    str stage
)

Build the subprocess command list for a given niche and stage.

Uses ``sys.executable`` for the Python interpreter so the same
environment is used for subprocess stages.

function _load_niches¶

List[str] _load_niches(
    self self
)

Load the list of specialist niches from configuration.```


### function _stage_index

```python
int _stage_index(
    self self,
    str stage_name
)

Return the zero-based index of *stage_name* in ``STAGES``.

Returns 0 if the name is not found (treat unknown as start).

function _is_complete¶

bool _is_complete(
    self self,
    str niche,
    str stage
)

Check whether a validated checkpoint exists for this niche/stage.```


### function _mark_complete

```python
None _mark_complete(
    self self,
    str niche,
    str stage
)

Validate stage outputs and write a checkpoint file if they pass.```


### function _print_success_output

```python
static None _print_success_output(
    str stage,
    StageResult result
)

Print a summary of successful stage output.```


### function _print_failure_output

```python
static None _print_failure_output(
    str stage,
    StageResult result
)

Print diagnostic information for a failed stage.```


## Public Attributes Documentation

### variable STAGES

```python
static list STAGES =  [
        "data_prep",
        "synthetic_data",
        "dedup",
        "train",
        "evaluate",
        "distill",
        "quantize",
    ];

variable STAGES¶

STAGES;

Protected Attributes Documentation¶

variable _kDefaultRetryCount¶

static int _kDefaultRetryCount =  1;

variable _kDefaultBackoffSeconds¶

static float _kDefaultBackoffSeconds =  5.0;

variable _kDefaultStageTimeout¶

static int _kDefaultStageTimeout =  3600;

variable _checkpoint¶

_checkpoint;

variable _config¶

_config;

variable _config_path¶

_config_path;

variable _stage_retry_count¶

_stage_retry_count;

variable _stage_backoff_seconds¶

_stage_backoff_seconds;

Updated on 2026-07-25 at 22:56:57 +0000