eval::evaluator¶

Classes¶

	Name
class	eval::evaluator::SpecialistEvaluator

Attributes¶

	Name
	parser
	required
	True
	help
	args
	project_root
	evaluator
str	test_path
list	test_samples
	line
dict	results
str	out_dir
	parents
	exist_ok
	f
	indent

Detailed Description¶

Per-specialist evaluation: perplexity, BLEU/ROUGE, latency via MLX.```



## Attributes Documentation

### variable parser

```python
parser =  argparse.ArgumentParser(description="Evaluate a specialist model");

variable required¶

required;

variable True¶

True;

variable help¶

help;

variable args¶

args =  parser.parse_args();

variable project_root¶

project_root =  Path(__file__).resolve().parent.parent;

variable evaluator¶

evaluator =  SpecialistEvaluator(project_root);

variable test_path¶

str test_path =  project_root / "data" / "specialists" / args.niche / "test.jsonl";

variable test_samples¶

list test_samples =  [];

variable line¶

line =  line.strip();

variable results¶

dict results =  {
        "niche": args.niche,
        "num_samples": len(test_samples),
        "accuracy": 0.0,
        "perplexity": 0.0,
        "latency_ms_per_token": 0.0,
    };

variable out_dir¶

str out_dir =  project_root / "artifacts" / "evaluations";

variable parents¶

parents;

variable exist_ok¶

exist_ok;

variable f¶

f;

variable indent¶

indent;

Updated on 2026-07-25 at 22:56:57 +0000