eval::benchmark_mlx_model¶

Classes¶

	Name
class	eval::benchmark_mlx_model::MLXBenchmarkModel

Attributes¶

	Name
int	kDefaultMaxLength
int	kDefaultMaxGenToks

Detailed Description¶

MLX model wrapper for lm-eval-harness LM interface.

Subclasses ``lm_eval.api.model.LM`` to enable in-process inference with
MLX quantized specialist models through lm-eval's standard evaluation protocol.

Per RESEARCH.md: model is loaded once in ``__init__`` and reused for all
benchmark tasks in a ``simple_evaluate()`` call (Pitfall 3 avoidance).

Threat mitigations:
- T-04-01: Paths validated with FileNotFoundError before any MLX import.
- T-04-04: max_gen_toks capped at model context window in generate_until().

Attributes Documentation¶

variable kDefaultMaxLength¶

int kDefaultMaxLength =  2048;

variable kDefaultMaxGenToks¶

int kDefaultMaxGenToks =  256;

Updated on 2026-07-25 at 22:56:57 +0000