An initiative of Knowlyr

Benchmark

What we are working on, and where it stands.

  1. Enterprise task benchmarksWorkspace and software-engineering tasks drawn from real operations, with executable checks and scoring criteria.
  2. RL environmentsLegacy systems and business software wrapped as interactive, resettable sandboxes for long-horizon agent evaluation.
  3. Evaluation methodsDifficulty grading, contamination control, reproducible scoring and expert-designed rubrics.

Nothing has been released yet. When it is, we will report only results we have run ourselves, and publish the evaluation protocol. For methods, see our research note; for tools, Knowlyr Open Source.