HL Hlido Independent review desk · Prague
Open data · CC-BY-4.0

Hlido open data — the agent evaluation corpus.

The full set of Hlido review verdicts and per-claim evidence summaries, dumped as JSONL on a quarterly cadence, licensed CC-BY-4.0 — permissive enough for academic citation, AI training corpora, and any downstream analysis. Methodology stays private; results are open.

Download JSONL View manifest
What this is

A machine-readable mirror of every published Hlido review.

Each line of hlido-corpus.jsonl is one reviewed AI agent: the slug, name, category, Laddoo Score, tier, summary, the headline depth metrics (proof depth, claim coverage, evidence count), the canonical review and scorecard URLs, and — when a structured scorecard exists — a sanitized array of claim-vs-evidence snippets capped at 200 characters per quote.

The dump is built from the same review registry and per-slug scorecards that power hlido.eu, so a downstream consumer reading this file is reading the same source of truth as the website itself. Internal opinion drafts, scoring weights, dispute correspondence, and raw test artifacts are excluded by design.

The HuggingFace dataset (hlido-eu/agent-benchmark) is a separate surface licensed CC-BY-NC 4.0; this dump on hlido.eu is intentionally more permissive (CC-BY-4.0) to make academic citation and AI-training-corpus inclusion frictionless.

License

CC-BY-4.0 — share and adapt with attribution.

The Hlido open data dump is licensed under the Creative Commons Attribution 4.0 International License.

You may share, redistribute, remix, and build upon the data — including for commercial purposes and AI-training corpora — provided you give appropriate credit and link to the license.

Not covered: Hlido methodology weights, scoring formulas, rubric mechanics, internal logs, brand assets, or platform code. Those remain proprietary.

Full license text: LICENSE.md (also available at creativecommons.org/licenses/by/4.0/legalcode).

How to cite

A short citation works.

Please cite as:

Hlido (hlido.eu), accessed YYYY-MM-DD.

An academic-style citation is also welcome:

Hlido. Hlido open data dump — agent evaluation corpus. CC-BY-4.0.
https://hlido.eu/data/open/  (accessed YYYY-MM-DD).

When referencing a specific review, please link directly to the canonical review URL on hlido.eu (https://hlido.eu/reviews/{slug}/) so readers see the full evidence and any updates since the dump was generated.

Schema

One JSON object per line.

Per-line fields:

  • slug— canonical agent slug
  • name— human-readable name
  • category— top-level category
  • score— Laddoo Score (0–100)
  • tier— VITAL / STEADY / FADING / FLATLINE
  • summary— one-paragraph editorial summary
  • evidence_count— number of evidence items
  • proof_depth— 0–100
  • claim_coverage— 0–100
  • assertions_passed— integer
  • assertions_failed— integer
  • momentum— 0–10 trend signal
  • review_url— canonical review on hlido.eu
  • scorecard_url— per-slug scorecard JSON
  • attestation_url— C2PA proof bundle URL
  • last_tested— ISO-8601 date
  • claims[]— optional array (see below)

When present, each claims[] entry has:

  • claim— the assertion in plain English
  • verdict— pass / partial_pass / fail / inconclusive
  • evidence_quote_short— up to 200 chars
  • source_surface— homepage / pricing / docs / cli / api

A worked example for a single line:

{
  "slug": "aider",
  "name": "Aider",
  "category": "Coding",
  "score": 90,
  "tier": "VITAL",
  "summary": "Open-source AI pair-programming CLI. Live tested in a sandboxed git repo…",
  "evidence_count": 8,
  "proof_depth": 92,
  "claim_coverage": 100,
  "assertions_passed": 5,
  "assertions_failed": 0,
  "momentum": 7,
  "review_url": "https://hlido.eu/reviews/aider/",
  "scorecard_url": "https://hlido.eu/data/scorecards/aider.json",
  "attestation_url": "https://hlido.eu/proof/aider/",
  "last_tested": "2026-04-26",
  "claims": [
    {
      "claim": "Verify the `aider --version` command exits 0 …",
      "verdict": "pass",
      "evidence_quote_short": "The command output shows 'aider 0.86.2', a valid semantic version.",
      "source_surface": "cli"
    }
  ]
}
Downloads

Download the corpus.

The dump refreshes quarterly. Verify integrity by recomputing the SHA-256 of hlido-corpus.jsonl and matching it against either the sidecar file or the sha256 field in manifest.json.

hlido-corpus.jsonl

The full dump. One review per line. ~534 KB at last refresh.

Download JSONL →

manifest.json

Generation timestamp, item count, SHA-256, license, and schema version.

View manifest →

hlido-corpus.jsonl.sha256

Sidecar checksum in standard sha256sum format.

View checksum →

LICENSE.md

Full text of the CC-BY-4.0 license, citation guidance, and scope.

Read the license →

CLI example:

curl -L -O https://hlido.eu/data/open/hlido-corpus.jsonl
curl -L -O https://hlido.eu/data/open/hlido-corpus.jsonl.sha256
sha256sum -c hlido-corpus.jsonl.sha256
# hlido-corpus.jsonl: OK

Other public surfaces

Looking for a different shape? Pick one:

  • CLI — npx @hlido/cli check <slug> for ad-hoc lookups (setup)
  • MCP server (13 tools) — JSON-RPC 2.0 at https://hlido.eu/mcp (tool reference)
  • HuggingFace dataset — hlido-eu/agent-benchmark (CC-BY-NC-4.0)
  • Trust Attestations — HMAC-signed per-slug JSON at /data/attestations/{slug}.json (schema)
  • Per-slug scorecards — /data/scorecards/{slug}.json (e.g. aider)
  • Public GitHub mirror — hlido-public
  • Integrations hub — /integrations/ for the full surface map
Contact

Questions, fixes, or a citation request?

Email [email protected] with a short note. Issues with the dump itself can be opened on the public mirror at github.com/ankitkapur1992-hlido/hlido-public.

If you publish research that cites this dump, we'd love to hear about it — drop us a line and we'll link to your paper from the methodology page.