← News

ResearchSources: arXiv · 16 Nov 2026

ScrapeGraphAI-100k dataset pairs real web pages with schema-constrained LLM extraction data

Researchers released 93,695 schema-constrained extraction events from opt-out telemetry, aiming to fill a gap left by small, synthetic or text-only public datasets.

In plain words

Large language models are often asked to return answers that fit a fixed JSON structure, for example when software calls them as tools. The team behind ScrapeGraphAI-100k published a large set of real examples of this task, each pairing a web page with the prompt, schema and model response used in practice. The dataset is free to download on HuggingFace.

What was announced

The arXiv abstract describes ScrapeGraphAI-100k as 93,695 schema-constrained extraction events gathered through opt-out ScrapeGraphAI telemetry during Q2–Q3 2025, deduplicated and balanced by schema from 9 million raw events. The corpus covers 25,453 unique schemas and 15 named languages, though English accounts for 89% of detected content. Every instance combines Markdown-converted page content with a prompt, a schema, the LLM response and a structural conformance label produced per example with jsonschema-rs.

The authors report characterising structural diversity across the corpus and identifying sharp failure thresholds as schema complexity grows. In a case study, a 1.7B-parameter student model fine-tuned on the data closely tracked the output distribution of its GPT-5-nano teacher, but still trailed a 30B-A3B reference model with 3.3B active parameters on schema compliance. The team presents this distillation result as preliminary evidence that training on real practitioner workloads enables work that earlier synthetic or text-only corpora could not support.

What it does not show

  • The evidence is an abstract only; full methodology, benchmarks and caveats are not covered by the supplied passages.
  • The distillation result is described by the authors as preliminary and does not establish that the student model matches the larger reference model.
  • Language coverage is skewed, with English at 89% of detected content, so findings may not extend evenly across the 15 named languages.

For security and IT teams

Scope
The dataset was collected via opt-out ScrapeGraphAI telemetry and is publicly available on HuggingFace, per the abstract.

Sources

Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.

Coverage: arXiv: abstract.

Updates and corrections

No corrections so far.

First published 17 Nov 2026

New to the subject? AI from the Ground Up starts from the beginning.