Larch optimizes semantic filters in AI SQL queries to cut token costs
A research paper introduces Larch, a framework that reorders and optimizes semantic filter execution in database queries, reporting lower token costs than existing techniques across tested text and image workloads.
In plain words
Databases increasingly let users run queries that use AI models to filter unstructured data such as text or images, but each AI call is slow and costly. Larch is a research framework that decides the order in which these AI filters are applied to each row, reducing how many model calls a query needs. The authors report lower token costs than prior optimization methods on the datasets they tested.
What was announced
The paper, published on arXiv, explains that semantic operators let analytical queries run over unstructured data like text, images and videos, but their high inference costs and latencies make semantic SQL queries hard to apply at scale. Database engines also treat these operators as black boxes, which makes AI SQL queries difficult to optimize. Larch builds on two observations: the high latency of semantic operators leaves room for computationally heavy runtime optimization, and unstructured data usually comes with embeddings that allow efficient comparison between filter prompts and data values.
The authors present two variants. Larch-A2C encodes a semantic filter expression tree with an embedding-augmented Gated Graph Neural Network and frames the evaluation order as a Markov decision process, while Larch-Sel uses a supervised model to predict filter selectivities and then applies dynamic programming to find a near-optimal order per row. In evaluations on real-world datasets and synthetic workloads spanning text and image modalities, both variants reportedly always beat existing semantic filter optimization techniques on token usage, with 6%–30% lower total token costs and 3x–19x lower token cost overhead than Palimpzest and Quest. These figures come from the authors' own experiments and have not been independently confirmed.
What it does not show
- The evidence is the paper's abstract only, so experimental setup, baselines and implementation details cannot be verified from the supplied passages.
- Performance figures are the authors' own reported results; no independent evaluation or confirmation is available in the sources.
For security and IT teams
- Scope
- Larch targets the optimization of semantic filters in AI SQL queries; the supplied abstract does not describe a deployed product or availability.
Sources
Source-support review completed 17 Nov 2026. This story was drafted with AI assistance and checked against retained source passages. Automated checking is not independent verification.
Coverage: arXiv: abstract.
Updates and corrections
No corrections so far.
First published 17 Nov 2026
New to the subject? AI from the Ground Up starts from the beginning.