Build an AI matching engine with Docling, Granite & OpenShift AI

Red Hat published a quickstart on October 12, 2026 that demonstrates an AI-driven matching engine called PeopleMesh, deployed on Red Hat OpenShift AI. The engine turns natural-language queries such as "Senior Xcode developer who knows how to play chess" into results, and it lets new members upload a resume to trigger a document pipeline that extracts structured information automatically. The pipeline uses Docling for document conversion and the Granite4:3B LLM for structured data generation.

Docling's hybrid approach to PDFs

Docling converts documents into an intermediate representation called DoclingDocument, built with Python's pydantic library. The class stores each document as a flat list of TextItem objects, each carrying a label that describes its role, such as section_header or text. For hierarchical layouts, the body key returns a list of child objects that can contain further children, similar to a table of contents with page references linking sections to content.

Docling supports 35 digital formats by combining third-party parsers like python-docx for Word, openpyxl for Excel, and BeautifulSoup for HTML. These formats preserve reading order well, but PDFs encode printing order instead. The PDF content stream positions text with operators like Tm, which sets the print cursor at coordinates, losing the original reading order that connects a name to a job title. Docling addresses this with an AI model trained on 150,000 annotated samples to understand layout, including bounding boxes and semantic labels. It maps each Tm coordinate to its parent bounding box, despite challenges like differing coordinate systems and text spanning boxes.

PeopleMesh starts where Docling leaves off, exporting the DoclingDocument to Markdown. The GPU-accelerated step runs on OpenShift with hardware accelerators, processing a resume in seconds. The Markdown file then goes to the Granite4:3B LLM, which produces a structured JSON document for ingestion into the PeopleMesh database.

Guided decoding versus defensive guardrails

Converting Markdown to JSON is non-trivial because Markdown is still unstructured text. LLMs bring background knowledge to map it to structured output, but smaller open-weight models can produce invalid output. Two strategies are in broad use: guided decoders during token generation and defensive guardrails applied to the final output, often with a retry loop.

Red Hat OpenShift AI's vLLM inference server supports guided decoding using the llguidance and xgrammar libraries. Guided decoding uses the provided JSON schema during token generation to eliminate options that would violate the schema, supporting 100% structural correctness. However, it has drawbacks. If the input is missing required information, the LLM may hallucinate a value to comply with the schema. For example, if years_of_experience is required but the resume lacks a tenure line, the LLM will invent a value. Guided decoding also struggles with mixed structured and unstructured output and complex validation beyond simple schema checks.

PeopleMesh uses the defensive guardrail approach instead. It applies validation and normalization to the final JSON output, including stripping fencing tags, deduplicating list items, trimming whitespace, clamping skill levels, checking controlled vocabularies (mapping any seniority value not in a defined list to unknown), and converting single values to singleton lists. This approach defers validation until the LLM emits its full output, using parsing libraries and deterministic logic.

The quickstart covers both explicit searches and profile-based suggestions. Explicit searches use natural-language queries, while suggestions rely on AI to compare people across multiple dimensions like skills, hobbies, and roles. The system aims to turn serendipitous encounters into continuous, purposeful connectivity within organizations.