In the Extraction Engine section at the top of the page, register and manage the engine instances used to extract document text.
Extraction is not “picking a single engine globally” — instead, you register named instances and select them per extension within a profile.
1
Register an engine instance
Add an instance with the + button in the Extraction Engine section, then specify a Name and an Engine Type. The selectable engine types are Tika, Docling, Document Intelligence, Mistral OCR, Google Cloud Document AI, and LLM Vision — six in total (see Content Extraction for type descriptions). You can register multiple instances of the same type with different settings.
2
Use it in a profile
Pick the registered instance as the Default Engine of a document processing profile, or attach it to a specific extension under Extension → Engine Mapping.
An instance name is whatever you type when registering it. You may see names like Default - Primary or LLM Semantic - Primary on screen — that - Primary is part of a name an operator chose, not a label the system adds. Cloosphere has no feature for designating an instance as “Primary”: the engine for extensions that aren’t mapped is set per profile via Default Engine.
Previously, only one global extraction/chunking setting was possible. With profiles, define different setting bundles per use case and pick the appropriate profile for each KB. Each profile consists of a default engine, an extraction engine mapping per file extension, and chunking settings.
In the Document Processing Profiles section at the top of the Documents settings page, create and manage profiles.
1
Create profile
Click + New Profile to open the profile creation modal.
Each profile sets a default engine, extension mapping, and chunking settings — there is no Primary step
Setting
Description
Default
Name
Profile identifier name
—
Default Engine
Engine used for extensions not listed in the mapping below. Pick Built-in Engine or one of the registered extraction engine instances
Built-in Engine
Extension → Engine Mapping
Use Add Extension to add a row and assign an engine per format group (PDF, Word, Excel, PowerPoint, Image, HTML, Text, Email, EPUB) or per custom extension you type in. A row can also select Built-in Engine
Chunks smaller than this are merged with the next one. Empty or 0 disables it
Empty
Max Tokens per Chunk
Chunks above this token count are re-split before embedding. Empty uses the global default; 0 disables it
Empty
Preserve Tables
Keep tables intact during chunking by attaching them to the preceding text chunk
On
Contextual Chunking
Add a context summary to each chunk to improve retrieval accuracy (increases processing cost)
See the warning below
Contextual chunking only takes effect when a Context Model is also selected. In a new profile the switch appears on, but with no model set it isn’t applied. Pick a model and save to use it. It also doesn’t apply to files attached ad hoc in chat — only to documents ingested into a Knowledge Base.
2
Set default profile
Click Set as Default in the profile list to sync that profile’s settings as the global default. KBs without a specified profile are processed with this default.
3
Pick profile in KB
In Workspace > Knowledge Base edit screen, pick the document processing profile. If unset, the default profile applies.
The default profile can’t be deleted. Set another profile as default before deleting.
These are the six engine types you can register as Extraction Engine instances.
Register an instance, then pick it as a profile’s Default Engine or under Extension → Engine Mapping.
Engine Type
Strengths
Best For
Additional Settings
Tika
Apache Tika server-based
Diverse file format support
Server URL
Docling
Advanced document processing engine
Documents with complex layouts
Server URL
Document Intelligence
Azure AI Document Intelligence
Azure environments, accurate OCR
Endpoint + API Key
Google Cloud Document AI
Google Cloud Document AI
Tables, forms, multi-column layouts
Location + Processor ID + service account key (JSON). Project ID and Processor Version are optional
Mistral OCR
Mistral-based OCR
Image-based documents
Mistral API Key
LLM Vision
Vision LLM-based extraction
Complex layouts, text in charts/images
Vision model + extraction prompt (optional)
The Built-in Engine is not in this list. It isn’t something you register as an instance — it’s an option you pick directly in a profile’s Default Engine select and in mapping rows. If you register no extraction engines at all, every document is processed with this built-in engine.
The built-in engine is sufficient for most text-based documents (PDF, DOCX, TXT). For mostly scanned documents or image-heavy PDFs, Document Intelligence or Mistral OCR is recommended.
Normalize heading levels (##, ###) across the whole document, since pages were extracted independently
Setting
Description
Vision Model
Pick a Vision-supporting model registered in the system (e.g., gpt-4o)
Extraction Prompt
Custom prompt (default prompt used if empty)
LLM Vision incurs one LLM call per page. A 10-page PDF → ~11 calls (10 page extractions + 1 heading-level normalization). Consider processing time and cost for large documents.
Heading-level normalization sends only the heading lines, not the full body, and applies just the levels it gets back. Heading text itself is never changed, and if this call fails the extraction result is kept as-is. The step is skipped for single-page documents.
LLM Vision requires the pypdfium2 package. Without it, PDF uploads using this engine will fail. It handles PDFs and image files only — Office files such as PPTX and DOCX can’t be extracted with this engine.
Configure how extracted text is split into search-friendly chunks.
Setting
Default
Description
Text Splitter
Character
Character: split by char count. Token: split by Tiktoken tokens. Semantic: meaning-based split
Chunk Size
1000
Chunk size (chars or tokens). Reference value in Semantic mode
Chunk Overlap
100
Overlap between chunks (preserves context flow)
How should I pick the Chunk Size?
Size
Pro
Con
Recommended
Small (≤ 500)
Precise search
Context may be cut
FAQ, short paragraphs
Medium (1000)
Balanced performance
—
Most cases (default)
Large (≥ 2000)
Wider context preserved
Lower search precision
Long narrative documents
Why is Chunk Overlap needed?
When sentences are cut at chunk boundaries, related content may be missed during search. Overlap shares some text between adjacent chunks to preserve context. The default 100 is appropriate in most cases.
Enabling Bypass Embedding and Retrieval skips text splitting and embedding, injecting the entire document directly into the LLM context. Use only for small documents — large documents may exceed token limits.
New feature — A method that splits chunks based on inter-sentence meaning similarity instead of fixed size.
When Text Splitter is set to Semantic, sentences are converted to embedding vectors, and chunk boundaries are auto-determined where similarity between adjacent sentences drops sharply.
Fixed Size (Character/Token)
Semantic
Split criterion
Character / token count
Meaning similarity change
Pro
Fast and predictable
Splits at topic boundaries, improves search accuracy
Con
May cut mid-sentence/paragraph
Requires embedding engine, longer processing
Recommended
General documents
Long documents with frequent topic shifts
Semantic chunking requires an embedding engine to be configured. Without one, picking Semantic causes errors.
New feature — Preserve tables (HTML/markdown format) in documents intact instead of splitting.
Standard chunking can split tables across multiple chunks, losing row/column information. With Table Preservation enabled:
Auto-detect tables in the document (HTML <table>, markdown |...|)
Standard chunking on text portions only
Tables are attached intact to the closest text chunk
Tables exceeding chunk size are split row-by-row while preserving the header
Controlled by the Preserve Tables option in profile advanced settings (default: enabled).
Especially effective for table-heavy financial reports and technical specifications. Search accuracy for specific cell values within tables is greatly improved.
New feature — Each chunk has a context summary of the entire document generated by the LLM and prepended. Implements Anthropic’s Contextual Retrieval technique.
In later chunks of long documents, earlier context is lost, causing search matching failures. With contextual preservation enabled:
After chunking completes, call LLM for each chunk
Generate a summary of “where this chunk is in the entire document and its context”
Prepend the summary to the chunk before vectorization
[Before contextual preservation] Chunk: "In conclusion, the initial hypothesis was confirmed." → Can't tell what "A hypothesis" refers to → search fails[After contextual preservation] Chunk: "This chunk is the conclusion of the A Hypothesis Verification Report, containing the final judgment integrating 50 experiment results. In conclusion, the initial hypothesis was confirmed." → Includes context → search succeeds
Setting
Description
Contextual Chunking
Activate/deactivate toggle
Model
LLM model used for context summarization
LLM calls happen per chunk count. A document with 100 chunks → 100 LLM calls. Bulk uploading large documents can greatly increase API costs. Use selectively for small or important documents.
