Skip to main content

Files Pipeline Configuration

Corporate administrators configure how files are processed after they arrive: extraction, collections, knowledge ingest, taxonomy, OCR, and maintenance. Open Dashboard → Settings → Files Configuration.

This subtab is corporate-only. Developer (Booga One) accounts do not get it; they still extract and re-extract from Files. The run-folder-pipeline API returns forbidden for those accounts.

Important: Turning on auto-ingest without a default vector index leaves files extracted but not searchable in chat.

Prerequisites​

  • Admin or SuperUser role
  • Corporate tenant with the Agents product (Files Configuration lives on that Settings tab)
  • A provisioned vector database if you intend to ingest

Pipeline Automation​

Enable auto-extract independently per source:

  • Manual upload
  • Cloud one-shot import
  • Cloud folder and mailbox sync
  • URL ingest

Optionally generate AI tags and an LLM excerpt at extract time.

Auto-ingest after extract sends files into the default vector index when they have collections (or after proposals are accepted). Default collections apply when a file has none yet (including folder-level defaults).

Guards skip files that are too large, have excluded extensions, or lack an organization when your tenant requires one. Pipeline status reports those as outside coverage, not as a stuck backlog.

Default collections and index​

Pick:

  • Default vector index for chained ingest
  • Default collection ids applied at create time

Folder defaults in the Files UI override tenant defaults for that subtree.

Taxonomy​

A taxonomy is a closed label set authored outside the app (JSON) and imported here.

  1. Optionally pick a starter template
  2. Preview — validate and see create vs already-exists per label
  3. Apply — seeds collections, stores the document, and switches categorization to match the existing set
  4. Download the stored document to edit and re-apply faithfully

There is no in-app label editor. Deleting a collection elsewhere removes it from classification with no error; the Taxonomy tab warns when a seeded label is missing.

Smart categorization​

When enabled, an LLM proposes collections:

  • Match existing — closed set (use with an applied taxonomy)
  • Confirmation — writes proposals for humans to accept; chained ingest waits
  • Auto-apply — applies when confidence clears your threshold
  • Optional prompt hint, model override, cool-off and give-up for retries

Collection discovery can suggest new clusters for unlabeled files; review those proposals before they become collections.

OCR and extraction​

Configure OCR tiers and extraction mode for scans and image-heavy PDFs. Legacy Office formats can be enabled if you still receive them. Extraction quality flags (needs review, low quality) appear in pipeline statistics.

URL ingestion and maintenance​

URL ingestion pulls a remote document into Files and onto the same pipeline. Maintenance includes storage/knowledge sync utilities and related healing actions — run them when counts in Files, storage, and the knowledge base disagree.

Developer vs corporate​

CapabilityCorporateDeveloper
Files Configuration subtabYesNo
Run configured pipeline on a folderYesNo (403)
Extract pending / re-extract / pauseYesYes
Folder and mailbox sync APIsYesBlocked

LLM result-cache controls (skip unchanged extract/categorize calls) are shown for developer tenants as a platform cost tool. Cached calls bill the same as uncached from the tenant’s point of view; prefer leaving cache on.

Best Practices​

  • ✅ Do: Create the vector index and collections (Setup Guide) before enabling auto-ingest
  • ✅ Do: Start categorization in confirmation mode, then switch to auto-apply once proposals look right
  • ✅ Do: Apply a taxonomy if your firm has a fixed filing vocabulary
  • ❌ Don't: Enable auto-ingest with no default index
  • ❌ Don't: Assume Pipeline status “pending extract” includes files still waiting on cloud import — check the Cloud sync card

Troubleshooting​

Issue: Uploads extract but never ingest
Cause: No default index, no collections, or confirmation-mode proposals waiting
Solution: Set the index, assign or accept collections, then re-run the folder pipeline if needed

Issue: Sync imports sit with pipeline_paused
Cause: A folder pipeline hold (including after Cancel)
Solution: Resume in the folder Pipeline status dialog — deleting a watch does not clear the hold

Issue: Taxonomy apply looks successful but labels are missing
Cause: Collections were deleted after apply
Solution: Re-apply the document or restore the collection; check the seeded warning on the Taxonomy tab

⏱️ Read time: 12 minutes | 📊 Difficulty: intermediate | 🔄 Last updated: 2026-08-30