# Document Workspace Backend

Local, CPU-only PDF extraction and SQLite FTS5 search. Search results cite the
original document ID and one-based PDF page number. There is no generative model,
remote OCR API, inferred answer, or automatic document upload to another service.

## Run

Use Python 3.11 or newer. From the product directory:

```sh
python -m venv .venv
.venv/bin/python -m pip install -r backend/requirements.txt
DOCWORK_FRONTEND_DIR=frontend/dist .venv/bin/python -m backend
```

On Windows the executable is `.venv\Scripts\python.exe`; set environment variables
in PowerShell with `$env:DOCWORK_FRONTEND_DIR = 'frontend/dist'`. The default server
listens only on `127.0.0.1:8765` and starts exactly one processing worker. Do not
run multiple Uvicorn workers against the same data directory.

Optional OCR requires system `pdftoppm` (Poppler) and `tesseract` with English
language data available on PATH. Without them, text PDFs still work; image-only
pages explicitly report the missing dependency. OCR is best-effort, not a claim
of perfect transcription. The parser does not preserve arbitrary visual layout,
infer table structure, extract named business fields, or guarantee reading order.

## Configuration

- `DOCWORK_DATA_DIR`: private storage directory, default `data` under the product.
- `DOCWORK_FRONTEND_DIR`: optional built frontend; absent directory means no UI is served.
- `DEMO_MODE=true`: reject all non-read-only HTTP actions before reading their bodies.
- `DOCWORK_ALLOWED_HOSTS`: comma-separated exact hostnames/IPs. Default local hosts.
- `DOCWORK_OCR_LANG`: `eng` (default), `deu`, `eng+deu` or `deu+eng`; matching Tesseract language packs must be installed.

The public demo must use a separate data directory containing only owned synthetic
fixtures. Demo mode is a write gate, not authentication or a classification engine.
Never expose a private workspace directly to the Internet. An actual deployment
with private documents requires TLS, access control, and an appropriate retention
policy outside this single-user local version. No permissive external CORS is enabled.

## API

- `GET /api/health`: real worker state, OCR executable availability, and limits.
- `GET /api/documents`: `{documents, total}` with queue status and extraction warnings.
- `POST /api/documents`: multipart repeated `files`, optional `ocr=auto|never|always`.
  Returns HTTP 202 and queued document summaries; acceptance is not extraction success.
- `GET /api/documents/{id}`: summary plus `pages` with `number,text,method,warning`.
- `GET /api/search?q=...&document_id=...`: plain-word AND search, up to 100 page hits.
  `document_id` is optional. Returns `{query,results,total}`. Snippets are plain text.
- `GET /api/documents/{id}/source`: original PDF with a fixed, safe inline filename.
- `GET /api/documents/{id}/export?format=json|csv|txt`: completed page-level results.
- `DELETE /api/documents/{id}`: remove source, text and index; processing documents
  return HTTP 409. This is ordinary file deletion, not guaranteed forensic erasure.

CSV exports prefix spreadsheet-formula-like values with an apostrophe. JSON and
text exports preserve extracted text. Statuses are `queued`, `processing`,
`completed`, or `failed`. A completed document may contain empty pages or OCR
warnings; consumers must inspect these before relying on its contents.

## Bounds

10 MB per PDF, 50 pages per PDF, 10 files and 20 MB of PDF data per batch,
21 MB total multipart request size, 60 seconds per upload, two concurrent uploads,
12 pending documents, 100 stored documents, 250 MB of stored PDF source data.
No archives. Filenames are display metadata; stored files use server-generated UUIDs.
The request receive stream is capped even without Content-Length.

Each document runs in a separate child process with a 180-second wall-time limit;
rendering and OCR have separate 20/30-second per-page limits. Linux extraction
processes have 1 GB address-space and 150-second CPU limits. Windows does not have
the same address-space cap in this version. The subprocess boundary is resource
containment, not an operating-system security sandbox. Keep PDF/OCR dependencies
updated, and run the service as an unprivileged user. Extracted text is limited to
200,000 characters per page and 1,000,000 per document. OCR uses the configured English/German language selection and one
OpenMP thread. A service shutdown kills its processing tree and preserves failure
status rather than claiming success. An unclean restart resumes queued/interrupted
documents using the single data-directory worker lock.

## Reproducible Synthetic Demo

```sh
python -m pip install -r backendtests/requirements.txt
python -m unittest backendtests.test_backend -v
python -m backendtests.fixtures fixtures
python -m backend.seed --data-dir demo-data fixtures/operations-handbook.pdf fixtures/supplier-brief.pdf fixtures/scanned-delivery-note.pdf
```

The handbook has three pages and `ORCHID-2048` on page 2. The supplier brief has
two pages. The image-only delivery note has `SCAN-COPPER-732` and quantity `42` on
page 1; actual OCR must be available for its text to become searchable. Tests skip
the successful OCR assertion when dependencies are missing, while testing explicit
missing-dependency and timeout behavior separately. Fixture generation uses owned
synthetic content, ReportLab and Pillow, not customer data.

## Primary Implementation References

- [FastAPI multipart files](https://fastapi.tiangolo.com/tutorial/request-files/)
- [pypdf text extraction and limitations](https://pypdf.readthedocs.io/en/stable/user/extract-text.html)
- [SQLite FTS5](https://www.sqlite.org/fts5.html)

These official references were checked during implementation on 2026-09-05.
