System Overview

Memosa turns a real-estate deal’s raw documents — PDF offering memoranda, Excel underwriting models, CoStar market exports — into a structured, cited institutional investment memo that analysts edit, collaborate on, and export in a web application called Canvas.

This page is the map. It names the real subsystems, traces the data flow from intake to delivery, and links to the deep-dive page for each layer. Read it first; then follow the links into the specific subsystem you care about.

The data flow at a glance

 Slack / Web Chat  ──▶  Document Processors  ──▶  Pinecone (namespaced)
   (intake)                (PDF / Excel / CoStar)      (vector storage)
                                                            │
                                                            ▼
                                              LangGraph deal-analysis
                                              workflow (deep agents,
                                              RAG retrieval + reranking)
                                                            │
                                                            ▼
                                              Synthesis + footnotes
                                              (cited memo sections)
                                                            │
                                                            ▼
                                                   Canvas (editor)
                                              edit · collaborate · export

Every stage has a clean boundary. Processors hand structured chunks to the vector store; the vector store serves context back to the analysis workflow; the workflow’s agents synthesize cited sections; Canvas is where a human takes over. The sections below walk each boundary in turn.

1. Intake — how documents arrive

A deal begins when an analyst sends documents to Memosa. There are two front doors, and they converge on the same orchestration:

• Slack. Mention the bot and attach files in an allowed channel. The Slack bot handles user interaction, file batching, and final memo delivery. A 3-second debounce collects simultaneous file_shared events into a single batch so a multi-file drop is processed as one deal.
• Web Chat Intake. An HTTP-based intake flow inside Canvas, an alternative to Slack. Conversations are Redis-backed with web-{uuid4} thread IDs; the prefix routes completion notifications to Redis pub/sub instead of Slack.

Both paths share one orchestration component — IntakeCoordinator (src/services/intake_coordinator.py) — so the deal-creation logic is identical regardless of the front door.

Note

Channel isolation is enforced, not incidental. The staging worker listens only in #memosa-dev; production listens in #memosa-main and #initiative-memosa. An is_channel_allowed() guard in both the mention handler and the file handler keeps environments from cross-talking.

For what file types are accepted and how each is treated, see Supported Documents. For the two intake flows in detail, see Slack Intake and Web Chat Intake.

2. Document Processors — raw files to structured chunks

Three processors extract structured content, each behind an interface and wired via dependency injection in src/di/document_processors.py:

Processor	Class	Handles
PDF	PDFProcessor (src/processors/pdf/)	Offering memoranda, text + image intelligence
Excel	ExcelProcessor (src/processors/excel/)	Underwriting models, FormulaGraph dependency analysis
CoStar	CoStarProcessor (src/processors/costar/)	Market and comparable exports

A DocumentOrchestrator coordinates the three and verifies that vectors are queryable after upsert (Pinecone is eventually consistent, so the orchestrator confirms before moving on).

Two processor-side capabilities are worth knowing about up front because they shape downstream evidence quality:

• FormulaGraph. The Excel processor builds a dependency graph of a workbook’s cells so a metric like NOI or IRR can be traced back to the exact formula and source cells that produced it. The core range-expansion routine is memoized with an LRU cache of 32,768 entries , which is load-bearing for performance on real institutional workbooks. See FormulaGraph.
• Image Intelligence. The PDF pipeline renders pages and analyzes diagrams, site plans, and charts as first-class evidence, not just OCR’d text. See Image Intelligence.

When the same fact appears in more than one document, a fixed precedence governs which source wins:

USER INPUT (100)  >  EXCEL (90)  >  COSTAR (70)  >  PDF (50)

These weights are encoded in the ontology and drive conflict resolution when typed claims disagree. The full processing pipeline is documented in Document Processing.

3. Vector storage — Pinecone, namespaced

Extracted chunks are embedded and stored in Pinecone (serverless, AWS us-west-2, cosine metric). The non-negotiable property here is namespace isolation: no data leaks across deal or tenant boundaries.

Every deal gets its own namespace, produced by NamespaceManager.get_deal_identifier() (src/utils/namespace_manager.py). The identifier is deterministic per deal — derived from the org, the normalized deal name, and the first 8 digits of the deal’s Slack thread timestamp — so the same deal always resolves to the same namespace, and two deals never collide.

Caution

Namespace isolation is a correctness boundary, not a convenience. Vector operations must always pass an explicit namespace; the default-namespace sentinel in deep agents is "__unset__" specifically to force callers to propagate a real namespace rather than silently reading the wrong deal’s vectors.

See Namespacing & Isolation for the full identifier format and the guarantees it provides.

4. The LangGraph deal-analysis workflow

The heart of the system is a LangGraph workflow that orchestrates a set of specialized deep agents — one per analytical domain (financial, risk, comparables, property, market, exit strategy, construction, synthesis, and image intelligence). The orchestrator, DealAnalysisOrchestrator, is wired by get_production_langgraph_deal_analysis_orchestrator() in src/di/langgraph_agents.py, which constructs every subgraph internally and injects the shared infrastructure (vector store, checkpointer pool, LLM factory, learning services).

Each agent retrieves evidence through the RAG retrieval pipeline, which runs three retrieval stages plus a recovery cascade, then reranks results:

• Retrieval. The consolidated retrieval engine (src/utils/consolidated_retrieval/) runs primary retrieval, section-aware source-type recovery, and a pre-rerank quota stage, with a 4-tier recovery cascade for whole-empty pools and adaptive top-k per section. See Retrieval Pipeline.
• Reranking. A Voyage rerank-2.5 cross-encoder is the primary reranker; a SemanticBM25 70/30 hybrid is the fallback when Voyage is unavailable. See Reranking.

Workflow state is checkpointed in Redis (AsyncRedisSaver), and large artifacts — memo section content, agent outputs — are persisted to PostgreSQL via PGStoreClient rather than carried in graph state. The agents themselves all inherit a common BaseDeepAgent so circuit-breaking, budget handling, and memory wiring are shared rather than re-implemented per agent.

Note

A guaranteed-time budget envelope system governs the workflow. Each phase gets a protected time envelope and surplus is reallocated; downstream phases (synthesis, the final editor) carry explicit reservation floors so they never starve. This is why the system degrades gracefully under time pressure instead of failing outright — a hard-won lesson, covered in LangGraph & Deep Agents.

5. Synthesis and footnotes

Once the domain agents have produced their analysis, a synthesis stage assembles the cited memo sections and a multi-stage footnote pipeline resolves citations. Source markers in the prose use the form [SRC:n] and are re-indexed globally so the rendered memo’s footnotes are consistent and traceable back to the originating document. Section content lives in PostgreSQL (deal_store.set_memo_section); workflow state carries only a metadata-only manifest, never the full section bodies.

See Synthesis & Footnotes for the stage ordering and the citation-resolution rules.

6. Delivery — Canvas

The finished memo lands in Canvas, a FastAPI + React (ProseMirror) document editor where the analyst takes over. Canvas is where a human edits, accepts or rejects AI suggestions, comments, collaborates in real time, and ultimately exports the memo.

A document moves through a deliberately small lifecycle — DRAFT → EDITING → APPROVED — and approval is the gate that unlocks the high-fidelity exports (the Investor Packet, the Canvas PDF, OMCMS). See Document Lifecycle and The Approval Gate.

Canvas runs as its own Railway service with phase-ordered startup and a Redis pub/sub bridge ready for multi-worker operation. Its architecture, auth model, and subsystems (Deep Brushes, Quality Intelligence, Web Chat Intake) are covered in Canvas Architecture.

The tech stack, succinctly

Concern	Technology
Language / runtime	Python 3.11
Orchestration	LangGraph >= 1.0.9
Deep agents	deepagents SDK, all inheriting BaseDeepAgent
LLM	Anthropic (Claude) via ProductionLLMFactory — Sonnet for domain work, Opus for quality-critical paths
Embeddings	Voyage 4 (voyage-4-large docs / voyage-4 queries)
Vector store	Pinecone serverless (AWS us-west-2, cosine)
Reranking	Voyage rerank-2.5 (primary), SemanticBM25 hybrid (fallback)
Checkpointer	Redis 8.x (AsyncRedisSaver)
Structured store	PostgreSQL via PGStoreClient
Canvas	FastAPI + React (ProseMirror), Uvicorn, JWT auth + RBAC
Observability	Self-hosted DuckLake lakehouse (src/observability/)
Hosting	Railway (staging + production environments)

Cross-cutting principles

Two principles run through every layer above and are worth internalizing before reading any deep-dive page:

• Dependency injection is non-negotiable. Components never instantiate their own dependencies; production wiring lives in factory functions under src/di/, and tests inject fakes. See Dependency Injection.
• Namespace isolation is enforced end to end. Every vector operation passes an explicit, validated namespace. See Namespacing & Isolation.

Sources

• .claude/rules/00-foundation.md — core principles, system architecture, and component boundaries.
• CLAUDE.md — full system overview and tech stack (LLM factory, embeddings, vector store, reranker, checkpointer, store, budget system).
• .claude/rules/10-domains.md — directory structure and Railway service topology.
• src/di/main.py — the top-level production wiring (get_production_services) showing intake, processors, vector store, orchestrator, and notification components assembled.
• src/di/document_processors.py — PDF / Excel / CoStar processor factories and DocumentOrchestrator wiring.
• src/di/langgraph_agents.py — get_production_langgraph_deal_analysis_orchestrator (the deal-analysis workflow factory).
• src/processors/excel/formula_graph/graph_builder.py — _expand_range @lru_cache(maxsize=32768) (FormulaGraph range memoization).
• src/intel/ontology/cre_ontology.yaml — source precedence weights (USER_INPUT: 100, EXCEL: 90, COSTAR: 70, PDF: 50).
• src/utils/namespace_manager.py — NamespaceManager.get_deal_identifier() (per-deal namespace format).
• src/canvas/models/section_lifecycle.py — DocumentState (DRAFT/EDITING/APPROVED) and DOCUMENT_TRANSITIONS.