SpektraBot Architecture Overview

Comprehensive developer-facing documentation for the SpektraBot SEND Support System. This page covers the system architecture, technology stack, database design, AI/RAG pipeline, and API design.

---

1. Overview

SpektraBot is a GDPR-compliant AI assistant that helps UK families and education professionals navigate the Special Educational Needs and Disabilities (SEND) system. It provides real-time, context-aware guidance grounded in authoritative sources including the SEND Code of Practice, local authority Local Offers, national charity resources, and school SEND policies.

Target users:

• Parents and Carers -- Understanding SEND processes, EHCP applications, rights and entitlements
• Teachers and SENCOs -- SEND Code of Practice guidance, IEP development, professional documentation
• Administrators -- System management, analytics, knowledge base curation, crawler management, support tickets

Deployment model: Cloud-native on Azure Container Apps (UK South, UK data residency). Container images are pushed to Azure Container Registry (spektrabotacr.azurecr.io).

Brain documentation has moved. The deep technical content on prompts, message flow, agentic-RAG, retrieval, knowledge-base ingestion, safety / grounding, and evaluation now lives under The Brain. This page is now the system-wide architecture summary and keeps only the non-brain subsystems (auth, GDPR, deployment, admin).

---

2. Architecture

SpektraBot follows a monorepo architecture with a Next.js frontend, Express API backend, PostgreSQL database with pgvector for semantic search, Redis for caching and WebSocket coordination, and Azure OpenAI GPT-4.1 (UK South) for language model inference.

---

3. Technology Stack

Layer	Technology	Version
Frontend	Next.js (App Router)	15.5.7
UI Framework	React	19.0.1
UI Components	shadcn/ui, Radix UI, Recharts	--
Icons	lucide-react	0.468
Styling	Tailwind CSS	3.4
Backend Runtime	Node.js	22
Backend Language	TypeScript	5.6+
HTTP Framework	Express	4.21
WebSocket	Socket.IO + Redis Adapter	4.8.1
ORM	Prisma	6.18
Database	PostgreSQL + pgvector	16
Cache	Redis	7.4
AI LLM	Azure OpenAI GPT-4.1 (UK South)	--
AI Embeddings	Azure OpenAI text-embedding-3-large (1536d)	--
AI Framework	LangChain v1.x + LangGraph	1.0.2
Testing	Vitest + Supertest	2.1
Deployment	Azure Container Apps (UK South)	--
Container Registry	Azure Container Registry (spektrabotacr.azurecr.io)	--
Email	Resend + React Email	--
Analytics	Umami (privacy-focused)	--
Monitoring	Prometheus + Grafana	kube-prometheus-stack 81.6
Monorepo	pnpm workspaces	9+

---

4. Monorepo Structure

pilot/
├── packages/
│   ├── api/                       # Node.js backend API
│   │   ├── src/
│   │   │   ├── admin/             # Admin routes, service, analytics, crawler
│   │   │   ├── auth/              # JWT auth, registration, password reset
│   │   │   ├── chat/              # WebSocket, AI service, agentic RAG, langchain
│   │   │   ├── config/            # Zod-validated configuration
│   │   │   ├── crawler/           # Web crawler worker, URL validation, content extraction
│   │   │   │   └── school-policy/ # School policy crawler worker
│   │   │   ├── gdpr/              # GDPR compliance directory
│   │   │   ├── knowledge-base/    # Document upload, chunking, embedding, validation
│   │   │   ├── middleware/        # Auth, rate limiting, GDPR consent
│   │   │   ├── observability/     # Prometheus metrics, HTTP instrumentation, LLM metrics
│   │   │   ├── preferences/       # User preference routes and service
│   │   │   ├── prompts/           # Modular prompt system, admin management
│   │   │   ├── services/          # Shared services (analytics, children, email, GIAS, etc.)
│   │   │   ├── support/           # Support ticket routes and service
│   │   │   ├── types/             # TypeScript type definitions
│   │   │   ├── utils/             # Logger (Pino), DB client, Redis client
│   │   │   └── index.ts           # Express server entry point
│   │   ├── prisma/
│   │   │   ├── schema.prisma      # Database schema (37 models)
│   │   │   ├── seed.ts            # Database seed script
│   │   │   └── migrations/        # Prisma migration history
│   │   └── tests/                 # Test suite (30+ test files)
│   │       ├── unit/              # Unit tests (auth, chat, admin, config, etc.)
│   │       ├── integration/       # Integration tests (RAG, schools, user isolation)
│   │       ├── infrastructure/    # DB persistence, pgvector tests
│   │       └── services/          # Service-level tests (children, email, waitlist)
│   │
│   └── web/                       # Next.js frontend
│       └── src/
│           ├── app/               # App Router pages (45+ routes)
│           │   ├── admin/         # Admin dashboard (18 sub-pages)
│           │   ├── chat/          # AI chat interface
│           │   ├── children/      # Child profile management
│           │   ├── settings/      # User preferences
│           │   ├── support/       # Support ticket pages
│           │   └── ...            # Public pages, auth flows
│           ├── components/        # Reusable components
│           │   ├── charts/        # Recharts-based analytics charts (7+ types)
│           │   ├── chat/          # Chat UI components + HelpDrawer
│           │   ├── onboarding/    # WelcomeDialog wizard
│           │   └── ui/            # shadcn/ui base components
│           └── lib/               # API client (110+ methods), contexts, utilities
│
├── helm/
│   ├── spektrabot/                # Application Helm chart
│   │   ├── Chart.yaml
│   │   ├── values.yaml            # Default values
│   │   ├── values-civo.yaml       # Civo-specific overrides
│   │   ├── values-pilot.yaml      # Pilot deployment
│   │   ├── values-production.yaml # Production deployment
│   │   └── templates/             # 37 Kubernetes templates
│   │
│   └── monitoring/                # Monitoring Helm chart (kube-prometheus-stack)
│       ├── Chart.yaml
│       ├── values-civo.yaml       # Prometheus, Grafana, AlertManager config
│       ├── dashboards/            # Custom Grafana dashboards
│       └── templates/             # ServiceMonitors, PodMonitors, ConfigMaps
│
├── docs/                          # Technical documentation
├── scripts/                       # Utility scripts
├── docker-compose.yml             # Local development (PostgreSQL, Redis, ClamAV)
├── package.json                   # Root monorepo config
├── pnpm-workspace.yaml            # Workspace definition
└── tsconfig.json                  # Root TypeScript config

---

5. Backend API

5.1 Express Middleware Stack

The middleware is configured in packages/api/src/index.ts:

1. Helmet -- Security headers
2. CORS -- Configurable origin allowlist with multi-domain support
3. Body parsing -- JSON (10MB limit) and URL-encoded
4. Rate limiting -- Per-user rate limiting via express-rate-limit with Redis store
5. Authentication -- JWT verification via middleware (packages/api/src/middleware/auth.ts)
6. Authorization -- Role-based access control (parent, teacher_senco, admin)
7. Input validation -- express-validator on all route handlers
8. HTTP metrics -- Prometheus instrumentation recording golden signals

5.2 API Route Groups

All routes are mounted under the /api prefix (except /metrics).

Route Group	Mount Path	Key Endpoints	Auth
Health	/api/health	GET /	None
Metrics	/metrics	Prometheus scrape endpoint	None
Auth	/api/auth	POST /register, POST /login, PUT /consent, GET /me, POST /forgot-password, POST /reset-password, PATCH /profile	Mixed
Chat	/api/chat	POST / (sync), Socket.IO for streaming	User
Preferences	/api/preferences	GET /, PATCH /, POST /reset	User
Children	/api/children	CRUD for child profiles, school search	User
Support (User)	/api/support	POST /tickets, GET /tickets, GET /tickets/:id, POST /tickets/:id/messages, POST /tickets/:id/rate	User
Waitlist	/api/waitlist	Email verification + admin approval + account creation	Mixed
Analytics (public)	/api/analytics	Umami script proxy and event collection	Mixed
Admin	/api/admin	Users CRUD, stats, audit-logs, RAG trace, eval users	Admin
Admin Support	/api/admin/support	Ticket management, stats, internal notes, assignment	Admin
Knowledge Base	/api/admin/knowledge-base	Upload, upload-zip, list, stats, quality-stats, search, validate, bulk-validate, chunks, tags, delete	Admin
Prompts	/api/admin/prompts	Templates CRUD + activate, LLM settings CRUD + activate, Modules list/update/toggle, preview	Admin
Crawler	/api/admin/crawler	LA crawl trigger, batch crawl, job status, queue status, coverage	Admin
School Crawler	/api/admin/school-crawler	School crawl trigger, batch by LA, job status, queue status	Admin
Schools & GIAS	/api/admin/schools	School listing/search/detail, stats, GIAS sync trigger/status/history, CSV upload	Admin
Analytics (admin)	/api/admin/analytics	KPI, new-users, conversations, messages, conversation-volume, feedback-summary, LA coverage, user roles, persona distribution	Admin

Total: 20+ route groups, 110+ REST endpoints + real-time WebSocket

5.3 Chat Message Flow

Moved to Brain — Message Flow. Full sequence diagram, layer boundaries (socket.ts → ai-service.ts → langchain.ts → agentic-rag.ts), streaming internals and latency budget live there.

5.4 WebSocket Events

Client to Server:

Event	Payload	Description
message:send	{ conversationId?, content, attachments? }	Send a message (creates conversation if needed)
conversations:list	--	Request conversation list
conversation:history	{ conversationId }	Request message history
conversation:delete	{ conversationId }	Delete a conversation
typing:start	{ conversationId }	Typing indicator start
typing:stop	{ conversationId }	Typing indicator stop
message:feedback	{ messageId, feedback }	Thumbs up (1) / down (0) / remove (null)

Server to Client:

Event	Payload	Description
message:created	{ conversationId, message }	User message saved
message:ai:start	{ conversationId }	AI generation starting
message:ai:chunk	{ conversationId, content }	Streaming token chunk
message:ai:end	{ conversationId, message }	AI response complete (includes sources)
rag:stage	{ conversationId, stage }	RAG pipeline stage update
conversations:list	Conversation[]	Conversation list response
conversation:history	{ conversationId, messages }	Message history response
conversation:deleted	{ conversationId }	Deletion confirmed
conversation:title:updated	{ conversationId, title }	Auto-generated title
error	{ message }	Error notification

---

6. Agentic RAG Pipeline

Moved to Brain — Agentic RAG. The canonical state diagram (9 nodes, 10 conditional edges), state shape, entry point, default env vars, routing logic, tool calling, streaming and self-correction loop all live there.

Related: Retrieval covers the retrieve node (hybrid search + RRF + LA bias + chunk expansion). Safety & Grounding covers the post-generation grounding pass.

---

7. Knowledge Base System

Implemented in packages/api/src/knowledge-base/.

7.1 Document Upload

• Supported formats: PDF, DOCX, TXT, Markdown, HTML, CSV, ZIP (bulk upload)
• Upload via admin API with multer middleware
• File size limit: 50MB per file, 200MB per ZIP
• ZIP extraction for batch processing

7.2 Chunking Strategy

• Recursive text splitting via LangChain RecursiveCharacterTextSplitter
• Chunk size: configurable (default ~1000 characters with overlap)
• SHA-256 content hashing for deduplication
• Chunk index tracking for document reconstruction

7.3 Embedding Generation

• Model: Azure OpenAI text-embedding-3-large (1536 dimensions)
• Embeddings stored in pgvector columns on knowledge_base table
• L2 normalization for accurate cosine similarity
• Validation to prevent SQL injection via embedding arrays

7.4 Quality Scoring and Validation

• Content length tracking for quality assessment
• Admin validation workflow (pending/approved/rejected/flagged)
• Bulk validation interface with keyboard shortcuts
• LA-specific tagging for location-aware retrieval
• SEND tag classification
• Cross-LA deduplication to prevent duplicate content

7.5 Admin Approval Flow

1. Documents uploaded (status: pending)
2. Admin reviews in validation interface (grouped by Local Authority)
3. Admin approves or rejects (status: approved/rejected)
4. Approved documents available for RAG retrieval

---

8. Web Crawler

Implemented in packages/api/src/crawler/.

8.1 LA-Specific Document Discovery

• Seed URLs configured per Local Authority (152 English LAs)
• Additional seed URLs for national charity content (10+ domains)
• Depth-controlled crawling (configurable max depth)
• Breadth-first link discovery

8.2 Change Detection and Re-crawling

• HTTP caching headers (ETag, Last-Modified)
• SHA-256 content hashing for change detection
• Version history tracking on crawled URLs
• Scheduled re-crawl via admin interface

8.3 Content Quality

• SEND relevance scoring on discovered pages
• robots.txt compliance checking
• Content type filtering
• Devolved nation URL exclusion (England-only content)
• Base64 image stripping
• Boilerplate text filtering
• Minimum content length thresholds

8.4 Crawler Worker

• Background worker process (packages/api/src/crawler/crawler.worker.ts)
• Bull queue for job management
• Playwright-based browser rendering for JavaScript-heavy sites
• Browser-like user agent headers
• Rate limiting per domain

8.5 School Policy Crawler

A parallel crawler system in packages/api/src/crawler/school-policy/:

• Discovers SEND-relevant policies from school websites across England
• Targets 52,000+ schools in the GIAS database
• Admin UI for managing crawl jobs (single school, batch by LA, all schools)
• Discovered documents ingested into the knowledge base for RAG retrieval
• Separate tracking models (SchoolCrawlJob) for independent management

---

9. Data Sync Services

9.1 GIAS Schools Sync

Implemented in packages/api/src/services/gias-sync.service.ts:

• Daily synchronisation with DfE Get Information About Schools API
• ~52,000 English schools with comprehensive SEND metadata
• Derived fields: isOpen, isSpecialSchool, sendScore, SEN unit/resourced provision
• CronJob runs at 3 AM UTC (after database backups at 2 AM)

9.2 Ofsted Ratings Sync

Implemented in packages/api/src/services/ofsted-sync.service.ts:

• Ofsted inspection ratings for all rated schools
• Framework version tracking (pre-2024, 2024, 2025)
• Grade comparison with previous inspections
• Stored in OfstedRating model linked to schools

9.3 EHCP/SEN2 Statistics Sync

Implemented in packages/api/src/services/ehcp-stats-sync.service.ts:

• EHCP assessment requests and completions per LA per year
• 20-week timeliness metrics
• Tribunal appeal counts
• Stored in LaEhcpStatistics model

9.4 SEN Primary Need Breakdown Sync

Implemented in packages/api/src/services/sen-needs-sync.service.ts:

• Primary need prevalence per LA (ASD, SEMH, SLCN, MLD, etc.)
• Provision type breakdown (EHC plan vs SEN Support)
• Annual data per local authority
• Stored in LaSenNeedBreakdown model

9.5 SEND Tribunal Statistics Sync

Implemented in packages/api/src/services/tribunal-stats-sync.service.ts:

• Appeal rates and outcomes per LA
• Family success rates
• Mediation statistics
• Stored in LaTribunalStatistics model

9.6 Area SEND Inspections Sync

Implemented in packages/api/src/services/send-inspections-sync.service.ts:

• Joint Ofsted/CQC area SEND inspection outcomes
• LA-level inspection results
• Stored in LaSendInspection model

9.7 Postcode-to-LA Lookup

Implemented in packages/api/src/services/postcode-lookup.service.ts and location.service.ts:

• Reverse postcode lookup to determine local authority
• Cache management via PostcodeLaCache model
• Integration with child profiles: school URN resolves to LA context
• Enables LA-specific RAG retrieval for personalised guidance

---

10. Support Ticket System

Implemented in packages/api/src/support/.

10.1 Architecture

• Ticket model -- Core ticket with category, status, priority, assignment
• TicketMessage model -- Conversation thread on each ticket (user and staff messages)
• 6 categories: SUPPORT, FEEDBACK, BUG_REPORT, FEATURE_REQUEST, BILLING, OTHER
• 5 statuses: OPEN, IN_PROGRESS, WAITING_ON_USER, RESOLVED, CLOSED
• 4 priority levels: LOW, MEDIUM, HIGH, URGENT (admin-only)

10.2 User Flow

1. User creates ticket via /api/support/tickets with category and description
2. User views their tickets with status filtering
3. User can reply to tickets with messages
4. User rates satisfaction (1-5 scale + comment) after resolution

10.3 Admin Flow

1. Admin views all tickets with advanced filtering (status, category, priority, assignee, search)
2. Admin updates ticket status and priority
3. Admin assigns tickets to staff members
4. Admin adds public replies or internal notes (not visible to users)
5. Admin views aggregated support statistics (open count, resolution time, satisfaction)

---

11. Database Schema

The database uses PostgreSQL 16 with the pgvector extension for vector similarity search. 37 models across the full data lifecycle.

11.1 Key Models

Model	Table	Description
User	users	User accounts with role, persona, GDPR consent
UserPreferences	user_preferences	Accessibility settings, theme, chat preferences
Child	children	Child profiles with SEND diagnosis, linked school
Conversation	conversations	Chat sessions with detected persona and intent
Message	messages	Chat messages with RAG metadata (JSON)
MessageFeedback	message_feedback	Thumbs up/down per message
KnowledgeBase	knowledge_base	RAG chunks with 1024d vector embeddings
Document	documents	Parent documents (pre-chunking)
DocumentChunk	document_chunks	Chunked content with embeddings
DocumentShare	document_shares	User-specific document sharing with expiration
School	schools	GIAS school data (~52,000 records)
OfstedRating	ofsted_ratings	Inspection results with framework version tracking
LocalAuthority	local_authorities	152 English LAs with seed URLs
LaEhcpStatistics	la_ehcp_statistics	EHCP/SEN2 statistics per LA per year
LaSenNeedBreakdown	la_sen_need_breakdowns	Primary need percentages per LA per year
LaTribunalStatistics	la_tribunal_statistics	Tribunal appeal outcomes per LA
LaSendInspection	la_send_inspections	Area SEND joint inspection results
PostcodeLaCache	postcode_la_cache	Postcode-to-LA lookup cache
CrawlJob	crawl_jobs	LA crawler execution tracking
CrawledUrl	crawled_urls	Discovered URLs with change detection
SchoolCrawlJob	school_crawl_jobs	School policy crawler execution tracking
Ticket	tickets	Support tickets (6 categories, 5 statuses, 4 priorities)
TicketMessage	ticket_messages	Messages within support tickets (user/staff, public/internal)
PromptModule	prompt_modules	Modular prompt building blocks
PromptTemplate	prompt_templates	Versioned system prompts
LLMSettings	llm_settings	Model parameter configurations

| AuditLog | audit_logs | GDPR audit trail (30+ action types) | | DataSyncLog | data_sync_logs | GIAS/Ofsted/EHCP sync job history | | SystemConfig | system_config | Admin-adjustable settings | | WaitlistEntry | waitlist_entries | Waitlist with email verification | | PasswordResetToken | password_reset_tokens | SHA-256 hashed reset tokens | | UserEmbeddingCache | user_embedding_cache | Query embedding cache per user | | VectorSearchStats | vector_search_stats | Vector search performance tracking |

11.2 Vector Indexes

The knowledge_base and document_chunks tables use pgvector's vector(1536) column type (Azure OpenAI text-embedding-3-large) with the cosine-distance operator (<=>). Both tables have HNSW indexes (m=16, ef_construction=64, vector_cosine_ops). See Brain — Knowledge Base for the full index list and the 2026-03-05 dimension migration notes.

---

12. Frontend Application

The frontend is a Next.js 15 application using the App Router, located in packages/web/.

12.1 Pages and Routes

Public pages:

Route	Description
/	Landing page
/about	About SpektraBot
/audience	Target audience information
/faq	Frequently asked questions
/how-it-works	Platform walkthrough
/for-parents	Parent-specific information
/for-teachers	Teacher/SENCO information
/glossary	SEND terminology glossary
/resources	SEND resources directory
/trust	Trust and safety information
/policies/privacy	Privacy policy
/policies/terms	Terms of service

Authentication:

Route	Description
/login	User login
/signup	Waitlist signup
/register	Account creation (invite code required)
/forgot-password	Password reset request
/reset-password	Password reset form
/professional-setup	Professional user school linking
/waitlist/verify	Email verification

User pages:

Route	Description
/chat	AI chat interface (main application)
/children	Child profile management
/settings	User preferences and accessibility
/support	Support ticket creation and history
/support/[id]	Individual support ticket conversation

Admin dashboard (18 pages):

Route	Description
/admin	Dashboard with KPI cards and analytics charts
/admin/users	User management (CRUD, role assignment)
/admin/schools	School database browser (52,000+ records)
/admin/waitlist	Waitlist management and approval
/admin/audit-logs	GDPR audit trail viewer
/admin/analytics	Detailed analytics views
/admin/knowledge-base	Knowledge base document management
/admin/knowledge-base/validate	Content validation workflow
/admin/document-quality	Document quality metrics
/admin/la-coverage	Local Authority coverage map
/admin/prompts	Prompt module and LLM settings management
/admin/rag-workflow	RAG pipeline visualizer (live trace via SSE)
/admin/crawler	Web crawler management
/admin/school-policies	School policy crawler and document viewer
/admin/support	Support ticket management (admin view)
/admin/support/[id]	Admin support ticket detail

| /admin/evaluation | Evaluation framework management |

12.2 Admin Dashboard Components

12.3 Real-Time Features

• WebSocket streaming -- AI responses stream token-by-token via Socket.IO
• Typing indicators -- Real-time typing status
• RAG stage events -- Live pipeline progress (retrieving, grading, generating)
• Title updates -- Auto-generated conversation titles pushed in real-time
• Conversation sync -- List updates when conversations are created/deleted
• RAG trace streaming -- Admin RAG workflow visualizer streams via SSE

12.4 Help System & Onboarding

• WelcomeDialog -- Three-step onboarding wizard (persona-aware: parent vs professional tips)
• HelpDrawer -- In-app help panel with tips, limitations, keyboard shortcuts, and external links
• Links to SENDIASS, IPSEA, and Spectrum Dynamics for further support
• Completion tracked via localStorage

12.5 Accessibility

• WCAG 2.2 compliance with keyboard navigation and screen reader support
• Universal Design for Learning (UDL) principles
• Configurable font size, high contrast, reduced motion
• Theme support (light/dark/auto)
• Message density options (compact/comfortable/spacious)
• Copy button on AI responses
• Focus trapping in modals and drawers

---

13. Authentication and Authorization

13.1 JWT Flow

1. User registers or logs in via /api/auth/register or /api/auth/login
2. Server generates JWT with { userId, email, name, role } payload
3. Token returned in response body
4. Frontend stores token and sends via Authorization: Bearer <token> header
5. Socket.IO authentication via handshake auth token

13.2 Role-Based Access

Role	Value	Access
Parent/Carer	parent	Chat, children profiles, preferences, GDPR, support tickets
Teacher/SENCO	teacher_senco	Chat (professional mode), preferences, school linking, support tickets
Admin	admin	Full system access including admin dashboard

Authorization is enforced at the route level via authorize() middleware.

13.3 Waitlist and Invite System

1. User submits email to waitlist (/api/waitlist/signup)
2. Verification email sent via Resend
3. User verifies email (status: VERIFIED_PENDING_APPROVAL)
4. Admin approves in waitlist dashboard (status: APPROVED)
5. Approved user receives approval token for account creation
6. Registration requires valid INVITE_CODE environment variable

13.4 Password Reset Flow

1. User requests reset via /api/auth/forgot-password
2. Token generated, SHA-256 hashed before database storage
3. Reset email sent via Resend with token link
4. User submits new password with token via /api/auth/reset-password
5. Token verified, marked as used, password updated

---

14. GDPR Compliance

14.1 Consent Collection

• GDPR consent collected at registration (terms, privacy policy, data processing)
• Consent stored as JSONB with version, timestamp, and IP address
• Consent can be updated via /api/auth/consent
• Consent verification middleware on protected routes

14.2 Data Retention

• 90-day automatic data retention policy
• Soft delete via deletedAt timestamp on User model
• Cascade deletion: User -> Conversations -> Messages -> Attachments -> Feedback
• Audit log preserved after deletion (denormalized email for trail preservation)

14.3 Audit Logging

30+ tracked actions including:

• User authentication (login, logout)
• Consent updates
• Data export and deletion
• Conversation and message creation
• File uploads
• Admin actions (role changes, user deletion)
• Knowledge base operations
• Fine-tuning operations
• Password reset events
• Waitlist events
• Support ticket operations

14.4 User Data Isolation

• Every database query includes userId filter (enforced in service layer)
• WebSocket connections join user-specific rooms
• Conversation access verified before any read/write operation
• Admin-only routes gated by role middleware

---

15. Modular Prompt System

Moved to Brain — Prompt System (assembly order, crisis override, admin management) and Brain — Prompt Modules (per-module reference for all 12 modules).

---

16. Monitoring and Observability

16.1 Application Metrics

Implemented in packages/api/src/observability/:

• HTTP metrics middleware -- Records golden signals (latency, traffic, errors, saturation) per endpoint
• Custom LLM metrics -- Tracks LLM call counts, token usage, and latency
• Vector search stats -- Records search performance via VectorSearchStats model
• Prometheus endpoint -- /metrics exposed without authentication for scraping

16.2 Monitoring Stack

SpektraBot runs on Azure Container Apps; Azure-native monitoring provides the infrastructure layer (Container Apps metrics + Log Analytics workspace). Application-level metrics are exposed at /metrics on the API container and scraped into Azure Monitor. Historical references to Prometheus / Grafana / kube-prometheus-stack in this repository relate to the retired Civo Kubernetes setup and are no longer active.

16.3 Application metrics

• http_request_duration_seconds — golden signals per route
• llm_call_duration_seconds, llm_tokens_total — per-model Azure OpenAI usage
• vector_search_duration_seconds — RAG retrieval latency
• statutory_mismatch_detected_total{fact_name} — overclaim regressions (see Safety & Grounding)
• Custom counters for eval runs, crawler jobs, Bull queue state

---

17. Deployment

All services run on Azure Container Apps in resource group spektrabot-rg (UK South).

Service	Container App	Notes
API	spektrabot-api	Node.js 22, Express + Socket.IO, :3001
Web	spektrabot-web	Next.js 15 standalone output, :3000
Worker	spektrabot-worker	Bull queue consumer (crawlers, ingestion, eval spawning)
Docs	spektrabot-docs	This VitePress site

Container registry: spektrabotacr.azurecr.io (Azure Container Registry).

Build + deploy pattern:

# Build locally (ARM Mac → AMD64 target)
docker buildx build --platform linux/amd64 --no-cache \
  -t spektrabotacr.azurecr.io/<service>:<sha> \
  -f <path>/Dockerfile --push .

# Deploy
az containerapp update -g spektrabot-rg -n <service> \
  --image spektrabotacr.azurecr.io/<service>:<sha>

Image tagging: always the git commit SHA — never mutable tags like latest / staging / main (operational memory).

Web build gotcha: packages/web must be built with NEXT_PUBLIC_API_URL=https://api.spektrabot.co.uk (not the root domain) because next.config.ts rewrites are baked at build time. Using same-origin creates an infinite proxy loop (HTTP 431).

Database and Redis: managed Azure Postgres Flexible Server with pgvector, and managed Redis. UK South for data residency.

Backups: scheduled pg_dump → Azure Blob Storage, plus PITR on the managed Postgres.

DB operations note: az containerapp exec has been unreliable on this account; use Azure Container App Jobs for migrations and one-off scripts.

---