# Provisioning Flow ## Overview When a new company tenant is created, an asynchronous provisioning workflow sets up the complete environment. The process is idempotent, step-aware, and retryable. --- ## Sequence Diagram ``` Owner Portal Control Plane DB Provisioning Worker │ │ │ │ POST /tenants │ │ ├────────────────────►│ │ │ tenant.status=DRAFT │ │ │ │ │ │ POST /provision │ │ ├────────────────────►│ │ │ job inserted │ │ │ status=PENDING │ │ │ ◄──────────────────┤ │ │ │ │ │ │ poll / webhook │ │ ├───────────────────────►│ │ │ job claimed │ │ │ status=RUNNING │ │ │ │ │ │ step: create_supabase_project │ │◄───────────────────────┤ │ │ step: apply_schema │ │ │◄───────────────────────┤ │ │ step: configure_storage │ │◄───────────────────────┤ │ │ step: set_secrets │ │ │◄───────────────────────┤ │ │ step: seed_branding │ │ │◄───────────────────────┤ │ │ step: seed_modules │ │ │◄───────────────────────┤ │ │ step: create_admin │ │ │◄───────────────────────┤ │ │ step: activate │ │ │◄───────────────────────┤ │ │ │ │ │ tenant.status=ACTIVE │ │ │ │ ``` --- ## Provisioning Steps | Step | Description | Idempotency | |---------------------------|-----------------------------------------------------------|------------------------------------------------| | `create_supabase_project` | Create Supabase project via management API | Check if project with matching slug exists | | `apply_schema_migration` | Run SQL migrations on new tenant DB | Track migration versions; skip applied | | `configure_storage` | Create storage bucket `tenant-{tenantId}` | Bucket exists check before create | | `set_secrets` | Store DB URL, keys in secret manager / env | Overwrite is safe; idempotent by design | | `seed_branding` | Create initial branding record from plan defaults | Skip if branding_profile already exists | | `seed_modules` | Write feature overrides based on assigned plan | Upsert with ON CONFLICT DO NOTHING | | `create_tenant_admin` | Create admin user account in tenant Supabase Auth | Check user by email before creating | | `activate_tenant` | Set tenant.status = ACTIVE | Idempotent; no-op if already ACTIVE | --- ## Job Schema ```sql cp_provisioning_jobs: id UUID PRIMARY KEY idempotency_key TEXT UNIQUE NOT NULL -- e.g., "provision:{tenantId}:{attempt}" tenant_id UUID NOT NULL → cp_tenants status TEXT NOT NULL -- PENDING | RUNNING | COMPLETED | FAILED | CANCELLED triggered_by UUID → user_profiles -- owner user who triggered started_at TIMESTAMPTZ completed_at TIMESTAMPTZ error_message TEXT retry_count INT DEFAULT 0 created_at TIMESTAMPTZ cp_provisioning_job_steps: id UUID PRIMARY KEY job_id UUID NOT NULL → cp_provisioning_jobs step_name TEXT NOT NULL status TEXT NOT NULL -- PENDING | RUNNING | COMPLETED | FAILED | SKIPPED started_at TIMESTAMPTZ completed_at TIMESTAMPTZ error_detail TEXT output_data JSONB -- step-specific result (e.g., project URL) attempt_count INT DEFAULT 0 ``` --- ## Retry Behavior On failure: 1. Step status set to `FAILED` with `error_detail` 2. Job status set to `FAILED` 3. Tenant status set to `PROVISIONING_FAILED` 4. Owner receives notification 5. Retry button available in Owner Portal On retry: 1. New `idempotency_key` with incremented attempt counter 2. Steps that completed successfully are set to `SKIPPED` (idempotency check) 3. Failed step is re-executed from the beginning 4. Subsequent steps re-executed in order --- ## Execution Architecture ### Dispatcher Abstraction The provisioning execution mechanism is isolated behind a `ProvisioningDispatcher` interface (`lib/provisioning/dispatcher.ts`). The route calls: ```typescript getProvisioningDispatcher().enqueueProvisioning(jobId) ``` This separation means the trigger mechanism can be swapped without changing the step execution logic (`runProvisioningSteps`). | Dispatcher | Status | Mechanism | |-------------------|-----------|--------------------------------------------------| | `AfterDispatcher` | V1 active | `next/server after()` — request-detached | | `QueueDispatcher` | V2 stub | Durable queue (Vercel Queues / Inngest / BullMQ) | ### V1: AfterDispatcher Steps run inside the same Vercel function instance that handled the `/provision` request, using Next.js `after()` to execute after the HTTP response is sent. The route sets `export const maxDuration = 300` to keep the instance alive up to 5 minutes. The route returns `202 Accepted` with `{ jobId, status: 'PENDING', pollUrl }` immediately. The UI polls `/api/owner/provisioning/[jobId]` for progress. --- ## Durability Limitations (V1) > **The V1 `AfterDispatcher` is an interim bridge, not a production-grade solution.** > It is request-detached (responds before work completes) but **not crash-safe**. ### What can go wrong | Scenario | Effect | |----------|--------| | Vercel instance cold-starts mid-run (new deployment, preemption) | `after()` callback is lost; job stays `RUNNING` indefinitely | | Function timeout (> 5 min for a large seed) | Execution cut off; job stays `RUNNING` or partial `FAILED` | | Vercel deployment while a job is in-flight | New deployment invalidates the old instance; job silently abandoned | ### Detection and recovery 1. The workspace health check (`/api/owner/health`) flags jobs stuck in `RUNNING` for > 10 minutes as stale. 2. The operator retries via `POST /api/owner/provisioning/[jobId]/retry`. 3. The retry path re-uses `runProvisioningSteps`, which skips `COMPLETED` steps — recovery is always safe. ### Production target (V2) Replace `AfterDispatcher` with `QueueDispatcher` once a durable queue is provisioned: ```typescript // lib/provisioning/dispatcher.ts — one-line swap export function getProvisioningDispatcher(): ProvisioningDispatcher { return new QueueDispatcher() // was: new AfterDispatcher() } ``` The queue worker calls `runProvisioningSteps(jobId)` on its own isolated compute. Step logic is unchanged. This eliminates the `maxDuration` dependency and makes execution crash-safe. Candidate queues: **Vercel Queues** (native, at-least-once), **Inngest** (durable functions, built-in retry UI), **BullMQ + Redis** (self-managed). --- ## V1 Implementation Notes The provision route (`app/api/owner/tenants/[tenantId]/provision/route.ts`): - Creates `cp_provisioning_jobs` and `cp_provisioning_job_steps` records synchronously - Marks tenant `status = PROVISIONING` before returning - Uses an idempotency key (`provision:{tenantId}:{attempt}`) to prevent duplicate concurrent runs - Returns `202 Accepted` immediately; background execution via `AfterDispatcher` For Supabase-based tenants, `create_supabase_project` creates a new schema or configures an existing project (V1: configure existing project with tenant prefix). --- ## Failure States | Scenario | Recovery | |-----------------------------|------------------------------------------------------------| | Supabase project creation fails | Retry; provider error logged in step output | | Schema migration fails | Fix migration, increment schema version, retry | | Admin user creation fails | Check if email already used; retry with different email | | Network timeout | All steps check for existing state before acting | | Partial completion | Completed steps skipped on retry; only failed step retried |