Matheus Comin.
Overview  /  Case study 02
CASE STUDY 02 / PRODUCT SECURITY

Multi-Tenant SaaS Security Engineering.

I built this private SaaS security lab to test one idea: tenant isolation should survive the mistakes and timing problems that happen between the API, database, cache, browser and outbound HTTP clients.

12 security milestones793 / 793 backend tests17 / 17 Chromium tests

The problem

A multi-tenant API has an unusually unforgiving failure mode: one customer's data can become visible to another. As I examined the architecture, the important question became how to keep identity, persisted membership, operation permissions and database row visibility independent — so one forgotten filter is not the whole defense.

Engineering principle: A tenant identifier supplied in the request selects a context; it never creates authority. Every layer must use trusted identity and current persisted state.

Where the tenant boundary actually lives

CONCEPTUAL TRUST BOUNDARIES / FROM UNTRUSTED TO ENFORCED
  1. 01
    Untrusted tenant selectorRequest route · resource identifier
    NOT AUTHORITY
  2. 02
    Persisted membershipAuthenticated human or service-account identity
    VERIFY
  3. 03
    Explicit permissionAction-scoped authorization
    AUTHORIZE
  4. 04
    PostgreSQL FORCE RLSTransaction-local tenant context · restricted role
    ENFORCE

RLS does not replace application authorization. It is an independent constraint when query filtering is bypassed.

Human JWT / browser session / API key ↓ Authenticated principal + CSRF where applicable ↓ Current membership or persisted service-account binding ↓ TenantContext + explicit operation permission ↓ Authorized-operation resource budget ↓ EF tenant filter + transaction-local PostgreSQL tenant context ↓ FORCED PostgreSQL Row-Level Security ↓ Tenant-confined mutation + required transactional audit

These are complementary controls. RLS limits row visibility, but does not grant business permissions. A cache hit does not grant authority. An API key is not a substitute for a human administrator. Rate limiting cannot legalize an otherwise forbidden request.

Selected boundaries and attack tests

BoundaryDefenseTest evidence
Tenant membership / BOLAUse current persisted membership and explicit permissions, not route authority.Cross-tenant object tests and permission denials.
PostgreSQL isolationRestricted runtime role; EF filtering plus forced RLS and transaction-local tenant context.Actual PostgreSQL tests bypass EF filters and reuse connections.
Machine identitiesHash-only API keys, tenant binding, closed scopes and revocation.Cross-tenant, scope and key lifecycle tests.
Browser sessionsOIDC/PKCE, HttpOnly cookie, CSRF and current permissions.18 focused M10 cases within the full suite.
Outbound webhooksHTTPS destination policy and public IP revalidation at socket connection time.16 focused M8 SSRF cases, including mixed public/private DNS.
Resource abusePrincipal, tenant and operation-specific rate/concurrency budgets.16 focused M9 abuse cases.
Security auditTrusted actor and tenant; admin event stored in the same transaction.62 focused M11 cases, including PostgreSQL permission checks.
PipelineImmutable Action SHAs, locked dependencies and adversarial workflow guard tests.36 Node policy tests and exact-SHA CI validation.

My most useful investigation: a race across COMMIT

ENGINEERING NOTE / 02POSTGRESQL · TWO CONNECTIONS
WRITER
SaveChangescache evictionCOMMIT
READER
cache missold committed rowstale refill
Correction: keep immediate eviction, then revalidate affected tenant-local keys after commit.

In the Master Review, we identified a data-freshness race: cache eviction happened after EF SaveChangesAsync but before the enclosing PostgreSQL transaction committed. Another authorized reader could observe the old committed value, refill the now-empty cache and leave stale data after the writer committed.

Writer: SaveChanges → evict cache ──── COMMIT ↑ Reader: cache miss → read old committed row → refill stale cache Remediation: immediate eviction + post-COMMIT eviction Verification: distinct PostgreSQL connections; commit and rollback

The remediation preserved immediate eviction and added post-commit invalidation of affected tenant-local project keys. Deterministic ordering tests and real PostgreSQL Testcontainers concurrency tests covered both commit and rollback, with separate backend process IDs. No claim of distributed-cache linearizability was made.

Validated outcome

The merged source and documentation revision completed an exact-commit security CI check: 32/32 workflow steps, 793/793 .NET xUnit tests, 17/17 Chromium/Playwright tests, seven passing Angular test files and a warning-free Release build. Focused M8/M9/M10/M11 figures are subsets of 793, not additional unique backend tests.

To validate the merge commit with the existing workflow, a temporary-purpose branch pointed at the identical merge SHA. This proves that the integrated Git object passed the configured test run; it does not mean that pushes to main automatically require passing checks.

Scope and residual risks

  • Technical tests validate the specified security behavior, not universal vulnerability absence.
  • Persistent self-hosted Windows runner isolation and repository push/branch governance are not independently certified.
  • No production deployment, external penetration test, signed full SBOM/provenance or environment-specific operations assurance is claimed.
  • The audit is restricted to approved tenant-admin events; privileged database operators remain outside its append-only guarantee.
  • Cache consistency for future multi-instance/distributed deployments is not established by the two-connection lab test.
My takeaway: security reviews are most useful at component boundaries, where two individually reasonable operations can produce an incorrect combined result.
← Back to all case studies