Defense in Depth for Bedrock AI Chat

Every other feature on a typical serverless site has a cost that's either fixed or trivially small per hit — a pageview beacon, a static file served from a CDN. An AI chat feature is different: every message triggers a real model inference call with a real per-token price, and nothing about a public chat endpoint inherently stops someone from calling it in a loop. That's not a reason to avoid building one — it's a reason to treat cost control as a first-class part of the architecture, not an afterthought bolted on after a surprising bill.

The pattern below is a dozen layers, organized into three groups: what stops abuse at the edge before it costs anything, what the application enforces per request and per visitor, and what catches everything else as a financial backstop. They're not fully independent — several share DynamoDB or Parameter Store — but every shared-dependency failurefails closed: the request errors and no model call happens. The point is that a bug or gap in any one layer still leaves the rest standing.

How One Message Travels

This is the actual path a message takes from the chat widget on this site to the model and back. Every numbered step is a place the request can end — and the four checks inside the Lambda run in order, cheapest first, so a request that's going to be refused never reaches a billable model call.

The Three Groups, and What Each One Catches

Read it top to bottom as a funnel — each group catches what slips past the one before it, and the last group is the one that has to hold even if everything above it somehow doesn't.

One more would round it out: Lambda reserved concurrency, a hard ceiling on how many chat invocations can run at once. It isn't enabled here — this account's total Lambda concurrency quota is too low for AWS to allow any reservation without starving every other function — so it's a known, documented gap the dozen above cover until a quota increase lands. Naming the missing layer is part of the point.

Rate Limiting at the Edge

A WAF rate-based rule scoped to just the AI route is the cheapest place to stop abuse, because it's the earliest — a blocked request never reaches API Gateway, never invokes a Lambda, and never costs a single model token. It's a coarse filter by design: it counts requests per IP over a fixed evaluation window (60 to 600 seconds, with a floor of 10 requests), so it reliably stops one IP hammering the endpoint but leaves genuinely distributed abuse and fine-grained per-visitor limiting to the layers underneath it.

The rule only applies because the AI route has its own CloudFront behavior — separate from the static-asset default, caching disabled, the full HTTP method set enabled so a POST is even allowed through. That dedicated behavior isn't a defense layer in its own right; it's the wiring that lets the WAF rule and the origin-verify header apply to this path at all, and a side benefit is that a chat reply is never cached or served to the wrong visitor.

resource "aws_wafv2_web_acl" "site" {
  provider = aws.us_east_1   # required for CLOUDFRONT scope
  name     = "your-app-web-acl"
  scope    = "CLOUDFRONT"

  default_action { allow {} }

  rule {
    name     = "RateLimitAiRoute"
    priority = 1
    action   { block {} }

    statement {
      rate_based_statement {
        limit                 = 20    # AWS's floor is now 10; 20 sits well above real use
        evaluation_window_sec = 120   # allowed windows: 60 / 120 / 300 / 600 seconds
        aggregate_key_type    = "IP"

        scope_down_statement {
          byte_match_statement {
            search_string         = "/chat"
            positional_constraint = "STARTS_WITH"
            field_to_match { uri_path {} }
            text_transformation { priority = 0, type = "NONE" }
          }
        }
      }
    }

    visibility_config {
      cloudwatch_metrics_enabled = true
      metric_name                = "RateLimitAiRoute"
      sampled_requests_enabled   = true
    }
  }

  visibility_config {
    cloudwatch_metrics_enabled = true
    metric_name                = "your-app-web-acl"
    sampled_requests_enabled   = true
  }
}

Making Sure the Request Came Through the Front Door

A WAF rule bound to the CDN only helps if every request actually goes through the CDN. The API Gateway endpoint behind the chat route has its own public URL — hit it directly and you skip the WAF rate rule, the security headers, everything at the edge. The fix is a shared secret: CloudFront injects a high-entropy X-Origin-Verify header on the request it forwards to the origin, the secret lives in Parameter Store as a SecureString (not a plaintext Lambda env var anyone with GetFunctionConfiguration could read), and the Lambda's first action — before parsing, before touching the database — is to reject anything whose header doesn't match. The same guard is on the analytics endpoint too.

# CloudFront adds a secret header on the request it sends to the origin.
# It is never returned to the browser, so a direct caller can't replay it.
custom_header {
  name  = "X-Origin-Verify"
  value = random_password.origin_shared_secret.result   # stored in SSM as a SecureString
}

// --- and the very first thing the Lambda does, before any work ---
const expected = await getSsmSecret(process.env.ORIGIN_SECRET_PARAM); // cached per cold start
if (event.headers?.['x-origin-verify'] !== expected) {
  return { statusCode: 403, body: 'Forbidden' };
}

A Quota That Doesn't Need a Login

Most rate-limiting patterns assume an authenticated user to key off of. A public chat widget usually doesn't have one, and requiring a login just to ask a question is its own kind of failure. The workaround is a salted, same-day hash of the IP address alone — never stored raw, naturally different every day — used as a lightweight visitor identity for a conditional, atomic DynamoDB update. The key is IP only, not IP plus User-Agent: the UA is client-controlled, so including it would let a caller mint a fresh quota bucket on every request just by rotating the header. IPv6 is collapsed to the /64, and a viewer-supplied X-Forwarded-For doesn't change what the API sees for the source. Alongside the message count, each visitor carries a running cost total and is cut off at 5% of the daily pool — so one actor can't drain it for everyone even with expensive messages.

// Per-visitor daily quota — one conditional, atomic DynamoDB update.
// No login required: the "visitor" is a salted same-day hash of the IP
// alone, never stored raw. Not IP + User-Agent — the UA is client-controlled,
// so including it would let a caller mint a fresh bucket per request by
// rotating the header. (IPv6 is collapsed to the /64.)
const visitorHash = createHash('sha256')
  .update(`${ip}|${today}|${SALT}`)
  .digest('hex')
  .slice(0, 16);

try {
  await ddb.send(new UpdateCommand({
    TableName: LIMITS_TABLE,
    Key: { pk: `VISITOR#${visitorHash}`, sk: today },
    UpdateExpression: 'ADD msgCount :one SET expiresAt = if_not_exists(expiresAt, :ttl)',
    ConditionExpression: 'attribute_not_exists(msgCount) OR msgCount < :limit',
    ExpressionAttributeValues: {
      ':one': 1,
      ':limit': DAILY_MESSAGE_LIMIT,
      ':ttl': Math.floor(Date.now() / 1000) + 2 * 86400,
    },
  }));
} catch (err) {
  if (err.name === 'ConditionalCheckFailedException') {
    return respond({ reply: "You've hit today's limit — thanks for exploring!", limited: true });
  }
  throw err;
}

The Kill Switch That Resets Itself

Every layer above this one can have a bug. The kill switch is the one that has to hold regardless — a running cost total, checked before every model call, and if it's already over the ceiling the request never reaches the model. Keying the total by calendar date rather than resetting a counter means a new day is automatically a fresh record at zero: no EventBridge schedule, no cleanup Lambda, nothing that can fail to run. A second total keyed by month (GLOBAL#MONTH#{yyyy-mm}) adds a monthly gate at the same ceiling, so a slow burn across many days is caught even when no single day trips.

Two details make it hold. The cost is charged before the model call, not after — a conservative estimate is written to the running totals up front and reconciled down to actual token usage on success, so a request that bills the model and then errors still counts. And when either gate trips, the widget doesn't hard-refuse: it returns the best lexical match from the bundled index verbatim (zero model cost), turning a hard outage into reduced service.

// Cost kill switch + monthly gate — no scheduled reset job. The running
// totals are keyed by date and by month, so a new day (or month) is
// automatically a fresh item starting at zero.
const today = new Date().toISOString().slice(0, 10);   // 2026-09-02
const month = today.slice(0, 7);                        // 2026-09

const [day, mo] = await Promise.all([
  ddb.send(new GetCommand({ TableName: LIMITS_TABLE, Key: { pk: 'GLOBAL', sk: today } })),
  ddb.send(new GetCommand({ TableName: LIMITS_TABLE, Key: { pk: 'GLOBAL', sk: `MONTH#${month}` } })),
]);

if ((day.Item?.costCents || 0) >= DAILY_LIMIT_CENTS ||
    (mo.Item?.costCents  || 0) >= MONTHLY_LIMIT_CENTS) {
  // Degrade, don't stop: return the top lexical match from the bundled
  // index verbatim — zero model cost, still a useful answer.
  return respond({ reply: degradedAnswer(message), limited: true, degraded: true });
}

// --- Charge a conservative estimate BEFORE the model call, so a request
// that bills the model and then errors still counts against the gate above.
// Reconcile down to actual token usage on success; on error the pre-charge
// stands (over-counts, never goes blind).
await addCost(today, visitorHash, PRECHARGE_CENTS);   // GLOBAL#day + GLOBAL#MONTH + VISITOR#hash
const result = await callModel(...);
await addCost(today, visitorHash, actualCents(result.usage) - PRECHARGE_CENTS);

Budget Alerts as a Backstop, Not the Plan

A budget alert is deliberately the slowest layer in this stack — a monthly window, and billing data itself lags usage by hours. Useless as a real-time control, valuable as an independent account-level check that shares no code path with the app logic above it. Notifications:forecasted at 80% of trend, plus actual at 80% and 100%.

One thing to get right: a third-party model on Bedrock can bill through AWS Marketplace under its own service string, not "Amazon Bedrock". Here, Claude Haiku 4.5 (Amazon Bedrock Edition) lands on the bill separately, so a budget filtered toService = "Amazon Bedrock" alone would see only the Titan embeddings and Guardrails — a fraction of the real chat cost, with the model itself invisible. So the filter lists every string the spend actually lands under, checked against Cost Explorer grouped bySERVICE, and a second budget with no filter runs alongside it as the backstop that can only fail closed.

resource "aws_budgets_budget" "ai_chat_cost" {
  name         = "your-app-ai-chat-monthly-budget"
  budget_type  = "COST"
  limit_amount = "15"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  cost_filter {
    name = "Service"
    # A third-party model (Claude, Llama) on Bedrock can bill through AWS
    # Marketplace under its OWN service string, not "Amazon Bedrock" — which
    # then covers only embeddings + Guardrails. List every string the model's
    # spend actually lands under (Cost Explorer, group by SERVICE), and pair
    # this with a second budget that has NO filter as the real backstop.
    values = ["Amazon Bedrock", "Claude Haiku 4.5 (Amazon Bedrock Edition)"]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["you@example.com"]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 100
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_email_addresses = ["you@example.com"]
  }

  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 80
    threshold_type             = "PERCENTAGE"
    notification_type          = "FORECASTED"   # warns while spend is only trending toward the cap
    subscriber_email_addresses = ["you@example.com"]
  }
}

Retrieval — Grounding the Answers (Without Paying for a Vector Database)

This chat does do retrieval-augmented generation: before it answers, it embeds the visitor's question (with the previous turn folded in, so a follow-up like "what about the second one?" still retrieves the right section), finds the most relevant sections of this site by a hybrid of semantic similarity and lexical overlap — the latter matters for a corpus dense with acronyms and proper nouns — and puts that text plus the current page in front of the model. Accuracy and cost aren't separate concerns: a chatbot that guesses wastes tokens on wrong answers and follow-up corrections.

Scope is part of the same budget story. The system prompt keeps the guide to cybersecurity and this site's own material (plus a short, fixed bio of the author — a few facts, not the résumé), and tells it to ground answers in the retrieved text. Grounding is entirely server-derived — the page path plus retrieval — with no client-supplied context field: anything a client can set that ends up near the system instructions is an injection surface, and the prompt-attack guardrail doesn't inspect the system block.

What it doesn't use is a vector database. The whole site ismaybe 24,000 tokens of prose — it fits in the model's context window several times over — so a managed vector store (roughly $350–700/month, or ~$45 for a serverless SQL one) would be the single most expensive thing in the stack, standing in for a file. Instead, a build step splits every page into heading-delimited chunks (~124; the author's résumé pages are excluded, and the build fails if a forbidden term slips into an indexable chunk — a deterministic check, not a trust in the model's output filter), embeds each once at 1024 dimensions, and writes a ~1.25 MB JSON file that ships inside the Lambda's own deployment package. At request time — afterthe quota gate — the function embeds the query (rounding-error cost) and hybrid-ranks the ~124 vectors in memory. Net cost: about$0.005–0.006 per message — ~$0.004 in model input tokens plus ~$0.0009 guardrail, the latter kept small by scoping the guardrail to the current turn rather than the whole conversation.

// A build step embeds every page section into knowledge.json (~124 chunks,
// 1024-dim; résumé pages excluded, and the build FAILS if a forbidden term
// slips in), bundled into the Lambda zip — no vector database, because the
// whole site is only ~24K tokens.
const KB = JSON.parse(readFileSync(new URL('./knowledge.json', import.meta.url)));

// ...at request time, AFTER the quota gate:
const q = await embedQuery(prevUserTurn + '\n' + message);  // one Titan call; prev turn
                                                            // included so follow-ups work
const ranked = KB.chunks
  .map((c) => ({ c, score: dot(q, c.embedding)              // semantic (both normalized)
                        + 0.15 * lexicalOverlap(message, c.text) })) // + acronym/proper-noun match
  .sort((a, b) => b.score - a.score)
  .slice(0, 5);

// the current page's own text + the top matches, delimited as inert
// reference in the system prompt — never treated as instructions.
// Grounding is entirely server-derived: no client-supplied "context" field
// (anything a client sends near the system block is an injection surface,
//  and the prompt-attack guardrail doesn't inspect the system block).
const grounding = pageChunks(currentPath).concat(ranked.map((r) => r.c));

How I'd Build This

• Start with the ceiling, not the feature. Decide the maximum acceptable daily/monthly spend before writing the first line of the chat handler — it should shape the design, not get bolted on once something looks expensive.
• Put the cheapest checks first. Edge-level blocking costs nothing per request; a model call has a real price. Order the layers so the expensive one is the last thing evaluated, not the first.
• Make every layer sufficient to matter on its own. If removing one layer means "we're basically unprotected now," it wasn't a separate layer. Layers can share a dependency (DynamoDB, Parameter Store) as long as that shared failure fails closed — no model call — rather than fails open.
• Prefer state that expires itself. A quota or kill switch keyed by date needs no cleanup job. A cleanup job is one more thing that has to keep running correctly forever.
• Meter cost before the call, not after. If the running total is written only on the success path, a request that bills the model and then errors is invisible to the kill switch. Pre-charge an estimate, reconcile to actual on success.
• Verify where your model actually bills. A third-party model on Bedrock may bill through Marketplace under its own service string. A budget scoped to "the obvious service" can miss the single biggest line item.
• Test the injection surface, not just the happy path. Any field a client can set — including "background context" that isn't the visible chat input — can carry an injection attempt if it ends up anywhere near the model's system-level instructions. If a field duplicates something the server can derive itself, delete it.
• Size the retrieval store to the corpus, not the pattern. A vector database is the right tool for millions of documents. For a knowledge base that fits in the context window, it's a monthly bill and an ingestion pipeline standing in for a file. Reach for the managed service when the in-memory version actually stops scaling.

Where This Actually Fails

• One layer treated as the whole plan. "We have a WAF rule" is not a cost-control strategy — it's one filter that can be bypassed by calling the origin directly if nothing downstream also enforces limits.
• Quotas that require infrastructure to reset. A scheduled job that resets a counter is a dependency that can fail silently — right when the quota was supposed to matter most.
• A budget alert mistaken for a kill switch. Billing data lags. An alert that arrives after the money's already spent is a postmortem, not a control.
• Trusting every field a client sends. Anything passed to the model — not just the obvious "user message" field — needs to be treated as untrusted input, including context, metadata, or history the client controls.
• No visibility into whether it's actually working. Protections nobody re-tests quietly rot. A layer that was correct on launch day and never checked again is a claim, not a fact.