Enterprise environments (DEV/UAT/PROD)

Purpose

Provide enterprise teams a repeatable, auditable way to build, test, and promote assistant and squad configurations across environments.

Audience

• Platform admins: environment boundaries, access control, and compliance
• DevOps/Eng: CI/CD and automation
• Forward-deployed engineers: day-to-day configuration and migrations

Principles

• Isolation: Separate organizations per environment: dev, uat (or staging), prod.
• Config as Code: Store assistant/squad/tool/knowledge-base configs as JSON/YAML in Git.
• Immutability + Promotion: Create in dev, validate in uat, promote to prod via automation.
• Least privilege: RBAC, secrets isolation, and data boundaries per environment.
• Reproducibility: Idempotent apply, drift detection, and rollbacks from Git history.

Environment topology

• Organizations: One org per environment, e.g., acme-dev, acme-uat, acme-prod.
• Networking & data:
  • dev: synthetic or scrubbed data
  • uat: production-like data, avoid real PII when possible
  • prod: real data with strict logging/audit
• Access:
  • dev: engineers only
  • uat: QA, SMEs
  • prod: restricted operators; changes via CI/CD only

Resources under management

Treat these as declarative resources:

• Assistants: system prompt, tools, routing, grounding, safety settings
• Squads/Teams: membership and permissions
• Tools/Integrations: function schemas, external service configs
• Knowledge Bases: document sources, embedding settings
• Runtimes/Policies: rate limits, safety policies, fallback models

Reference resources by stable logical names (slugs) in config; resolve to IDs at apply time.

Repository structure (example)

/platform
  /assistants
    order-agent.yaml
    support-agent.yaml
  /squads
    support-level1.yaml
  /tools
    jira.yaml
    zendesk.yaml
  /knowledge
    product-faqs.yaml
  /policies
    safety.yaml
  environments.yaml       # maps env → org IDs, model defaults, endpoints
  schemas/                # JSONSchema for validation

Do not commit secrets. Store them in your secret manager (e.g., Vault, AWS Secrets Manager, GCP Secret Manager) and reference via placeholders.

Config format (YAML examples)

1
kind: Assistant
2
apiVersion: v1
3
metadata:
4
  name: order-agent
5
  description: Handles order inquiries
6
spec:
7
  systemPromptRef: prompts/order-agent.md
8
  model: gpt-4.1
9
  tools:
10
    - ref: jira
11
    - ref: zendesk
12
  knowledge:
13
    - ref: product-faqs
14
  safetyPolicyRef: policies/safety.yaml

1
kind: Tool
2
apiVersion: v1
3
metadata:
4
  name: jira
5
spec:
6
  type: http
7
  authRef: secrets/jira-token        # resolved from secret manager
8
  endpoint: https://jira.example.com
9
  operations:
10
    - name: createIssue
11
      method: POST
12
      path: /rest/api/3/issue
13
      schemaRef: schemas/jira-create-issue.json

Promotion workflow

1. Develop in DEV
   • Create/modify configs in Git.
   • Run local validation (schema/lint) and a plan/diff against dev.
   • Apply to dev; run unit/integration tests and data access checks.
2. Promote to UAT
   • Open a PR; CI runs plan against uat and posts a diff.
   • On approval, CI applies to uat using a service principal for the uat org.
3. Promote to PROD
   • Change window + ticket if required.
   • CI runs plan against prod, requires approvals from owners.
   • CI applies to prod; record the change set and artifacts.
4. Rollback
   • Revert Git commit → CI reapplies previous config (idempotent).
   • Keep backup exports from each apply job for audit.

Applying configs via API

Use a small deployer that:

• Reads YAML/JSON
• Resolves references and secrets for the target environment
• Translates to API payloads
• Uses idempotency keys and labels to detect drift

Example pseudo-commands:

$
# Export (backup)
>
curl -sS -H "Authorization: Bearer $TOKEN" \
>
  GET `https://api.vendor.com/v1/assistants?label=order-agent` > backups/order-agent-dev.json
>
>
# Apply (create or update)
>
curl -sS -H "Authorization: Bearer $TOKEN" -H "Idempotency-Key: $KEY" \
>
  -H "Content-Type: application/json" \
>
  -X PUT `https://api.vendor.com/v1/assistants/order-agent` \
>
  --data-binary @rendered/order-agent.dev.json

Recommendations:

• Idempotency: One key per resource per pipeline run
• Labeling: Tag resources with env, app, owner, sha for traceability
• Drift: Fetch current → compute diff → fail pipeline on unmanaged drift

CI/CD example (GitHub Actions)

1
name: Platform Deploy
2
3
on:
4
  pull_request:
5
  push:
6
    branches: [ main ]
7
8
jobs:
9
  plan:
10
    runs-on: ubuntu-latest
11
    steps:
12
      - uses: actions/checkout@v4
13
      - uses: actions/setup-node@v4
14
        with: { node-version: 20 }
15
      - run: npm ci
16
      - name: Validate
17
        run: npm run validate:all
18
      - name: Plan UAT
19
        env:
20
          ORG_ID: ${{ secrets.UAT_ORG_ID }}
21
          API_TOKEN: ${{ secrets.UAT_TOKEN }}
22
        run: npm run plan -- --env uat --out plan-uat.txt
23
      - uses: actions/upload-artifact@v4
24
        with: { name: plan-uat, path: plan-uat.txt }
25
26
  deploy-prod:
27
    if: github.ref == 'refs/heads/main'
28
    needs: [ plan ]
29
    permissions: { contents: read }
30
    runs-on: ubuntu-latest
31
    environment:
32
      name: prod
33
      url: https://console.vendor.com/orgs/${{ secrets.PROD_ORG_ID }}
34
    steps:
35
      - uses: actions/checkout@v4
36
      - uses: actions/setup-node@v4
37
        with: { node-version: 20 }
38
      - run: npm ci
39
      - name: Apply PROD
40
        env:
41
          ORG_ID: ${{ secrets.PROD_ORG_ID }}
42
          API_TOKEN: ${{ secrets.PROD_TOKEN }}
43
        run: npm run apply -- --env prod --approve

Naming and referencing

• Use unique slugs (e.g., order-agent) per environment
• Prefer logical refs in specs; map to environment-specific IDs at render/apply time
• Save Git commit SHA as a label on each resource for traceability

Security and compliance

• RBAC: Developers write to dev, read uat, no direct prod writes; CI principals per org
• Secrets: Keep out of Git. Resolve via secrets://path at apply time; rotate per policy
• Audit: Keep apply logs, request/response checksums, and exported snapshots per run; enable API audit logs in each org

Testing and validation

• Static: JSONSchema validation; lint refs and schema compatibility
• Dynamic: Dry-run/plan renders and diffs
• Behavioral: Golden-path chat transcripts in dev and uat; tool execution smoke tests; canary in prod

Operational runbooks

• Create a new assistant: add YAML → PR → CI plans → approve → deploy to uat → UAT signoff → deploy to prod
• Change a tool: update tool YAML; bump assistant spec.tools; ensure backward compatibility; run smoke tests
• Incident rollback: revert commit; re-run apply; confirm labels reverted

FAQ

• How do we copy an assistant to another environment? Export from source org (GET), normalize to YAML/JSON, check into Git, then apply to target org via CI using the deployer.
• What exactly is the “config”? The full API payload needed to create/update the assistant, its referenced tools, knowledge bases, and policies. Store it declaratively and resolve environment-specific references at apply time.
• We don’t have built-in versioning yet. What should we do now? Use Git as the source of truth, add labels with commit SHAs to resources, and require CI-only writes to prod.
• How do we handle environment-specific differences (models, endpoints)? Parameterize via environments.yaml and templates; keep the logical spec identical across envs, only vary parameters.

Promotion checklist

• Config: validated and reviewed
• Secrets: present in target environment
• Diff: plan shows expected changes only
• Tests: UAT signoff recorded
• Approvals: change ticket and reviewers complete
• Backups: exported current prod state saved
• Monitoring: alerts enabled for error rate and tool failures

Minimal example: render + apply (Node)

1
import { readFileSync } from 'fs';
2
import yaml from 'js-yaml';
3
import fetch from 'node-fetch';
4
5
const token = process.env.API_TOKEN;
6
const orgId = process.env.ORG_ID;
7
8
async function upsertAssistant(doc) {
9
  const url = `https://api.vendor.com/v1/assistants/${doc.metadata.name}?org=${orgId}`;
10
  const res = await fetch(url, {
11
    method: 'PUT',
12
    headers: {
13
      Authorization: `Bearer ${token}`,
14
      'Content-Type': 'application/json',
15
      'Idempotency-Key': process.env.IDEMPOTENCY_KEY
16
    },
17
    body: JSON.stringify(render(doc))
18
  });
19
  if (!res.ok) throw new Error(`Apply failed: ${res.status} ${await res.text()}`);
20
}
21
22
function render(doc) {
23
  return {
24
    name: doc.metadata.name,
25
    description: doc.metadata.description,
26
    model: doc.spec.model,
27
    tools: doc.spec.tools.map(t => ({ name: t.ref })),
28
    labels: { env: process.env.ENV, sha: process.env.GIT_SHA }
29
  };
30
}
31
32
const doc = yaml.load(readFileSync(process.argv[2], 'utf8'));
33
upsertAssistant(doc)
34
  .then(() => console.log('Applied'))
35
  .catch(e => { console.error(e); process.exit(1); });