Diagnostics & Verification

This page provides a layered UAT verification matrix for validating your Client-Side Defense deployment end-to-end. Each test case follows the infrastructure dependency chain — from DNS resolution through CSD telemetry — so you can systematically prove every component is working correctly.

These commands are the API equivalent of the CSD Console Walkthrough — use them when you need to verify from the terminal, automate monitoring, or demonstrate CSD capabilities without the UI.

Prerequisites

Set up your environment variables as described in API Automation — Environment Setup:

set -a && source .env && set +a

All commands below use the xTOKENx placeholder format. Substitute with your environment variables ($F5XC_API_TOKEN, $F5XC_NAMESPACE, etc.) or use the interactive form at the top of the page.

API Behavior Notes

Note

PUT and DELETE return empty \{\} on HTTP 200 — this is the expected success response, not an error. Only POST returns the created object. List endpoints (e.g., /healthchecks) and individual GET endpoints (e.g., /healthchecks/\{name\}) return different response structures — see API Reference — API Conventions for details on the correct jq paths for each.

Time Range Helpers

Many CSD endpoints require epoch timestamps (seconds since Unix epoch). These one-liners compute start/end times for common ranges.

Cross-platform (macOS + Linux):

# Current time as epoch seconds
NOW=$(date +%s)

# 1 hour ago
START_1H=$(( NOW - 3600 ))

# 24 hours ago
START_24H=$(( NOW - 86400 ))

# 7 days ago
START_7D=$(( NOW - 604800 ))

# 30 days ago
START_30D=$(( NOW - 2592000 ))

Preset	Seconds	Shell expression
1 hour	3,600	$(( $(date +%s) - 3600 ))
24 hours	86,400	$(( $(date +%s) - 86400 ))
7 days	604,800	$(( $(date +%s) - 604800 ))
30 days	2,592,000	$(( $(date +%s) - 2592000 ))

Note

CSD endpoints (/api/shape/csd/) use epoch seconds (e.g., 1710000000). Access log endpoints (/api/data/) use ISO 8601 timestamps (e.g., 2024-03-10T00:00:00Z). The commands in each section use the correct format for that endpoint.

Test Case Format

Each test below follows this structure:

Field	Description
Test ID	Layer number + sequential ID (e.g., DNS-1, TLS-2)
What it proves	The specific infrastructure fact being verified
Command	Ready-to-run curl or dig command
PASS / FAIL	Expected output for healthy vs. unhealthy state
Fix	Link to the relevant setup or troubleshooting section

---

Layer 1: DNS Resolution

DNS is the foundation — if the domain does not resolve to the load balancer VIP, nothing else works.

DNS-1: A Record Resolution

What it proves: The domain resolves to the load balancer VIP IP address.

dig +short xF5XC_DOMAINNAMEx A

Result	Meaning
PASS — VIP IP returned (e.g., 72.19.3.185)	Domain resolves to the LB virtual IP
FAIL — empty response	DNS A record not configured

Fix: API Automation — Step 4: Configure DNS

Tip

Negative DNS cache: If the domain previously returned NXDOMAIN, recursive resolvers may cache that result. Verify against the authoritative nameserver to rule out stale cache: dig +short xF5XC_DOMAINNAMEx A @ns1.f5clouddns.com or use a public resolver like @8.8.8.8.

DNS-2: ACME Challenge Record

What it proves: The ACME challenge record exists for automatic TLS certificate provisioning.

dig +short _acme-challenge.xF5XC_DOMAINNAMEx CNAME
dig +short _acme-challenge.xF5XC_DOMAINNAMEx TXT

Result	Meaning
PASS — CNAME to *.autocerts.ves.volterra.io. (external DNS)	ACME challenge CNAME is in place
PASS — TXT record with domain value (F5 Distributed Cloud (F5 XC) managed DNS)	Platform-managed ACME challenge via TXT record
FAIL — both empty	ACME record not configured; certificate will stay in PreDomainChallengePending

Note

When using F5 XC managed DNS with allow_http_lb_managed_records enabled, the platform auto-creates a TXT record (not a CNAME) in the x-ves-io-managed RR set group. This TXT-based ACME validation is handled internally. When using external DNS, you must manually create a CNAME to *.autocerts.ves.volterra.io.

Fix: For external DNS, create CNAME: _acme-challenge.xF5XC_DOMAINNAMEx → *.autocerts.ves.volterra.io. For F5 XC managed DNS, enable allow_http_lb_managed_records on the DNS zone — see DNS-4. See API Automation — Step 4.

DNS-3: Nameserver Authority

What it proves: Whether F5 XC is the authoritative DNS provider for the root domain.

dig +short NS xF5XC_ROOT_DOMAINx

Result	Meaning
Includes ns1.f5clouddns.com and ns2.f5clouddns.com	F5 XC managed DNS — records can be auto-created
Other nameservers	External DNS — records must be created manually

Fix: API Automation — Detect DNS Authority

DNS-4: F5 XC Managed Records Enabled

What it proves: The F5 XC DNS zone allows automatic record creation for load balancers (F5 XC managed DNS only).

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/dns/namespaces/system/dns_zones/xF5XC_ROOT_DOMAINx" \
  | jq '.spec.primary.allow_http_lb_managed_records'

Result	Meaning
PASS — true	Platform will auto-create A and ACME records for LBs
FAIL — false or null	Managed records disabled; enable via zone update

Caution

A false result is a common misconfiguration when using F5 XC managed DNS. Without managed records enabled, the platform cannot auto-create the A record (pointing to the VIP) or the ACME challenge CNAME (required for TLS certificate provisioning). This causes the load balancer to remain stuck in VIRTUAL_HOST_PENDING_A_RECORD and the certificate to stay in PreDomainChallengePending. Enable managed records on the DNS zone or create the records manually.

Fix: API Automation — Option A: F5 XC Managed DNS

---

Layer 2: TLS Certificate

After DNS resolves, the load balancer must have a valid TLS certificate.

TLS-1: Certificate State

Note

This test applies to the HTTPS LB (${F5XC_LB_NAME}-https) only. The HTTP LB does not have a certificate. In demo environments, TLS-1 is informational — if it shows AutoCertDomainRateLimited, the demo proceeds normally using http:// URLs via the HTTP LB.

What it proves: The automatic TLS certificate has been issued on the HTTPS LB.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-https" \
  | jq '.spec.cert_state'

Result	Meaning
PASS — "CertificateValid" or "AutoCertRenewing"	Certificate is valid and active
WARN — "DomainChallengePending" or "DomainChallengeStarted"	ACME challenge in progress — wait 5–10 minutes
INFO — "AutoCertDomainRateLimited"	Let’s Encrypt rate limit hit — expected in demo environments, does not affect HTTP LB
FAIL — "PreDomainChallengePending"	ACME challenge record missing — see DNS-2

Fix: Ensure the ACME challenge record is in place (CNAME for external DNS, or enable managed records for F5 XC DNS). Certificate provisioning takes 5–10 minutes after the record is configured. If stuck beyond 15 minutes, see Certificate Stuck — Clean Recreation. If rate-limited, the HTTP LB is unaffected.

TLS-2: Certificate Details

What it proves: The certificate covers the expected domain and shows expiry information. Only applicable to the HTTPS LB.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-https" \
  | jq '{
    cert_state: .spec.cert_state,
    auto_cert_info: .spec.auto_cert_info
  }'

Result	Meaning
PASS — auto_cert_info contains dns_records and certificate metadata	Certificate provisioning details available
FAIL — auto_cert_info is null or empty	Certificate not yet provisioned

TLS-3: Live TLS Handshake

What it proves: The TLS certificate is valid and the handshake completes from the client side. Only applicable when the HTTPS LB has a valid certificate.

Note

In demo environments, use curl http://xF5XC_DOMAINNAMEx/ (HTTP, not HTTPS) as the primary verification path. The TLS handshake test is optional — it only works when the HTTPS LB certificate is CertificateValid.

curl -sv "https://xF5XC_DOMAINNAMEx/" 2>&1 \
  | grep -E 'SSL connection|subject:|expire date:|issuer:'

Result	Meaning
PASS — shows SSL connection using TLSv1.3, valid subject and expiry	End-to-end TLS is working
FAIL — connection refused or certificate error	Check DNS resolution and cert state; if rate-limited, use HTTP instead

TLS-4: ACME Record Target from LB

What it proves: The HTTPS load balancer reports the correct ACME target for certificate validation.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-https" \
  | jq '{
    vip_ip: .spec.dns_info[0].ip_address,
    acme_target: .spec.auto_cert_info.dns_records
  }'

Result	Meaning
PASS — vip_ip and acme_target populated	DNS records can be verified against these values
FAIL — null values	LB may not have https_auto_cert configured

---

Layer 3: HTTP Load Balancer

The load balancer must be in a ready state and correctly configured.

LB-1: Operational State

What it proves: The HTTP load balancer (primary) is fully operational and accepting traffic.

Note

HTTP LB is the primary verification target. In demo environments, check the HTTP LB (${F5XC_LB_NAME}-http) first — it reaches VIRTUAL_HOST_READY as soon as DNS resolves, with no certificate dependency. Use curl http://xF5XC_DOMAINNAMEx/ for the primary HTTP-based verification.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '.spec.state'

Result	Meaning
PASS — "VIRTUAL_HOST_READY"	LB is operational
FAIL — "VIRTUAL_HOST_PENDING_A_RECORD"	DNS A record not configured — see DNS-1

Fix: LB Stuck in VIRTUAL_HOST_PENDING_A_RECORD

LB-2: Domain Configuration

What it proves: The HTTP load balancer is configured for the correct domain(s).

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '.spec.domains'

Result	Meaning
PASS — array contains your FQDN (e.g., ["app.example.com"])	LB is configured for the expected domain
FAIL — missing or wrong domain	Update the LB spec with the correct domain

LB-3: CSD Enabled on LB

What it proves: Client-Side Defense is enabled on the load balancer with the correct JS injection policy.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '{
    csd_enabled: (if .spec.client_side_defense then true else false end),
    js_policy: .spec.client_side_defense.policy
  }'

Result	Meaning
PASS — csd_enabled: true with js_insert_all_pages in policy	CSD active with JS injection on all pages
FAIL — csd_enabled: false	CSD not configured on the LB

Fix: API Reference — Enable CSD on a Load Balancer

LB-4: Default Route Pool

What it proves: The load balancer references the correct origin pool with expected weight and priority.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '.spec.default_route_pools[] | {
    pool: .pool.name,
    namespace: .pool.namespace,
    weight: .weight,
    priority: .priority
  }'

Result	Meaning
PASS — shows your origin pool name with weight/priority	LB is routing to the correct backend
FAIL — empty or wrong pool	Update the LB default_route_pools configuration

LB-5: Deployment Status (Per-Site Conditions)

What it proves: The load balancer is deployed and reports healthy conditions across all sites.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '{
    state: .spec.state,
    dns_info: .spec.dns_info,
    host_name: .spec.host_name
  }'

Result	Meaning
PASS — state is VIRTUAL_HOST_READY, dns_info populated	LB fully deployed with VIP assigned
FAIL — pending state or empty dns_info	DNS or certificate issue blocking deployment

Tip

When the LB is not in VIRTUAL_HOST_READY state, check .status[].virtual_host_status.error_description for the same error details shown in the F5 XC console UI:

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '[.status[]? | select(.virtual_host_status != null) | .virtual_host_status | {state, error_description, suggested_action}]'

LB-6: Full LB Summary

What it proves: Comprehensive snapshot of both LB configurations for debugging.

HTTP LB (primary):

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '{
    name: .metadata.name,
    state: .spec.state,
    domains: .spec.domains,
    csd_enabled: (if .spec.client_side_defense then true else false end),
    route_pools: [.spec.default_route_pools[] | .pool.name],
    advertise: (if .spec.advertise_on_public_default_vip then "public_default_vip" else "custom" end)
  }'

HTTPS LB (secondary — includes cert state):

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-https" \
  | jq '{
    name: .metadata.name,
    state: .spec.state,
    cert_state: .spec.cert_state,
    domains: .spec.domains,
    csd_enabled: (if .spec.client_side_defense then true else false end),
    route_pools: [.spec.default_route_pools[] | .pool.name]
  }'

---

Layer 4: Origin Pool

The origin pool defines the backend servers that the load balancer routes traffic to.

OP-1: Origin Pool Configuration

What it proves: The origin pool exists with the correct backend server, port, and LB algorithm.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/origin_pools/xF5XC_ORIGIN_POOLx" \
  | jq '{
    name: .metadata.name,
    origin_servers: [.spec.origin_servers[] | {
      ip: .public_ip.ip,
      labels: .labels
    }],
    port: .spec.port,
    lb_algorithm: .spec.loadbalancer_algorithm,
    endpoint_selection: .spec.endpoint_selection
  }'

Result	Meaning
PASS — shows correct IP, port, and algorithm	Origin pool configured correctly
FAIL — 404 or wrong values	Origin pool missing or misconfigured

Fix: API Automation — Step 2: Create Origin Pool

OP-2: TLS to Origin

What it proves: Whether TLS is configured for the connection between the LB and origin server.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/origin_pools/xF5XC_ORIGIN_POOLx" \
  | jq '{
    tls_config: (if .spec.no_tls then "no_tls (plaintext)" elif .spec.use_tls then "use_tls (encrypted)" else "unknown" end)
  }'

Result	Meaning
no_tls (plaintext)	LB connects to origin over HTTP (expected for Juice Shop demo)
use_tls (encrypted)	LB connects to origin over HTTPS

OP-3: Healthcheck Association

What it proves: The origin pool has the correct healthcheck linked (if applicable).

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/origin_pools/xF5XC_ORIGIN_POOLx" \
  | jq '.spec.healthcheck'

Result	Meaning
PASS — array with healthcheck reference (name, namespace, kind)	Healthcheck is linked
OK — empty array []	No healthcheck (acceptable — CSD does not require one)
FAIL — expected a healthcheck but array is empty	Healthcheck was not found at creation time — see Healthcheck Not Linked

OP-4: Origin Connectivity

What it proves: The origin server is reachable from the client side (does not test LB-to-origin path).

curl -s -o /dev/null -w '%{http_code}' \
  "http://xF5XC_ORIGIN_IPx:xF5XC_ORIGIN_PORTx/"

Result	Meaning
PASS — 200 (or other valid HTTP code)	Origin server is responding
FAIL — 000 or connection refused	Origin server unreachable from this network

Note

This tests direct client-to-origin connectivity. The LB-to-origin path may use a different network path. A failure here does not necessarily mean the LB cannot reach the origin.

---

Layer 5: Health Check

Health checks monitor backend availability. They are optional for CSD but useful for production deployments.

HC-1: Healthcheck Configuration

What it proves: The healthcheck exists with the correct type, path, timing, and thresholds.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/healthchecks/xF5XC_HC_NAMEx" \
  | jq '{
    name: .metadata.name,
    type: (if .spec.http_health_check then "HTTP" elif .spec.tcp_health_check then "TCP" else "unknown" end),
    path: .spec.http_health_check.path,
    expected_status: .spec.http_health_check.expected_status_codes,
    timeout: .spec.timeout,
    interval: .spec.interval,
    unhealthy_threshold: .spec.unhealthy_threshold,
    healthy_threshold: .spec.healthy_threshold
  }'

Result	Meaning
PASS — shows HTTP type with path / and expected 200	Healthcheck configured correctly
FAIL — 404 response	Healthcheck does not exist (may have been skipped — see Phase 1 Step 1)

HC-2: List All Healthchecks

What it proves: Enumerates all healthchecks in the namespace to verify naming and count.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/healthchecks" \
  | jq -r '
    ["NAME", "NAMESPACE", "DESCRIPTION"],
    (.items[] | [
      .name,
      .namespace,
      (.description | if length == 0 then "—" else . end)
    ])
    | @tsv' | column -t

Note

The list endpoint returns summary data only (name, namespace, description). To see full configuration details (type, path, interval, timeout), query an individual healthcheck using HC-1.

---

Layer 6: CSD Configuration

CSD must be enabled at the tenant level and have the JavaScript injection tag configured.

CSD-1: Tenant CSD Status

What it proves: CSD is configured and enabled for the tenant.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/status" \
  | jq '{isConfigured, isEnabled}'

Result	Meaning
PASS — both true	CSD is active for this tenant
FAIL — isConfigured: false	CSD not enabled at tenant level — contact F5 XC administrator
FAIL — isEnabled: false	CSD configured but not active

CSD-2: JS Injection Tag

What it proves: The CSD JavaScript injection tag is generated and ready for injection.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/js_configuration" \
  | jq '{has_script_tag: (.scriptTag | length > 0)}'

Result	Meaning
PASS — has_script_tag: true	JS injection tag is configured
FAIL — has_script_tag: false	No script tag — verify CSD is enabled and a protected domain is registered

CSD-3: JS Tag Content

What it proves: Shows the full script tag for verification or manual injection.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/js_configuration" \
  | jq '.scriptTag'

CSD-4: Protected Domain Registration

What it proves: The root domain is registered as a CSD protected domain on the tenant.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/protected_domains" \
  | jq '.items[] | {name, namespace, description}'

Result	Meaning
PASS — shows an item with a non-empty name	Domain is protected
FAIL — items have empty name and namespace fields	No protected domains registered — see Phase 1 Step 6

Note

The protected domains list endpoint returns summary data. Items with all empty fields indicate no protected domains are configured. The CSD JavaScript tag (CSD-2) is a more reliable indicator that domain protection is active.

CSD-5: Live JS Injection Verification

What it proves: The CSD JavaScript is actually being injected into page responses served by the load balancer.

curl -s "http://xF5XC_DOMAINNAMEx/" \
  | grep -oE '(zeronaught|shape)\.com[^"]*' | head -1

Result	Meaning
PASS — returns a zeronaught.com or shape.com URL fragment (e.g., zeronaught.com/__imp_apg__/js/...)	CSD JavaScript is being injected into pages
FAIL — empty output	JS not injected — check LB-3 and CSD-1

---

Layer 7: Traffic Verification

Verify that live traffic is reaching the load balancer and being processed correctly.

TV-1: Request Count (Last 24 Hours)

What it proves: Traffic is reaching the load balancer. Zero results means no traffic is arriving.

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d '{
    "start_time": "'"$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-24H +%Y-%m-%dT%H:%M:%SZ)"'",
    "end_time": "'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'"
  }' \
  "xF5XC_API_URLx/api/data/namespaces/xF5XC_NAMESPACEx/access_logs/aggregation" \
  | jq '{total_requests: .total_hits}'

Result	Meaning
PASS — total_requests is a non-zero string (e.g., "380")	Traffic is flowing through the LB
FAIL — "0" or no data	No traffic reaching the LB — check DNS-1 and LB-1

Note

The total_hits value is returned as a string (e.g., "380"), not an integer. The namespace is inferred from the URL path — do not include it in the request body.

TV-2: Status Code Distribution

What it proves: The breakdown of response status codes reveals error patterns.

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d '{
    "start_time": "'"$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-24H +%Y-%m-%dT%H:%M:%SZ)"'",
    "end_time": "'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'",
    "sort": "DESCENDING",
    "limit": 100
  }' \
  "xF5XC_API_URLx/api/data/namespaces/xF5XC_NAMESPACEx/access_logs" \
  | jq -r '
    [.logs[] | fromjson | .rsp_code_class]
    | group_by(.) | map({class: .[0], count: length})
    | sort_by(-.count)
    | ["STATUS_CLASS", "COUNT"], (.[] | [.class, .count])
    | @tsv' | column -t

Note

The aggregation endpoint does not support terms bucketing — aggs always returns empty. This command fetches the most recent 100 log entries and computes the distribution client-side. Each log entry is a JSON string that must be parsed with fromjson.

Expected output for a healthy site:

STATUS_CLASS                  COUNT
2xx                           82
downstream_remote_disconnect  18

Result	Meaning
PASS — majority 2xx	Site is serving successful responses
WARN — high 4xx count	Client errors (bad URLs, missing resources)
FAIL — high 5xx count	Server errors — check origin server health

The downstream_remote_disconnect class indicates the client closed the connection before the response completed (common for long-polling or WebSocket upgrade requests).

TV-3: Recent Request Samples

What it proves: Individual requests are being logged with correct fields.

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d '{
    "start_time": "'"$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-1H +%Y-%m-%dT%H:%M:%SZ)"'",
    "end_time": "'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'",
    "sort": "DESCENDING",
    "limit": 10
  }' \
  "xF5XC_API_URLx/api/data/namespaces/xF5XC_NAMESPACEx/access_logs" \
  | jq -r '
    ["TIMESTAMP", "METHOD", "PATH", "STATUS", "SRC_IP"],
    (.logs[] | fromjson | [.["@timestamp"], .method, .req_path, .rsp_code, .src_ip])
    | @tsv' | column -t

Caution

Each entry in the .logs[] array is a JSON string, not a parsed object. You must pipe through fromjson in jq before accessing fields (e.g., .logs[] | fromjson | .method).

TV-4: CSD JS Injection in Access Logs

What it proves: Access logs confirm CSD JavaScript is being injected into responses.

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d '{
    "start_time": "'"$(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-1H +%Y-%m-%dT%H:%M:%SZ)"'",
    "end_time": "'"$(date -u +%Y-%m-%dT%H:%M:%SZ)"'",
    "sort": "DESCENDING",
    "limit": 20
  }' \
  "xF5XC_API_URLx/api/data/namespaces/xF5XC_NAMESPACEx/access_logs" \
  | jq '[.logs[] | fromjson | select(.csd_js_injection == "true")] | length as $injected |
    {injected_count: $injected, total_sampled: 20}'

Result	Meaning
PASS — injected_count > 0	CSD JS is being injected into page responses
FAIL — injected_count: 0	JS not injected — check CSD-1 and LB-3

Note

The csd_js_injection field is only present on log entries where injection occurred. In testing, 40% of sampled log entries had csd_js_injection: "true" (HTML page requests), while 60% had the field absent (static assets like .ico, .js, .css files and WebSocket connections). HTML page requests should show injection; API calls and static asset requests typically will not.

TV-5: End-to-End Connectivity Test

What it proves: A complete request flows from client through DNS, LB, and origin and returns a valid response.

curl -sv "http://xF5XC_DOMAINNAMEx/" 2>&1 \
  | grep -E 'Connected to|< HTTP|< content-type'

Result	Meaning
PASS — shows connection, HTTP 200, and content-type	Full stack is operational
FAIL — connection refused or DNS error	Start debugging at Layer 1: DNS

Tip

For HTTPS end-to-end testing (only when the HTTPS LB certificate is valid), use curl -sv "https://xF5XC_DOMAINNAMEx/" and check for SSL connection in the output.

---

Layer 8: CSD Telemetry & Domain Policy

These commands query the same data displayed in the CSD Console dashboard, script list, form fields, and network views.

TEL-1: Script Inventory

What it proves: CSD is detecting and cataloging scripts running on the protected domain.

NOW=$(date +%s)
START=$(( NOW - 604800 ))

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d "{
    \"startTime\": \"${START}\",
    \"endTime\": \"${NOW}\"
  }" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/scripts" \
  | jq -r '
    ["SCRIPT", "RISK", "STATUS", "FIELDS", "USERS"],
    (.scripts[] | [
      (.script_name | if length > 50 then .[:47] + "..." else . end),
      .risk_level,
      .status,
      (.form_fields_read // 0),
      (.affected_users_count // 0)
    ])
    | @tsv' | column -t

Result	Meaning
PASS — scripts listed with risk levels	CSD is actively monitoring scripts
FAIL — empty array	See Troubleshooting: Empty Scripts Array

Tip

Script names are full URLs and can be very long. The command above truncates names to 50 characters for terminal readability. Remove the truncation filter to see full URLs.

TEL-2: Detected Domains

What it proves: CSD has detected script source domains and classified them by status.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/detected_domains" \
  | jq '{
    summary: {
      action_needed: .domain_summary.actionNeededCount.count,
      mitigated: .domain_summary.mitigatedDomains.count,
      allowed: .domain_summary.allowedDomains.count,
      total: .domain_summary.totalDomains.count
    },
    domains: [.domains_list[] | {
      domain: .domain,
      category: .category,
      status: .status,
      first_seen: (.firstSeenDate | tonumber | todate),
      latest_seen: (.latestSeenDate | tonumber | todate)
    }]
  }'

Result	Meaning
PASS — total > 0 with domains listed	CSD is tracking script domains
WARN — action_needed > 0	Some domains need review
FAIL — empty response	No telemetry data — check CSD-5

TEL-3: Domain Policy State

What it proves: Shows the distribution of allowed vs. mitigated domains and whether policy is consistent.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/detected_domains" \
  | jq '{
    action_needed: .domain_summary.actionNeededCount.count,
    mitigated: .domain_summary.mitigatedDomains.count,
    allowed: .domain_summary.allowedDomains.count,
    total: .domain_summary.totalDomains.count,
    domains_needing_action: [.domains_list[] | select(.status == "AN") | .domain]
  }'

Result	Meaning
PASS — action_needed: 0	All domains have been reviewed and classified
WARN — action_needed > 0	Domains listed in domains_needing_action require review

TEL-4: Form Field Inventory

What it proves: CSD has detected form fields that scripts are reading, with sensitivity classification.

NOW=$(date +%s)
START=$(( NOW - 604800 ))

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/formFields?startTime=${START}&endTime=${NOW}" \
  | jq -r '
    ["FIELD", "SENSITIVITY", "SCRIPTS"],
    (.form_fields[] | [
      .name,
      .analysis,
      (.scripts_count // 0)
    ])
    | @tsv' | column -t

Result	Meaning
PASS — form fields listed with sensitivity	CSD is tracking form field access
INFO — empty array	No form fields detected (expected if no forms on the site)

TEL-5: Script Deep Dive

What it proves: Detailed information about a specific script including risk, behaviors, and network interactions.

First, get a script ID from TEL-1, then query its details:

SCRIPT_ID="your-script-id"

# Overview (risk level, type, source domain)
curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/scripts/${SCRIPT_ID}/dashboard" \
  | jq '{script_name, risk_level, type, source_domain, status}'

# Behaviors over time
curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/scripts/${SCRIPT_ID}/behaviors" \
  | jq '.behaviors'

# Network interactions (domains the script communicates with)
curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/scripts/${SCRIPT_ID}/networkInteractions" \
  | jq '.network_interactions'

TEL-6: Affected Users

What it proves: Lists users impacted by a specific script, showing the scope of exposure.

SCRIPT_ID="your-script-id"
NOW=$(date +%s)
START=$(( NOW - 604800 ))

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d "{
    \"startTime\": \"${START}\",
    \"endTime\": \"${NOW}\"
  }" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/scripts/${SCRIPT_ID}/affectedUsers" \
  | jq -r '
    ["IP_ADDRESS", "DEVICE_ID", "GEO", "CHANNEL", "USER_AGENT"],
    (.affected_users[] | [
      .ip_address,
      (.device_id | if length > 12 then .[:12] + "..." else . end),
      .geolocation,
      .channel,
      (.user_agent | if length > 30 then .[:27] + "..." else . end)
    ])
    | @tsv' | column -t

TEL-7: Script Count by Risk Level

What it proves: Aggregated risk distribution across all detected scripts.

NOW=$(date +%s)
START=$(( NOW - 604800 ))

curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d "{
    \"startTime\": \"${START}\",
    \"endTime\": \"${NOW}\"
  }" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/scripts" \
  | jq '[.scripts[] | .risk_level] | group_by(.) | map({risk_level: .[0], count: length}) | sort_by(-.count)'

Result	Meaning
PASS — all No Risk	No risky scripts detected
WARN — Low Risk or High Risk present	Review flagged scripts in TEL-5

TEL-8: Telemetry Freshness

What it proves: CSD telemetry data is recent, confirming active monitoring.

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/detected_domains" \
  | jq '{
    total_domains: .domain_summary.totalDomains.count,
    latest_update: (.domain_summary.totalDomains.lastUpdated // "unknown"),
    most_recent_domain: (.domains_list | sort_by(.latestSeenDate) | last | {
      domain: .domain,
      latest_seen: (.latestSeenDate | tonumber | todate)
    })
  }'

Result	Meaning
PASS — latest_seen is within the last 24 hours	Telemetry is actively collecting data
WARN — latest_seen is older than 24 hours	Traffic may have stopped or CSD processing is delayed

---

Full-Stack Dashboard

Run a single command that checks all critical health indicators across every layer and produces a summary table:

echo "=== CSD Verification Dashboard ==="
echo ""

# Layer 1: DNS
DNS_A=$(dig +short xF5XC_DOMAINNAMEx A | head -1)
DNS_ACME=$(dig +short _acme-challenge.xF5XC_DOMAINNAMEx CNAME | head -1)

# Layers 2-3: LB + TLS
LB=$(curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http")
LB_HTTPS=$(curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-https")

LB_STATE=$(echo "$LB" | jq -r '.spec.state // "UNKNOWN"')
CERT_STATE=$(echo "$LB_HTTPS" | jq -r '.spec.cert_state // "UNKNOWN"')
CSD_ON_LB=$(echo "$LB" | jq -r 'if .spec.client_side_defense then "ENABLED" else "DISABLED" end')
DOMAINS=$(echo "$LB" | jq -r '.spec.domains | join(", ")')
ROUTE_POOL=$(echo "$LB" | jq -r '[.spec.default_route_pools[] | .pool.name] | join(", ")')

# Layer 6: CSD status
CSD=$(curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/status")

CSD_CONFIGURED=$(echo "$CSD" | jq -r '.isConfigured // false')
CSD_ENABLED=$(echo "$CSD" | jq -r '.isEnabled // false')

# Layer 6: JS config
JS_TAG=$(curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/js_configuration" \
  | jq -r 'if (.scriptTag | length) > 0 then "PRESENT" else "MISSING" end')

# Layer 7: Traffic (last 24h)
TRAFFIC=$(curl -s -X POST \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  -H "Content-Type: application/json" \
  -d "{
    \"start_time\": \"$(date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ 2>/dev/null || date -u -v-24H +%Y-%m-%dT%H:%M:%SZ)\",
    \"end_time\": \"$(date -u +%Y-%m-%dT%H:%M:%SZ)\"
  }" \
  "xF5XC_API_URLx/api/data/namespaces/xF5XC_NAMESPACEx/access_logs/aggregation" \
  | jq -r '.total_hits // "0"')

printf "%-28s %s\n" "CHECK" "STATUS"
printf "%-28s %s\n" "----------------------------" "----------------------------"
printf "%-28s %s\n" "[DNS] A Record" "${DNS_A:-NOT_FOUND}"
printf "%-28s %s\n" "[DNS] ACME CNAME" "${DNS_ACME:-NOT_FOUND}"
printf "%-28s %s\n" "[TLS] Certificate" "$CERT_STATE"
printf "%-28s %s\n" "[LB] State" "$LB_STATE"
printf "%-28s %s\n" "[LB] Domains" "$DOMAINS"
printf "%-28s %s\n" "[LB] Route Pool" "$ROUTE_POOL"
printf "%-28s %s\n" "[LB] CSD on LB" "$CSD_ON_LB"
printf "%-28s %s\n" "[CSD] Configured (tenant)" "$CSD_CONFIGURED"
printf "%-28s %s\n" "[CSD] Enabled" "$CSD_ENABLED"
printf "%-28s %s\n" "[CSD] JS Script Tag" "$JS_TAG"
printf "%-28s %s\n" "[Traffic] Requests (24h)" "$TRAFFIC"

---

Access Log Field Reference

Caution

Each entry in the .logs[] array is a JSON string, not a parsed object. You must pipe through fromjson in jq before accessing fields (e.g., .logs[] | fromjson | .method).

Field	Type	Description
@timestamp	string	Request timestamp (ISO 8601). Note the @ prefix — access with .["@timestamp"] in jq
method	string	HTTP method (GET, POST, etc.)
req_path	string	Request URI path
rsp_code	string	HTTP response status code as a string (e.g., "200", "404")
rsp_code_class	string	Status code class (2xx, 3xx, 4xx, 5xx, or downstream_remote_disconnect)
src_ip	string	Client source IP address
dst_ip	string	Destination (VIP) IP address
domain	string	Request Host header value
user_agent	string	Client User-Agent string
rsp_size	string	Response body size in bytes (returned as string)
req_size	string	Request body size in bytes (returned as string)
duration_with_data_tx_delay	string	Total request duration in seconds (returned as string, e.g., "0.024219")
csd_js_injection	string	"true" when CSD JavaScript was injected (only present when active)

CSD Telemetry Field Reference

Field	Endpoint	Description
isConfigured	status	CSD enabled at tenant level
isEnabled	status	CSD active for this namespace
scripts[]	scripts	Array of detected script objects
.script_name	scripts	Full URL of the JavaScript file
.risk_level	scripts	Risk level (No Risk, Low Risk, High Risk)
.status	scripts	AN (Action Needed) or NA (No Action Needed)
.form_fields_read	scripts	Number of form fields the script reads
.affected_users_count	scripts	Number of unique users/sessions affected
domain_summary	detected_domains	Counts by status: .actionNeededCount.count, .mitigatedDomains.count, .allowedDomains.count, .totalDomains.count (each has .count and .lastUpdated)
domains_list[]	detected_domains	Array of detected domain objects with .domain, .status, .category, .firstSeenDate, .latestSeenDate (epoch seconds as strings)
form_fields[]	formFields	Array of detected form field objects
.analysis	formFields	Sensitivity classification (Sensitive, Not Sensitive)

Troubleshooting

Zero Access Logs

If TV-1 returns 0:

1. Check DNS resolution — verify the domain resolves to the LB VIP:

   dig +short xF5XC_DOMAINNAMEx A

   If empty, DNS is not configured. See API Automation — Step 4.

2. Check LB state — the load balancer must be VIRTUAL_HOST_READY:

   curl -s \
     -H "Authorization: APIToken xF5XC_API_TOKENx" \
     "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
     | jq '.spec.state'

3. Send a test request — generate traffic to confirm end-to-end connectivity:

   curl -sv "http://xF5XC_DOMAINNAMEx/" 2>&1 | head -20

   Look for a successful HTTP response. Use https:// only if the HTTPS LB certificate is valid.

Managed Records Disabled on DNS Zone

If DNS-4 returns false, automatic record creation is disabled even though F5 XC is the authoritative DNS provider. This is a common misconfiguration that causes both the A record and ACME CNAME to be missing, which blocks the load balancer from reaching VIRTUAL_HOST_READY and the certificate from being issued.

To enable managed records, update the DNS zone configuration to set allow_http_lb_managed_records: true. See API Automation — Option A: F5 XC Managed DNS for the API call.

LB Stuck in VIRTUAL_HOST_PENDING_A_RECORD

The load balancer is waiting for a DNS A record pointing to its VIP. See LB Stuck in VIRTUAL_HOST_PENDING_A_RECORD for detailed resolution steps.

Certificate Stuck in PreDomainChallengePending

The automatic TLS certificate requires an ACME challenge record. The method depends on your DNS provider:

F5 XC Managed DNS: Enable allow_http_lb_managed_records on the DNS zone (DNS-4). The platform auto-creates a TXT-based ACME challenge record in the x-ves-io-managed RR set group. If the certificate remains stuck after enabling managed records, delete and recreate the load balancer to trigger a fresh certificate request.

External DNS: Create a CNAME record at your DNS provider:

_acme-challenge.xF5XC_DOMAINNAMEx  CNAME  *.autocerts.ves.volterra.io

Verify the record exists:

dig +short _acme-challenge.xF5XC_DOMAINNAMEx CNAME
dig +short _acme-challenge.xF5XC_DOMAINNAMEx TXT

If both are empty, the ACME record is not configured. Certificate provisioning takes 5–10 minutes after the record is in place. Check the LB error status for specific ACME validation errors:

curl -s \
  -H "Authorization: APIToken xF5XC_API_TOKENx" \
  "xF5XC_API_URLx/api/config/namespaces/xF5XC_NAMESPACEx/http_loadbalancers/xF5XC_LB_NAMEx-http" \
  | jq '[.status[]? | select(.virtual_host_status != null) | .virtual_host_status.error_description]'

CSD isConfigured is false

CSD must be enabled at the tenant level by an F5 XC administrator. This is a tenant-wide setting that cannot be configured via the namespace API. Contact your administrator to enable Client-Side Defense.

Empty Scripts Array

If TEL-1 returns an empty array:

1. Check the protected domain — CSD monitors scripts only on registered domains:

   curl -s \
     -H "Authorization: APIToken xF5XC_API_TOKENx" \
     "xF5XC_API_URLx/api/shape/csd/namespaces/xF5XC_NAMESPACEx/protected_domains" \
     | jq '.items[] | {name, namespace}'

2. Verify JS injection — confirm the CSD script tag is being injected into pages:

   curl -s "http://xF5XC_DOMAINNAMEx/" | grep -oE '(zeronaught|shape)\.com[^"]*' | head -1

   If no match, CSD JavaScript is not being injected. Check that the LB has client_side_defense enabled.

3. Allow processing time — after first enabling CSD or registering a new protected domain, script detection can take 5–15 minutes. Generate traffic to the site and wait before re-checking.

Quick Reference

Time Range Presets

Period	Epoch offset	ISO 8601 (Linux)	ISO 8601 (macOS)
1 hour	$(( $(date +%s) - 3600 ))	date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ	date -u -v-1H +%Y-%m-%dT%H:%M:%SZ
24 hours	$(( $(date +%s) - 86400 ))	date -u -d '24 hours ago' +%Y-%m-%dT%H:%M:%SZ	date -u -v-24H +%Y-%m-%dT%H:%M:%SZ
7 days	$(( $(date +%s) - 604800 ))	date -u -d '7 days ago' +%Y-%m-%dT%H:%M:%SZ	date -u -v-7d +%Y-%m-%dT%H:%M:%SZ
30 days	$(( $(date +%s) - 2592000 ))	date -u -d '30 days ago' +%Y-%m-%dT%H:%M:%SZ	date -u -v-30d +%Y-%m-%dT%H:%M:%SZ

Verification Layer Summary

Layer	Tests	What it covers
1. DNS Resolution	DNS-1 through DNS-4	A record, ACME CNAME, nameserver authority, managed records
2. TLS Certificate	TLS-1 through TLS-4	Cert state, cert details, live handshake, ACME target
3. HTTP Load Balancer	LB-1 through LB-6	State, domains, CSD flag, route pools, deployment, summary
4. Origin Pool	OP-1 through OP-4	Config, TLS mode, HC association, origin connectivity
5. Health Check	HC-1 through HC-2	HC config, list all HCs
6. CSD Configuration	CSD-1 through CSD-5	Tenant status, JS tag, protected domains, live injection
7. Traffic Verification	TV-1 through TV-5	Request count, status codes, samples, JS in logs, E2E test
8. CSD Telemetry	TEL-1 through TEL-8	Scripts, domains, policy, form fields, deep dive, users, risk, freshness

Endpoint Summary

Diagnostic	Endpoint	Method	Time Format
Request count	/api/data/namespaces/\{ns\}/access_logs/aggregation	POST	ISO 8601
Status codes	/api/data/namespaces/\{ns\}/access_logs	POST	ISO 8601
Recent requests	/api/data/namespaces/\{ns\}/access_logs	POST	ISO 8601
LB state	/api/config/namespaces/\{ns\}/http_loadbalancers/\{name\}	GET	None
Origin pool	/api/config/namespaces/\{ns\}/origin_pools/\{name\}	GET	None
Healthcheck	/api/config/namespaces/\{ns\}/healthchecks/\{name\}	GET	None
DNS zone	/api/config/dns/namespaces/system/dns_zones/\{zone\}	GET	None
CSD status	/api/shape/csd/namespaces/\{ns\}/status	GET	None
JS configuration	/api/shape/csd/namespaces/\{ns\}/js_configuration	GET	None
Protected domains	/api/shape/csd/namespaces/\{ns\}/protected_domains	GET	None
Script list	/api/shape/csd/namespaces/\{ns\}/scripts	POST	Epoch seconds
Detected domains	/api/shape/csd/namespaces/\{ns\}/detected_domains	GET	None
Form fields	/api/shape/csd/namespaces/\{ns\}/formFields	GET	Epoch seconds (query params)
Script details	/api/shape/csd/namespaces/\{ns\}/scripts/\{id\}/dashboard	GET	None
Script behaviors	/api/shape/csd/namespaces/\{ns\}/scripts/\{id\}/behaviors	GET	None
Script network	/api/shape/csd/namespaces/\{ns\}/scripts/\{id\}/networkInteractions	GET	None
Affected users	/api/shape/csd/namespaces/\{ns\}/scripts/\{id\}/affectedUsers	POST	Epoch seconds