Development Guide

This guide describes the development workflow for the F5 XC API Enrichment Pipeline.

Quick Start

First-Time Setup

# Clone the repository
git clone https://github.com/f5xc-salesdemos/api-specs-enriched.git
cd api-specs-enriched

# Create virtual environment and install dependencies
make install

# Install pre-commit hooks
make pre-commit-install

# Download specs and run pipeline
make build

# Preview documentation locally
make serve

Daily Development

# Quick rebuild (uses cached specs)
make rebuild

# Run all quality checks
make pre-commit-run

# Preview documentation
make serve

Architecture Overview

Two-Folder Design

┌─────────────────────┐     ┌─────────────────────────────┐
│   specs/original/   │────▶│   docs/specifications/api/  │
│   (READ-ONLY)       │     │   (GENERATED)               │
│   - Downloaded      │     │   - Domain specs            │
│   - Gitignored      │     │   - Master spec             │
│   - ETag cached     │     │   - GitHub Pages            │
└─────────────────────┘     └─────────────────────────────┘

Pipeline Flow

Download (ETag) → Enrich → Normalize → Merge → Lint → Serve
     │              │          │         │       │
     ▼              ▼          ▼         ▼       ▼
 270 specs     Branding    Fix refs   23 domains  Spectral
             + Grammar    + Types    + Master     rules

Directory Structure

Directory	Purpose	Git Status
specs/original/	F5 source specs	Gitignored
specs/discovered/	API discovery output	Tracked (openapi.json, session.json)
docs/specifications/api/	Generated domain specs	Gitignored
scripts/	Python pipeline scripts	Tracked
config/	Pipeline configuration	Tracked
reports/	Generated reports	Gitignored

Workflow Patterns

1. Discovery Workflow

Prerequisites:

• VPN connection to F5 XC environment
• Valid API credentials

# Set credentials
export F5XC_API_URL="https://your-tenant.console.ves.volterra.io/api"
export F5XC_API_TOKEN="your-api-token"

# Run discovery
make discover

# View results
jq '.statistics' specs/discovered/session.json

# Commit for CI/CD
make push-discovery

Discovery captures:

• Actual required/optional fields
• Enum values from live responses
• Default values
• Pattern validations
• Response examples

2. Release Workflow

Releases are automated:

1. Daily schedule (6 AM UTC) or push to main triggers workflow
2. ETag check determines if F5 specs changed
3. Pipeline processes and enriches specs
4. Version calculated from git tags: git describe --tags --abbrev=0
5. Direct commit + tag created (no version bump PR)
6. GitHub Release created with changelog
7. Documentation deployed to GitHub Pages

Version Bump Rules:

Condition	Bump Type	Example
[major] in commit	Major	1.0.0 → 2.0.0
BREAKING CHANGE in commit	Major	1.0.0 → 2.0.0
New domain spec added	Minor	1.0.0 → 1.1.0
Any other change	Patch	1.0.0 → 1.0.1

3. Development Workflow

# Create feature branch
git checkout -b feature/my-change

# Make changes to config or scripts

# Test locally
make pipeline
make lint

# Commit (pre-commit hooks run automatically)
git add .
git commit -m "feat: add new enrichment rule"

# Push and create PR
git push -u origin feature/my-change
gh pr create

Discovery Deep Dive

What is Discovery?

Discovery explores the live F5 XC API to find:

• Undocumented constraints: Required fields not marked in OpenAPI
• Actual enum values: Real values seen in production
• Default behaviors: Server-applied values when fields are omitted
• Response patterns: Actual data shapes

Discovery Configuration

config/discovery.yaml

discovery:
  exploration:
    namespaces:
      - "system"
      - "shared"
    methods:
      - "GET"
      - "OPTIONS"
    max_endpoints_per_run: 500

  schema_inference:
    sample_size: 3
    detect_patterns: true
    detect_constraints: true

Discovery Extensions

Discovery adds x-discovered-* extensions to specs:

{
  "properties": {
    "name": {
      "type": "string",
      "x-discovered-required": true,
      "x-discovered-pattern": "^[a-z][a-z0-9-]*$",
      "x-discovered-examples": ["my-app", "prod-lb"]
    }
  }
}

Constraint Analysis Report

reports/constraint-analysis.md

make constraint-report

Release Process

Release Package Contents

f5xc-api-specs-v1.0.14.zip
├── openapi.json       # Master combined spec
├── openapi.yaml       # YAML format
├── domains/           # Individual domain specs
│   ├── api_security.json
│   ├── load_balancer.json
│   └── ...
├── index.json         # Metadata
├── CHANGELOG.md       # Release notes
└── README.md          # Usage instructions

Manual Workflow Trigger

gh workflow run sync-and-enrich.yml
gh run watch

Configuration Guide

Enrichment Configuration

config/enrichment.yaml

enrichment:
  branding:
    replacements:
      "Volterra": "F5 Distributed Cloud"
      "VES": "F5 XC"

  acronyms:
    API: "Application Programming Interface"
    DNS: "Domain Name System"

  grammar:
    enabled: true
    language_tool: true

Normalization Configuration

config/normalization.yaml

normalization:
  orphan_refs:
    fix: true
    remove_if_unresolvable: true

  empty_operations:
    remove: true

  type_standardization:
    enabled: true

Spectral Linting Rules

config/spectral.yaml

extends:
  - spectral:oas

rules:
  operation-operationId: error
  operation-tags: warn

OpenAPI Extensions

The enrichment pipeline uses vendor extensions to embed validation and default value metadata.

Extension Categories

Extension	Meaning	Implication
x-f5xc-server-default	The F5 XC API server applies this value when the field is omitted	Field is optional; omitting it produces the documented default behavior
x-f5xc-recommended-value	The F5 XC web console pre-populates this value for new resources	Field has no server default but this value represents typical configuration
x-f5xc-recommended-oneof-variant	The F5 XC console preselects this OneOf variant	Identifies the typical choice when multiple mutually exclusive options exist
x-f5xc-conflicts-with	Lists other properties that cannot be used together with this field	Property is part of a OneOf group; only one of the conflicting properties can be specified

Required Field Extensions

Extension	Source	Purpose
x-ves-required: "true"	F5 XC original spec	Field requires non-zero/non-empty value
x-f5xc-required-for	Enrichment pipeline	Context-specific required status

x-f5xc-required-for contexts:

• create: Required when creating the resource
• update: Required when updating the resource
• minimum_config: Required for minimum viable configuration

Default Value and OneOf Extensions

Extension	Purpose
x-f5xc-server-default: true	Marks server-applied defaults
x-f5xc-recommended-value	F5 XC console pre-populated value
x-f5xc-recommended-oneof-variant	Recommended OneOf variant
x-f5xc-conflicts-with	Mutual exclusivity with other OneOf properties

x-f5xc-server-default

Type: boolean

When true, indicates the accompanying default value is enforced by the F5 XC API server. Fields with this extension can be safely omitted from API requests — the server applies the default automatically.

use_http2:
  type: boolean
  default: false
  x-f5xc-server-default: true

x-f5xc-recommended-value

Type: any (matches field type)

Specifies a value that the F5 XC web console uses as a pre-populated default. This value is not server-enforced but represents the typical starting configuration for new resources created via the console.

timeout:
  type: integer
  x-f5xc-recommended-value: 3

x-f5xc-recommended-oneof-variant

Type: object (map of group name to variant name)

For schemas with mutually exclusive field groups, identifies which variant is the default or most common choice. The key is the OneOf group name and the value is the recommended variant field name.

healthcheckCreateSpecType:
  type: object
  x-f5xc-recommended-oneof-variant:
    health_check: "http_health_check"

x-f5xc-conflicts-with

Type: array of strings

Added: v3.2.0 (Issue #494)

Declares mutual exclusivity relationships between OneOf group members. Auto-derived from x-ves-oneof-field-* extensions. This enables downstream tools (Terraform, CLI, MCP) to validate conflicts at schema level rather than runtime.

host_header:
  type: string
  x-f5xc-conflicts-with: ["use_origin_server_name"]

use_origin_server_name:
  type: object
  x-f5xc-conflicts-with: ["host_header"]

Use cases:

• Terraform providers can generate validation rules
• CLI tools can warn about conflicting field combinations
• AI assistants can generate correct OneOf configurations
• IDE extensions can provide conflict-aware autocompletion

Source: Auto-derived from F5 native x-ves-oneof-field-* extensions during pipeline enrichment.

Validation Rule Implications

The enricher inspects x-ves-validation-rules to infer required status:

Rule	Implication
ves.io.schema.rules.message.required: "true"	Field is required
ves.io.schema.rules.uint32.gte: N	If N >= 1 and no server default, field is required
ves.io.schema.rules.repeated.min_items: N	If N >= 1, array requires at least N items
ves.io.schema.rules.string.min_bytes: N	If N >= 1, string requires at least N bytes

Resources with Server-Applied Defaults

Some resources accept empty specs because the server applies defaults:

Resource	Server-Applied Defaults
app_firewall	monitoring: \{\}, default_detection_settings: \{\}
rate_limiter	limits: [], user_identification: []
api_definition	swagger_specs: [], default api_groups
healthcheck	jitter: 0, jitter_percent: 0, nested http_health_check defaults

For these resources, x-f5xc-required-for.create may be false even when x-ves-required is true.

Default Value Patterns

Pattern	Type	Example
\{\}	Empty object (choice selection)	monitoring: \{\}
[]	Empty array	expected_status_codes: []
0	Numeric	jitter: 0
""	String	expected_response: ""
false	Boolean	use_http2: false

Adding Discovered Defaults

1. Create resource in F5 XC with minimal spec via API
2. Read back the resource to see server-applied values
3. Document defaults in config/discovered_defaults.yaml
4. Run pipeline: make pipeline
5. Verify with: jq '.components.schemas | to_entries[] | select(.key | contains("resource_name"))'

Troubleshooting

Pre-commit Takes Too Long

The pipeline runs on every commit to ensure consistency. For work-in-progress commits:

git commit --no-verify -m "WIP: work in progress"
# Run before final commit: make pre-commit-run

Discovery Fails

# Check VPN
ping your-tenant.console.ves.volterra.io

# Check credentials
echo $F5XC_API_TOKEN | head -c 10

# Check API URL format
echo $F5XC_API_URL
# Format: https://tenant.console.ves.volterra.io/api

Lint Errors on Generated Specs

# Check lint report
cat reports/lint-report.json | jq '.errors'

Fix issues in enrichment/normalization config, not the output files.

Version System

Version is derived from git tags (e.g., v2.0.38), eliminating race conditions that caused merge conflicts on concurrent PRs.

# Version is calculated dynamically from git tags
from scripts.utils.version_calculator import get_version_from_tags
version = get_version_from_tags()  # Returns "2.0.38"

Missing Specs After Clone

Generated specs are gitignored:

make build  # Downloads and generates everything

Large File Blocked

Add exclusion to .pre-commit-config.yaml:

- id: check-added-large-files
  exclude: ^path/to/large/file\.json$

Quick Reference

Make Targets

Target	Description
make build	Full build (download + pipeline)
make rebuild	Quick rebuild (skip download)
make download	Download specs (ETag cached)
make download-force	Force download
make pipeline	Run enrichment pipeline
make lint	Spectral linting
make serve	Local documentation server
make discover	API discovery (VPN required)
make push-discovery	Commit discovery data
make clean	Remove generated files
make pre-commit-run	Run all quality checks

Environment Variables

Variable	Purpose
F5XC_API_URL	F5 XC tenant API URL
F5XC_API_TOKEN	API authentication token

Key Files

File	Purpose
.etag	Last downloaded ETag
CHANGELOG.md	Auto-generated changelog
config/enrichment.yaml	Enrichment rules
config/normalization.yaml	Normalization rules
config/discovery.yaml	Discovery settings
config/spectral.yaml	Linting rules
scripts/utils/version_calculator.py	Tag-based version calculation

Note: Version is derived from git tags (e.g., v2.0.38), not from a .version file.