Skip to content

Environment B — Air-Gapped Build & Release (v2.0.0)

Scope: OpenStack 2024.1 image builds using Kolla + Zuul in a fully disconnected LAN Status: v2.0.0 baseline (manual imports, manual build trigger)

How to read and implement (numbered path)

Follow this order to stand up the air-gapped environment without missing dependencies between hosts:

  1. Baseline connectivity: Air-Gapped Implementation 1 — Network Connectivity Requirements
  2. Control plane services on VM2:
    1. Air-Gapped Implementation 2 — Gerrit Stack — VM2
    2. Air-Gapped Implementation 3 — Zuul Stack — VM2
  3. Runner host on VM3: Air-Gapped Implementation 4 — Nodepool Static Node — VM3 Runner
  4. Tenant + CI wiring (Gerrit ⇄ Zuul): Air-Gapped Implementation 5 — Repository setup and CI trigger

Each guide title is numbered to reinforce the execution order. Runbooks referenced inside each step should be executed before moving to the next number.

1) Purpose

Environment B is the fully air-gapped environment responsible for building OpenStack 2024.1 container images using offline-delivered artifacts only. There is no internet access. All inputs (Docker images, Compose files, required OS packages) are delivered offline.

2) High-level responsibilities (v2.0.0)

  • Host the CI control plane (Gerrit + Zuul + ZooKeeper + Postgres).
  • Host internal artifact endpoints (Nexus for artifacts and images).
  • Execute Zuul jobs to build Kolla images using a controlled runner.
  • Produce and retain build outputs (images, logs/manifests) within the LAN.

Non-goals for v2.0.0:

  • Automated upstream polling, automated artifact refresh, automated promotions.
  • Internet-backed dependency retrieval of any form (must fail by design).

3) Topology (3 VMs)

Environment B is deployed on three VMs in the air-gapped LAN.

VM1 — Nexus (Docker Compose)

Role: internal artifact endpoints.

  • Nexus deployed using Docker Compose.
  • Functions:
    • Artifacts Nexus: raw repos / package proxies / hosted repos (as designed in LLD).
    • Images Nexus: Nexus Docker registry repositories for built images (staging/release).

VM2 — CI Control Plane (Docker Compose)

Role: source control + CI orchestration.

  • Gerrit (with its PostgreSQL)
  • Zuul components:
    • scheduler
    • web
    • executor
    • merger
    • nodepool-launcher
    • PostgreSQL (for Zuul, if enabled/used)
    • ZooKeeper

All services are deployed using Docker Compose.

VM3 — Build Worker Host (Static Node + Containerized Runner)

Role: actual execution runtime for Zuul jobs.

  • Host OS: Ubuntu Jammy.
  • Nodepool model: static node (not ephemeral).
  • Due to failures using the VM OS directly as the job runtime, the effective runner is:
    • an Ubuntu Jammy container running on VM3,
    • with Docker socket access to the host,
    • and SSH configured so Zuul can run jobs inside this container as the “node”.

This is the v2.0.0 execution workaround for reliability and environmental consistency.

4) Network and access constraints

  • Environment B is LAN-only.
  • No internet access (no NAT, no proxy, no outbound routes).
  • Expected network connectivity (minimum):
    • Zuul components (VM2) must reach:
      • Gerrit (VM2)
      • ZooKeeper (VM2)
      • Nodepool target (VM3 container via SSH)
      • Nexus endpoints (VM1)
    • Build runner (VM3 container) must reach:
      • Nexus (VM1) for artifacts and/or base images
      • Gerrit (VM2) if required for checkouts during job execution (depending on merger strategy)
      • Any internal log storage endpoint (if used)

Any attempt to access public endpoints must be treated as a defect (inputs incomplete or configuration leakage).

5) Offline delivery inputs (what must be shipped into Environment B)

Environment B is provisioned and updated only through offline delivery, including: - Docker images required for: - Nexus stack (VM1) - Gerrit + PostgreSQL (VM2) - Zuul + ZooKeeper + PostgreSQL (VM2) - Any helper containers used on VM3 - Docker Compose files for each VM’s stack - OS packages (offline): - docker-ce and dependencies (deb packages) - any additional packages needed for host bootstrap - (Optional but recommended) checksums/manifests for each delivery set

6) Operational flow in Environment B (v2.0.0)

  1. Operators receive an offline delivery package (artifacts/images/composes/packages).
  2. Operators import/update Nexus repositories and Docker images locally.
  3. Operators ensure Gerrit has the required repos/refs for the target build (2024.1).
  4. Operators manually trigger the Zuul build job for the Kolla image build.
  5. Images are pushed to Images Nexus and logs/manifests are retained inside the LAN.

7) Definition of Done (Environment B baseline)

Environment B is considered “ready” when all items below are true:

Infrastructure readiness

  • VM1 Nexus stack is up and reachable from VM2 and VM3.
  • VM2 Gerrit is reachable and usable for repo access and events.
  • VM2 Zuul is healthy (scheduler/web/executor/merger) and connected to ZooKeeper.
  • VM2 Nodepool launcher can manage the static node definition.
  • VM3 runner container is reachable via SSH from Zuul executor and can run job commands.

Air-gap compliance

  • No internet egress exists from any VM (validated by network policy and practical tests).
  • Builds fail fast if required artifacts are missing (rather than silently fetching from the internet).

Build readiness

  • A manual Zuul trigger successfully runs a minimal test job on VM3 runner.
  • The Kolla build job can pull all dependencies from Nexus-only endpoints.
  • Built images are published to Images Nexus with predictable tags.

8) Known constraints / design tradeoffs (v2.0.0)

  • Static nodepool node reduces elasticity and increases coupling to VM3 health.
  • Containerized runner with Docker socket access simplifies builds but increases blast radius:
    • treat VM3 as a controlled, privileged build host.
  • HLD: architecture.md
  • LLD: lld/
  • Runbooks:
    • runbooks/offline-transfer-and-import.md
    • runbooks/airgapped-build.md
    • gerrit.md
  • Troubleshooting:
    • troubleshooting/known-issues.md

Supporting guides