OpenStack Air-Gapped Image Factory (SA/Qi V2)¶
Purpose¶
This repository documents the design, implementation, and operation of an air-gapped OpenStack image build system based on Kolla and orchestrated by Zuul.
The system is intentionally split into two environments:
- Mirror Factory (Connected): prepares and versions all required artifacts using controlled internet access.
- Air-Gapped Build & Release (Disconnected): imports artifacts offline and builds OpenStack images with no internet egress.
This index is the authoritative navigation entry point for all versions.
Quick start navigation¶
Choose what you need:
-
Understand the architecture and design
- HLD: architecture.md
- LLD Index: lld/
-
Deploy the system
- Implementation guides: implementation/
- Configuration templates: templates/ (sanitized)
-
Run an update/build cycle
- Mirror Factory runbook: runbooks/mirror-factory-update.md
- Offline transfer + import: runbooks/offline-transfer-and-import.md
- Air-gapped build (manual Zuul trigger): runbooks/airgapped-build.md
-
Troubleshoot and learn from prior incidents
- Known issues: troubleshooting/known-issues.md
- Debug playbooks: troubleshooting/
- Battle Log: battle-log/
-
Review results/evidence per version
Versions and roadmap¶
Current baseline¶
- v2.0.0 — Two-environment model, manual artifact preparation, offline delivery, manual Zuul build trigger for OpenStack 2024.1.
Planned (examples — adjust as needed)¶
- v2.1.0 — Bundle integrity hardening (signatures), completeness checks, stronger “no-egress” enforcement.
- v2.2.0 — Promotion workflow (staging → production), approval gates, standardized release bundles.
- v3.x — Expanded parity with upstream gating, automated update detection, deeper supply-chain controls.
The roadmap is tracked in: releases/ROADMAP.md
Release notes and evidence policy (per version)¶
Every version MUST have:
- A release note describing scope, changes, and non-goals.
- A repeatable “how to run” section referencing runbooks.
- Evidence artifacts (digests/manifests/log pointers) sufficient to prove:
- air-gap compliance (no internet dependency),
- build success and output correctness,
- traceability from artifact bundle to built images.
See:
Architecture and design¶
High-Level Design (HLD)¶
- Primary architecture: architecture.md
Low-Level Design (LLD)¶
- LLD directory index: lld/
- Suggested LLD set (v2.0.0 baseline):
Architecture Decision Records (ADRs)¶
ADRs capture why we made decisions, including rejected alternatives.
- ADR index: adr/
- Start here: adr/0000-adr-index.md
Implementation documentation¶
Implementation docs should be actionable, step-by-step, and environment-specific.
- Mirror Factory deployment: implementation/mirror-factory/
- Air-gapped deployment (follow numbered order 1 → 5): implementation/airgapped/
- Service configuration:
- Gerrit: implementation/services/gerrit.md
- Zuul: implementation/services/zuul.md
- Nodepool (if used): implementation/services/nodepool.md
- Nexus: implementation/services/nexus.md
- Network controls: implementation/network/
Operations and runbooks¶
Runbooks describe the recurring operational cycles and emergency procedures.
- Update cycle runbooks: runbooks/
- Backup/restore: runbooks/backup-and-restore.md
- Capacity planning: runbooks/capacity-planning.md
- Monitoring/alerting: runbooks/observability.md
Troubleshooting¶
Troubleshooting docs must be symptom-based and include concrete checks.
- Known issues: troubleshooting/known-issues.md
- Debug playbooks: troubleshooting/
Battle Log (what we went through)¶
This is the candid engineering record of the path to success:
- misdocumentation,
- undefined behavior,
- bugs and sharp edges,
- dead ends and reversals,
- workarounds, patches, and hard-earned lessons.
Battle Log entries MUST be written so a new engineer can:
- reproduce the issue context,
- understand root cause,
- apply a fix,
- prevent regressions.
Battle Log index¶
Battle Log entry template¶
Create entries as battle-log/YYYY-MM-DD-short-title.md using:
- Context
- What we were trying to do (component/version/environment).
- Expected behavior
- What the docs/assumptions suggested would happen.
- Actual behavior
- What happened (include relevant log snippets, error strings).
- Investigation
- Hypotheses tested, commands run, configs reviewed.
- Root cause
- The true underlying cause (include references to code/docs when possible).
- Fix / workaround
- Exact changes applied (configs, patches, procedures).
- Preventative control
- How we ensure this never happens again (tests, checks, guardrails).
- Outcome
- What changed, what improved, remaining limitations.
- Links
- Related ADRs, commits, issue tickets, log artifacts, runbooks.
Repository structure¶
This repository is organized as:
index.md(this file)architecture.md(HLD)lld/(LLD modules)implementation/(deployment & configuration)runbooks/(operations)troubleshooting/(known issues & debug playbooks)battle-log/(the real story)adr/(decision records)releases/(release notes + roadmap)results/(evidence per version)templates/(sanitized config templates overview)
Change log (index)¶
- v2.0.0: Project documentation baseline created (HLD/LLD/ops/troubleshooting/battle-log structure).