← All projects
GovernancePythonSystems

Orchestration Hub

The central nervous system of eight GitHub organizations

The Problem

Every system with 81 repositories needs a single place that answers "what exists, what's its status, and who owns it?"[1] Meadows identifies information flows as the most powerful leverage points in complex systems — when participants can see the system's state, they make better decisions. Most multi-repo systems rely on tribal knowledge: someone remembers which repo does what, which ones are active, which ones were experiments. That breaks at scale. It breaks when you onboard new contributors — human or AI. And it breaks when you need to answer a simple question: is this system healthy?[2]

graph TD A[Registry v2.json] --> B[Dependency Validator] A --> C[Monthly Audit] A --> D[Promotion Engine] A --> E[Content Publisher] A --> F[POSSE Distributor] B --> G{Merge Gate} C --> H[Health Report] D --> I[State Transition] E --> J[Public Process] F --> K[Mastodon + Discord]
The orchestration hub as information nexus — all governance flows pass through the registry

The Solution

A machine-readable JSON registry (registry-v2.json) that serves as the single source of truth for the entire eight-organ system.[3] Fowler's principle of a "Repository" pattern — an abstraction layer that mediates between the domain and data mapping layers — applies here at the organizational scale. Every repository entry carries its current status, documentation tier, dependency declarations, promotion state, implementation status, and portfolio relevance score. The registry isn't documentation about the system — it is the system's self-knowledge. When the registry and reality disagree, either the registry must be updated or reality must be fixed. This is Article I of the system constitution.[4]

            ┌──────────────────────────────┐
            │   orchestration-start-here    │
            │                              │
            │  registry-v2.json            │
            │  governance-rules.json       │
            │  5 workflows · 3 scripts     │
            └──────────┬───────────────────┘
                       │
          ┌────────────┼────────────────┐
          │            │                │
 ┌────────▼──────┐ ┌──▼──────────┐ ┌───▼──────────┐
 │   ORGAN-I     │ │  ORGAN-II   │ │  ORGAN-III   │
 │   Theory      │─▶  Art        │─▶  Commerce    │
 │   18 repos    │ │  22 repos   │ │  21 repos    │
 └───────────────┘ └─────────────┘ └──────────────┘
          │            │                │
 ┌────────▼──────┐ ┌──▼──────────┐ ┌───▼──────────┐
 │   ORGAN-V     │ │  ORGAN-VI   │ │  ORGAN-VII   │
 │   Process     │ │  Community  │ │  Marketing   │
 └───────────────┘ └─────────────┘ └──────────────┘
                       │
            ┌──────────▼───────────────────┐
            │       meta-organvm           │
            │       (Umbrella Org)         │
            └──────────────────────────────┘

Flow enforced: I → II → III (no back-edges)
Hub validates all 31 dependency edges.
The orchestration hub coordinates all 8 organs through a centralized registry with enforced dependency flow

The Registry Schema

Each of the 80 entries in the registry carries structured metadata that enables both automated validation and human reasoning about system state.[5] The schema follows the Composite pattern — each entry is a leaf node, organs are composites, and the registry is the root — allowing queries at any level of granularity.

FieldTypePurpose
statusenumACTIVE, ARCHIVED, PLANNED
tierenumflagship, standard, stub, archive, infrastructure
dependenciesarrayCross-organ dependency declarations
promotion_statusenumLOCAL → CANDIDATE → PUBLIC_PROCESS → GRADUATED → ARCHIVED
implementation_statusenumPRODUCTION, PROTOTYPE, SKELETON, DESIGN_ONLY
portfolio_relevanceenumCRITICAL, HIGH, MEDIUM, LOW, INTERNAL
registry-v2.json (excerpt)
{
  "orchestration-start-here": {
    "organ": "IV",
    "status": "ACTIVE",
    "tier": "flagship",
    "promotion_status": "GRADUATED",
    "implementation_status": "PRODUCTION",
    "dependencies": [],
    "portfolio_relevance": "CRITICAL",
    "ci_workflow": "ci-python.yml",
    "platinum_status": true,
    "description": "Central nervous system of the eight-organ architecture"
  }
}
Complete registry entry — every field is validated by automated scripts

Governance as Code

The governance model is codified in governance-rules.json — a machine-readable constitution with six articles and four amendments.[6] Scott's critique of "high modernist" schemes warns against legibility imposed without local knowledge — these governance rules encode both top-down constraints (no back-edges, mandatory documentation) and bottom-up flexibility (promotion is earned, not assigned). The most important constraint: no back-edges in the dependency graph. Theory feeds art feeds commerce (I→II→III), validated automatically across all 31 dependency edges.[7]

graph LR A1[Art I: Registry Truth] --> V1[Reconciliation Script] A2[Art II: No Back-Edges] --> V2[Dependency Validator] A3[Art III: Promotion Gates] --> V3[Promote Workflow] A4[Art IV: Monthly Audit] --> V4[Audit Workflow] A5[Art V: Documentation] --> V5[Tier Checker] A6[Art VI: Transparency] --> V6[Public Process Pipeline]
Governance article enforcement — each article maps to an automated check

Five Automation Workflows

The automation layer transforms governance from aspiration to enforcement.[8] Humble and Farley's core argument — that the deployment pipeline should be the only way to make changes — applies here: the registry pipeline is the only legitimate way to modify system state.

  • validate-dependencies — Checks the dependency graph on every registry change. Cycle detection, transitive depth analysis, back-edge blocking.
  • monthly-organ-audit — Full system health check with Markdown report + JSON metrics. Creates a GitHub Issue with findings.
  • promote-repo — Handles state machine transitions with pre/post validation against governance rules.
  • publish-process — Syncs ORGAN-V content to public channels. README extraction + essay scaffolding.
  • distribute-content — POSSE distribution to Mastodon and Discord. Verified live (HTTP 200 + 204).
scripts/validate-deps.py (core logic)
def validate_dependencies(registry: dict) -> list[str]:
    """Validate all dependency edges. Returns list of violations."""
    violations = []
    organ_order = {'I': 1, 'II': 2, 'III': 3, 'IV': 4,
                   'V': 5, 'VI': 6, 'VII': 7, 'META': 8}

    for repo, meta in registry.items():
        src_organ = meta['organ']
        for dep in meta.get('dependencies', []):
            dst_organ = dep.split('/')[0].split('-')[-1].upper()
            if organ_order.get(dst_organ, 0) < organ_order.get(src_organ, 0):
                violations.append(f"BACK-EDGE: {repo} -> {dep}")

    # Cycle detection via topological sort
    if has_cycle(build_graph(registry)):
        violations.append("CIRCULAR DEPENDENCY DETECTED")

    return violations  # Must be empty for merge gate to pass
Dependency validation — cycle detection and back-edge enforcement across 31 edges

Why a Registry Matters

A machine-readable registry enables automation (CI/CD workflows query it), auditing (monthly health checks generate reports from it), and honest portfolio presentation (the portfolio site reads from the same source of truth as the governance scripts).[9] Deming's principle — "In God we trust; all others must bring data" — is enacted literally: the registry is the data. It's what makes the eight-organ system governable rather than merely large. It's the difference between 81 repositories and a system.[10]

Tradeoffs & Lessons

  • Single file vs. distributed config — Keeping the entire registry in one JSON file means every change touches one file and every query is a local read. The alternative (per-repo config files aggregated at build time) would be more distributed but harder to validate atomically. At 80 entries the single-file approach works; at 500 it might not.[11]
  • Strict governance vs. velocity — The no-back-edges rule and promotion state machine add friction to every cross-organ change. This is intentional: the friction prevents the most common failure mode of multi-repo systems (tangled dependencies that nobody can reason about). But it slows rapid prototyping across organ boundaries.
  • Monthly audit cadence — Monthly is frequent enough to catch drift but infrequent enough to be actionable. Weekly would create audit fatigue; quarterly would let too much drift accumulate.
  • Constitution as code — Writing governance rules as JSON rather than prose forces precision but loses nuance. Article V ("Portfolio-Quality Documentation") requires human judgment that can't be reduced to a JSON field. The compromise: automated checks for measurable criteria, manual review gates for qualitative ones.[12]
80
Registry Entries
31
Dependencies
5
Workflows
6+4
Articles + Amendments
228
Validation Checks
0
Circular Deps
Orchestration hub: system-wide health metrics as of launch

References

  1. Meadows, Donella H.. Thinking in Systems: A Primer. Chelsea Green Publishing, 2008.
  2. Brooks, Frederick P.. The Mythical Man-Month: Essays on Software Engineering. Addison-Wesley, 1975.
  3. Fowler, Martin. Patterns of Enterprise Application Architecture. Addison-Wesley, 2002.
  4. Ostrom, Elinor. Governing the Commons: The Evolution of Institutions for Collective Action. Cambridge University Press, 1990.
  5. Gamma, Erich, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1994.
  6. Scott, James C.. Seeing Like a State: How Certain Schemes to Improve the Human Condition Have Failed. Yale University Press, 1998.
  7. Martin, Robert C.. Clean Architecture: A Craftsman's Guide to Software Structure and Design. Prentice Hall, 2017.
  8. Humble, Jez and David Farley. Continuous Delivery: Reliable Software Releases through Build, Test, and Deploy Automation. Addison-Wesley, 2010.
  9. Deming, W. Edwards. Out of the Crisis. MIT Press, 1986.
  10. Senge, Peter M.. The Fifth Discipline: The Art and Practice of the Learning Organization. Doubleday, 1990.
  11. Tanenbaum, Andrew S. and Maarten van Steen. Distributed Systems: Principles and Paradigms. Pearson, 2007.
  12. McConnell, Steve. Code Complete: A Practical Handbook of Software Construction. Microsoft Press, 2004.