← Back to overview

Audience Pipelines & Parallelization

How the skill orchestrates three independent build pipelines — and why generating fewer files is the real efficiency gain

01

One Pipeline per Audience

The core architectural principle: each audience receives its own independent build pipeline. This pipeline determines which depth levels are generated and when the process stops. Pipelines are fully decoupled from one another — they share neither state nor wait times.

Pipeline depth per audience

📊 Decision-Maker
L0
L1
STOP
👤 Practitioner
L0
L1
L2
STOP
🔧 Developer
L0
L1
L2
L3
STOP

Core principle: Pipelines do not wait for each other. The 📊 Decision-Maker pipeline finishes after 2 level steps while the 🔧 Developer pipeline still has 2 more stages to go. Each pipeline knows only its own max_depth and terminates independently.

02

Two-Level Parallelization

The skill's parallelization model operates on two tiers: at the upper level, Pipeline Agents (one per audience) run in parallel; at the lower level, File Agents (one per file to generate within a level) also run in parallel. Within a pipeline, however, levels are strictly sequential.

Hierarchical parallelization model

🎯 Orchestrator controls all pipelines

TIER 1: Pipeline Agents (parallel)

📊 Pipeline 👤 Pipeline 🔧 Pipeline

TIER 2: File Agents per level (parallel)

topic-a_de topic-a_en topic-b_de topic-b_en ...
↔️
Between Pipelines
Fully independent. The 📊 pipeline can be finished while the 🔧 pipeline is still building L1. No shared state, no waiting.
↕️
Within a Pipeline
Levels are sequential: L1 may only start once all L0 files of that pipeline are complete. Within a level, File Agents run in parallel.
03

What Can and Cannot Run in Parallel

The parallelization rules follow a clear logic: anything with no data dependency can run in parallel. Anything that builds on the output of a previous stage must wait.

Operation Parallel? Rationale
Different audience pipelines No shared state, no data dependencies
Themes within a level Themes are independent of each other
DE + EN of the same theme Language versions share no dependencies
Levels within a pipeline L1 references L0 pages — L0 must be finished
Phase 3 → Phase 4 Phase 4 requires the CSS/JS foundation from Phase 3
Phase 5 (Polish) Requires the complete output of all pipelines
// Execution model: what can run when? Phase 3: Foundation // MUST complete before Phase 4 build_css_js_foundation() // Once, globally Phase 4: Build // Per-pipeline execution 📊 pipeline (parallel to 👤 and 🔧) L0: [topic_de, topic_en] // File Agents in parallel --- barrier: all L0 complete --- L1: [topic_de, topic_en] // File Agents in parallel --- STOP (max_depth = 1) --- 🔧 pipeline (parallel to 📊 and 👤) L0: [topic_de, topic_en, ...] --- barrier --- L1: [topic_de, topic_en, ...] --- barrier --- L2: [topic_de, topic_en, ...] --- barrier --- L3: [topic_de, topic_en, ...] --- STOP (max_depth = 3) --- Phase 5: Polish // MUST wait until ALL pipelines done verify_links(), add_switches()

Execution model: Phase 3 (Foundation) runs once globally and must complete before Phase 4 begins. In Phase 4, the three pipelines work independently: within each pipeline, levels are processed sequentially (L0 must finish before L1 starts), but within a level all themes and language versions can be generated in parallel. Each pipeline stops at its individual max_depth. Phase 5 (Polish) may only begin once all pipelines are complete, since it needs the entire output for link validation and cross-references.

✏️ Quick check

The 🔧 pipeline is building L2 files. Can the 📊 pipeline build L1 at the same time?

Yes — different pipelines are independent of each other
No — only one pipeline can be active at a time
Only if both pipelines are on the same level
04

Practice vs. Ideal

Theory allows full parallelization. In practice, however, this is unreliable: LLM context windows are limited, parallel API calls can cause timeouts, and errors in a parallel branch are hard to debug. The recommendation is therefore sequential execution or at most 2 parallel agents (one DE+EN pair).

Timeline: Ideal vs. Practice

💡 Ideal: Fully parallel
📊 L0 → L1
👤 L0 → L1 → L2
🔧 L0 → L1 → L2 → L3

All 3 pipelines start simultaneously. Total time = longest pipeline.

✅ Practice: Sequential
📊 L0 → L1
then
👤 L0 → L1 → L2
then
🔧 L0 → L1 → L2 → L3

Pipelines one after another. Slower, but reliable and debuggable.

Key insight: The pipeline logic still determines order and depth, even during sequential execution. Whether pipelines run in parallel or sequentially does not change the result — only the runtime. The orchestrator decides which mode to use based on system load.

Fully Parallel
3 Pipeline Agents + N File Agents simultaneously. Fast but error-prone. Only for robust infrastructure.
Risky
DE+EN Parallel
Max. 2 File Agents simultaneously (one language pair). Good compromise between speed and stability.
Recommended
Fully Sequential
One file at a time. Slower but deterministic and easy to debug.
Safe
05

Efficiency Through Demand-Based Depth

The real efficiency gain is not parallelization but generating fewer files. Through demand-based depth, the system produces approximately 52 files instead of 100+ with 3 audiences — a 48% reduction. The 📊 Decision-Maker pipeline finishes early (2 level steps), the 🔧 Developer pipeline traverses all 4 stages.

File count: Uniform vs. Demand-based

Uniform (all L0–L3)
~105 files
Demand-based
~52 files

Breakdown per pipeline

📊 Decision-Maker
~14 files
2 steps
👤 Practitioner
~18 files
3 steps
🔧 Developer
~20 files
4 steps

Why this matters: Every file not generated saves LLM tokens, API costs, and runtime. The combination of Helpfulness Scoring (filters irrelevant topics) and demand-based depth (limits levels per audience) yields the total savings of ~48%.

Deep Dive Subagent Architecture in Detail → Pipeline Agents, File Agents, and the internal communication between tiers