apache/airflow's dev tooling carries the highest activity risk — 5 functions to flag

Five critical-band functions in airflow's Breeze dev tooling and registry scripts combine cyclomatic complexity as high as 105 with fan-out reaching 74 — all touched within the last 30 days, making them live regression risks rather than backlog items.

Stephen Collins ·
Generated by hotspots · free & open source
Activity Risk20.73Low
Hottest Functionrun_command

Antipatterns Detected

complex_branching5exit_heavy5god_function5long_function5deeply_nested4hub_function1

Run this on your own codebase

Hotspots runs locally in under a minute — no account, no data leaves your machine.

pip
$ pip install hotspots-cli
npm
$ npm install -g @stephencollinstech/hotspots
Run in any repo
$ hotspots analyze .
★ Star on GitHub

Key Points

What is a god function and why does it matter in airflow?

A god function is a single function that accumulates so many responsibilities that it becomes the primary coordination point for a broad slice of the system's behavior — it knows too much and does too much. In practice this means the function has high cyclomatic complexity (many branches), high fan-out (many distinct callees), and resists decomposition because its internal state threads through all of its responsibilities at once. In airflow's top five hotspots, every function carries the god_function pattern, with fan-out values ranging from 22 (`validate_reproducible_build`) to 74 (`main` in extract_metadata.py) — meaning a single change to any of them can ripple into dozens of downstream callees in ways that are hard to predict and harder to test. The concrete risk is that fixing a bug in one responsibility of the function inadvertently alters behavior in another responsibility that shares the same code path.

How do I reduce cyclomatic complexity in Python?

The most direct technique is extract-method refactoring: identify cohesive groups of branches that serve a single purpose and move them into a named function with a clear contract. A cyclomatic complexity above 15 is a signal to look for extraction candidates; above 30 it should be treated as a blocking issue for new feature additions. For `main` in `extract_metadata.py`, which has a CC of 105, a concrete first step is to extract the provider YAML discovery loop into a `_discover_provider_yamls()` function — that single extraction removes the iteration branching from the main body and immediately makes the discovery logic independently testable. In Python specifically, replacing multi-level `if/elif` chains with dispatch dictionaries or strategy objects can also cut CC significantly without changing external behavior.

Is airflow actively maintained?

The data makes a strong case for active development: all five top-ranked functions are in the "fire" quadrant, meaning they combine high structural complexity with recent commit activity. Every one of them has at least one commit in the last 30 days and was last changed the same day this analysis ran, placing them at the leading edge of current work. Of the 21,737 analyzed functions, 6,819 are in the fire quadrant and zero are in the debt quadrant, which means there are no functions flagged as structurally complex but completely untouched — the active codebase is being worked on. High structural complexity and active development are not mutually exclusive; the data here suggests a project under continuous development where the tooling and release infrastructure has grown in complexity alongside the core.

How do I reproduce this analysis?

The analysis was run against apache/airflow at commit `01b425a` using the Hotspots CLI, available at github.com/hotspots-dev/hotspots. After running `git checkout 01b425a` in a local clone of the repository, execute `hotspots analyze . --mode snapshot --explain-patterns --force` to reproduce the exact results. The same command works on any local git repository without additional configuration — no project-specific setup is required.

What does activity-weighted risk mean?

Activity-weighted risk combines a function's structural complexity — derived from cyclomatic complexity, nesting depth, and fan-out — with how frequently that function has been touched by recent commits. A function with cyclomatic complexity of 105 that has not been modified in two years poses lower near-term regression risk than one with cyclomatic complexity of 20 that is being changed every few days, because the complex-but-dormant function is not actively generating new bugs right now. This prioritization helps teams focus refactoring effort where it reduces the probability of introducing defects in the current development cycle, not just where the code appears complicated in the abstract. In airflow's top five, all five functions combine structural extremity with recent touches — each with a commit in the last 30 days and last changed the same day this analysis ran — which is what pushes their activity-weighted risk scores between 19.48 and 20.73.

Out of 21,737 functions analyzed in apache/airflow at commit 01b425a, 2,187 land in the critical band — and the five highest-ranked are all in the “fire” quadrant, meaning they are both structurally complex and actively changing right now. The top entry, run_command in Breeze’s run_utils.py, carries an activity-weighted risk score of 20.73 with a cyclomatic complexity of 38 and fan-out of 29, and was touched zero days ago. I would start there, because the live-churn argument is strongest: this is not cleanup work you can schedule for next sprint — it is a function being modified today that has 38 execution paths any one of which could regress.

The table below ranks functions by activity-weighted risk — a score that multiplies structural complexity by recent commit frequency. A function that is both hard to understand (high cyclomatic complexity) and actively changing is a higher priority than one that is complex but untouched. CC = cyclomatic complexity (independent execution paths); ND = max nesting depth; FO = fan-out (distinct callees).

Top 5 Hotspots

FunctionFileRiskCCNDFO
run_commanddev/breeze/src/airflow_breeze/utils/run_utils.py20.738429
generate_issue_content_providersdev/breeze/src/airflow_breeze/commands/release_management_commands.py20.483760
createStreamairflow-core/src/airflow/api_fastapi/auth/managers/simple/ui/openapi-gen/requests/core/serverSentEvents.gen.ts19.7351126
maindev/registry/extract_metadata.py19.7105674
validate_reproducible_builddev/breeze/src/airflow_breeze/utils/airflow_release_validator.py19.556822

Large Repo Analysis

airflow is a large repository. To stay within memory constraints, this analysis used hybrid touch mode: structural complexity — CC, ND, FO — is measured precisely for every function. Git activity is tracked at the function level (via git log -L) only for files with 5 or more commits in the last 30 days; other files use a file-level approximation. Rankings therefore surface functions that are both structurally complex and in the most actively-changing parts of the codebase. Dormant code with high structural complexity will rank lower than it would under a full per-function analysis — to surface it, run hotspots analyze . --per-function-touches on a machine with sufficient memory.

Codemod / Tooling Files in Results

The function createStream in airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui/openapi-gen/requests/core/serverSentEvents.gen.ts is generated TypeScript code — the .gen.ts suffix and openapi-gen/ path segment both confirm it is emitted by an OpenAPI client generator and should not be reviewed or refactored directly. Its high complexity scores (CC 35, ND 11) are artifacts of the generator’s imperative SSE parsing template, not hand-authored design choices. To exclude it from future Hotspots runs, add the following to your .hotspotsrc.json: { "exclude": ["**/openapi-gen/**"] }. If the SSE reconnection logic has correctness issues, the fix belongs in the generator configuration or an upstream template, not the emitted file.

Airflow is one of the largest Python monorepos in open source, and the sheer scale — 21,737 analyzed functions, 6,819 in the “fire” quadrant — means triaging by raw complexity alone would bury the most urgent work. Activity-weighted risk cuts through that by surfacing functions that are both hard to understand and being changed right now.

Triage Band Distribution
Fire6819Watch14918

21,737 functions analyzed

Every function in this repo is either actively changing or structurally safe enough not to worry about. There are no dormant complexity landmines in the quadrant sense — but the fire quadrant count of 6,819 means the live regression surface is wide. The five functions below are where I would focus first.

Detected Antipatterns
Complex Branching×5Complex Branching
High cyclomatic complexity — many independent execution paths, each a potential bug surface and required test case.
Exit Heavy×5Exit Heavy
Multiple return or throw paths dispersed through the body — each exit needs separate test coverage.
God Function×5God Function
Calls an unusually large number of distinct functions (high fan-out), making it the structural centre of gravity for a subsystem.
Long Function×5Long Function
Function body is too long to review in a single pass; likely contains multiple distinct responsibilities.
Deeply Nested×4Deeply Nested
Control structures nested 4+ levels deep, making it hard to reason about the full execution state at inner branches.
Hub Function×1Hub Function
Many other functions call this one — a change here ripples widely through callers.

Every top hotspot carries the god_function, long_function, complex_branching, and exit_heavy patterns simultaneously. That combination is not coincidental — it describes functions that accumulated responsibilities over time, with each new feature adding a branch and a return path rather than a new module.


run_command — run_utils.py

run_command
dev/breeze/src/airflow_breeze/utils/run_utils.py
20.73
critical
CC 38
ND 4
FO 29
touches/30d 1

run_command is the central process-execution wrapper for the Breeze developer CLI. From the source excerpt, it accepts a broad parameter surface — command as either a list or string, optional environment overrides, working directory, stdin input, output routing, verbose/dry-run overrides, a quiet flag, and a **kwargs passthrough to Popen — and then branches across all of those combinations to decide how to invoke and report the subprocess.

A cyclomatic complexity of 38 means there are 38 independent execution paths through this function. In Python, where **kwargs forwarding and duck-typed Mapping inputs add implicit dispatch on top of explicit branching, the real behavioral surface is wider still. The fan-out of 29 is equally telling: this function calls 29 distinct other functions, making it a hub in the dependency graph. That hub pattern means any caller that exercises a previously-untested combination of flags is traversing a code path that may not have been validated.

The source shows two nested helper closures (exclude_command and shorten_command) used for title heuristics, followed by environment construction, output routing, and the actual Popen invocation — all in one body. With nesting depth 4 and five concurrent antipatterns (complex_branching, exit_heavy, god_function, long_function, hub_function), this function is doing at least four distinct jobs: title generation, environment setup, output configuration, and subprocess execution.

My concrete recommendation: extract the title-generation logic (the exclude_command/shorten_command closures and the heuristic assembly) into a standalone _build_command_title() function, and pull environment construction into _build_command_env(). That alone would cut CC by roughly a third and make the Popen invocation path testable in isolation. The fact that this file has a single author in the last 90 days means the knowledge is concentrated — documentation of the branching logic should accompany any refactor.


generate_issue_content_providers — release_management_commands.py

generate_issue_content_providers
dev/breeze/src/airflow_breeze/commands/release_management_commands.py
20.39
critical
CC 83
ND 7
FO 60
touches/30d 1

This function orchestrates the generation of GitHub release issue content for provider packages — pulling PR metadata, filtering exclusions, querying the GitHub API, and templating the result. The source excerpt shows it doing all of this in a single function body: it defines an inline ProviderPRInfo named tuple, iterates over provider distributions, handles new-provider edge cases by falling back to git log, filters excluded PRs, and then opens a GitHub API session to resolve pull request objects.

A cyclomatic complexity of 83 is severe. Combined with a nesting depth of 7 and fan-out of 60, this is the most structurally dense function in the top five. In Python, fan-out of 60 means 60 distinct callees are invoked — and because Python resolves types at runtime, the actual coupling footprint could be broader than static analysis captures. The deeply_nested pattern at ND 7 means a reader must track seven levels of conditional context simultaneously to reason about a single execution path.

The function is doing at minimum: input normalization, provider filtering, changelog PR extraction, git-log fallback for new providers, exclusion list management, GitHub API connection, and PR metadata retrieval. Each of those is a testable unit in its own right. I would start by extracting the provider-filtering and PR-collection loop into a _collect_provider_prs() helper, which would immediately make the GitHub API interaction layer independently testable. The 83-path branching structure also means the exit_heavy pattern is carrying real test-coverage debt — each of those paths is a required test case that almost certainly doesn’t exist yet.


createStream — serverSentEvents.gen.ts

createStream
airflow-core/src/airflow/api_fastapi/auth/managers/simple/ui/openapi-gen/requests/core/serverSentEvents.gen.ts
19.73
critical
CC 35
ND 11
FO 26
touches/30d 1

Despite the .ts extension making this a TypeScript file in a Python monorepo, it surfaces in the top five by activity-weighted risk. The path — openapi-gen/requests/core/serverSentEvents.gen.ts — and the .gen.ts suffix indicate this is generated code, almost certainly produced by an OpenAPI client generator rather than hand-authored.

Max Nesting Depth 11
threshold: 4

The source excerpt confirms the structure: an async generator function (async function*) implementing SSE stream reconnection logic with an infinite while(true) loop, nested fetch error handling, body reader management, buffer accumulation, and event parsing — all in one body. The nesting depth of 11 reflects the nested loops and try/catch blocks that SSE parsing inherently requires when written imperatively, but in generated code this structure is produced by tooling, not by a developer making design choices.

Because this is generated code (the .gen.ts suffix is unambiguous), I would not refactor it directly. The right action is to exclude this path from Hotspots analysis using a .hotspotsrc.json ignore rule (see the vendor note below), and if the SSE parsing logic itself needs hardening, that work belongs in the generator template or in a hand-authored wrapper — not in the emitted file. See the vendor note for the exact exclude pattern.


main — extract_metadata.py

main
dev/registry/extract_metadata.py
19.71
critical
CC 105
ND 6
FO 74
touches/30d 1

main in dev/registry/extract_metadata.py is the entry point for the Airflow registry metadata extraction pipeline. From the excerpt it handles argument parsing, provider YAML discovery via recursive glob, provider ID construction from path parts, release tag filtering, and output directory management — all within a single function body.

Cyclomatic Complexity 105
threshold: 30

A cyclomatic complexity of 105 is in the extreme tier. This is not a function that is hard to understand in spots — it is a function where holding the full decision tree in working memory is not practically possible. The fan-out of 74 reinforces that: 74 distinct callees means this function is both an orchestrator and a direct participant in a very wide range of subsystems. With nesting depth 6, there are multi-level conditional chains inside the provider iteration loop visible in the excerpt — the release-tag filtering logic alone branches on --allow-unreleased, tag existence, and provider YAML validity in nested sequence.

A main() function with CC 105 is a classic sign that the script grew organically without decomposition. The extract-method refactoring here is straightforward to identify: argument parsing, provider YAML discovery, release tag loading, per-provider extraction, and output writing are each distinct phases that should be named functions. Extracting just the YAML discovery loop into _discover_provider_yamls() and the release-tag filtering into _load_release_tags() would cut the complexity roughly in half and make each phase unit-testable. A single author in the last 90 days means this refactoring should be paired with inline documentation of the phantom-version filtering logic, which the excerpt comments explain but the code doesn’t encode as a named concept.


validate_reproducible_build — airflow_release_validator.py

validate_reproducible_build
dev/breeze/src/airflow_breeze/utils/airflow_release_validator.py
19.48
critical
CC 56
ND 8
FO 22
touches/30d 1

This method verifies that a release can be built reproducibly by checking out a specific git tag, building from source, and comparing the result against SVN artifacts. The source excerpt shows a sequence of git subprocess calls — status check, branch detection, HEAD resolution, and tag checkout — each followed by early-return ValidationResult objects on failure.

Max Nesting Depth 8
threshold: 4

The nesting depth of 8 is the highest in the top five and a strong refactoring signal on its own. In a method that issues multiple sequential git commands and returns early on each failure, deep nesting typically means error-handling branches are nested inside control-flow branches rather than being flattened. The exit_heavy pattern here is structurally appropriate — early returns for validation failures are intentional — but at CC 56, the number of paths through the function means a test suite would need 56 test cases to achieve branch coverage, which is unlikely for a release validation utility.

The deeply_nested pattern at ND 8 suggests the checkout, build, and comparison phases are collapsed into one cascading conditional structure rather than being separated into guard-clause sequences. I would introduce a _checkout_tag_safely() helper that handles the branch-save, checkout, and restore lifecycle with a context manager or try/finally, reducing both the nesting and the CC by encapsulating the git state management. Fan-out of 22 indicates this method also coordinates a meaningful number of external utilities — extracting the git coordination layer would make mocking those dependencies in tests significantly easier.

Patterns Found

Antipatterns detected across the top functions in this snapshot:

PatternOccurrences
complex_branching5
exit_heavy5
god_function5
long_function5
deeply_nested4
hub_function1

These labels belong to two tiers — Tier 1 (structural): complex_branching, deeply_nested, exit_heavy, long_function, god_function. Tier 2 (relational/temporal): hub_function, cyclic_hub, middle_man, neighbor_risk, stale_complex, churn_magnet, shotgun_target, volatile_god.

Reproduce This Analysis

git clone https://github.com/apache/airflow
cd airflow
git checkout 01b425a06f6998950cada673582a17b6b2264b41
hotspots analyze . --mode snapshot --explain-patterns --force --hybrid-touches 5

To run the same analysis on your own codebase, run hotspots analyze . --mode snapshot in any local git repo — no configuration required.

I use Hotspots to highlight structural and activity risk — not “bad code.” I treat these findings as a prioritization aid, not a bug predictor. Editorial policy →

Related Analyses