Black's linegen.py carries the highest activity risk — 5 functions to address first

All five of the highest activity-risk functions in psf/black live in src/black/linegen.py, each touched 28 times in the last year and changed today.

Stephen Collins ·
Generated by hotspots · free & open source
pip
$ pip install hotspots-cli
Activity Risk3.18Low
Hottest Functionnormalize_invisible_parens

Run this on your own codebase

See if your own repo has a normalize_invisible_parens-style hotspot — run this in any local git repo:

pip
$ pip install hotspots-cli
npm
$ npm install -g @stephencollinstech/hotspots
Run in any repo
$ hotspots analyze .
★ Star on GitHub

Key Points

What makes a cyclomatic complexity of 84 in normalize_invisible_parens a problem in black?

Cyclomatic complexity counts the independent execution paths through a function, and each path is a potential bug surface and a test case you need to write. A value of 10 is moderate and 30 is high, so 84 is far into extreme territory for a single function. In black's case `normalize_invisible_parens` branches on statement type, preview-feature flags and child node type, and it calls itself recursively, so a change to one branch can alter how another is reached. It was touched 28 times in the last year and last changed 0 days ago, so that complexity sits in live code. Covering every combination with tests is impractical, which is why this function is where I would start.

How do I reduce cyclomatic complexity in python?

The main technique is extract-method combined with decompose-conditional: pull each distinct branch group into a named function and replace long boolean chains with named predicates. As a rough rule, a value above 15 warrants splitting and above 30 warrants immediate attention. A concrete first step for `normalize_invisible_parens` (cyclomatic complexity 84) is to move the preview-gated block that dispatches on statement type into its own function, and to separate the early check for a formatting-off directive. Those two extractions are mechanical and should remove a large share of the paths.

Is black actively maintained?

The data says yes. Each of the top five functions is in the active tier, was touched 28 times in the last year and was last changed 0 days ago, and the file shows 14 distinct authors in the last 90 days. All 1,342 functions in this snapshot are in the active tier. Active development and high structural complexity are not mutually exclusive, and here they coincide in `src/black/linegen.py`.

How do I reproduce this analysis?

Install the hotspots CLI from its GitHub repository, clone psf/black and run `git checkout 81687b5` to match the commit I analyzed. Then run `hotspots analyze . --mode snapshot --force` from the repository root. The same command works on any local git repo without configuration.

What does activity-weighted risk mean?

Activity-weighted risk combines structural complexity (cyclomatic complexity, nesting depth and fan-out) with how much and how recently the code has changed, so functions that are both hard to understand and in changing code score the highest. A function with a cyclomatic complexity of 80 that has not been touched in two years scores much lower than a less complex function touched every week, because the dormant one has lower near-term regression risk. This helps focus refactoring effort where it reduces the chance of introducing bugs right now, not just where the code looks complicated in the abstract.

All five of the top hotspots in psf/black are in one file, src/black/linegen.py, and I think that concentration matters more than any single function in it. Each of the five is in the critical band, in the active tier, touched 28 times in the last year, and was last changed 0 days ago; the highest, normalize_invisible_parens, carries an activity-weighted risk score of 3.18 with a cyclomatic complexity of 84. Black is a Python code formatter, and this analysis covers 1,342 functions, 125 of them critical. I would start with linegen.py as a module rather than picking one function from it, because these are complex functions in code that is changing right now, which makes them worth reviewing sooner rather than later.

The table below ranks functions by activity-weighted risk — a score that multiplies structural complexity by recent commit frequency. A function that is both hard to understand (high cyclomatic complexity) and actively changing is a higher priority than one that is complex but untouched. CC = cyclomatic complexity (independent execution paths); ND = max nesting depth; FO = fan-out (distinct callees).

Top 5 Hotspots

FunctionFileRiskCCNDFO
normalize_invisible_parenssrc/black/linegen.py3.284529
visit_powersrc/black/linegen.py3.132821
delimiter_splitsrc/black/linegen.py3.136323
maybe_make_parens_invisible_in_atomsrc/black/linegen.py3.147322
generate_trailers_to_omitsrc/black/linegen.py3.12949
Activity Tier Distribution
Active1342

1,342 functions analyzed

Every function in this snapshot is in the active tier (1,342 of 1,342), so tier does not separate the top five from the rest of the repo. What separates them is structure. Of the 125 critical functions, the top five I cover below all come from the same file, and they share an identical history profile: 28 touches in the last year, 14 distinct authors in the last 90 days, and a file-level bug-fix rate of 54%. That last figure is file-level, so it describes linegen.py as a whole, not any one function. The same file shows zero reverts and zero bug-linked commits in the enrichment data, so I read the 54% as a sign that this file attracts corrective work, not as proof that any of these five functions is defective.

The shared file is also why the five scores sit so close together (3.07 to 3.18). The activity side of the score is identical for all of them, so the ordering is driven almost entirely by structure. That tells me the review priority here is about the module, and the ranking within it is secondary.

normalize_invisible_parens — linegen.py

normalize_invisible_parens
src/black/linegen.py
3.18
critical
CC 84
ND 5
FO 29
touches/yr 28

This is the top-ranked function, and I would start with it because a cyclomatic complexity of 84 puts it well into the extreme range while 28 touches in the last year and a last change 0 days ago show it is being edited constantly.

Cyclomatic Complexity 84
threshold: 10

From the excerpt, the function does several different jobs in one body. It first scans the node’s prefix comments and bails out early on a formatting-off directive. It then runs a block gated by a preview feature flag that branches on statement type (for loops, comprehensions, assignments, delete statements, with statements) and hands each case to an unpacking-normalization helper. After that it wraps with-statement context managers, and finally it loops over children with a series of type checks, recursing into itself for annotated assignments, case blocks and guards. The recursion matters: a change to one branch can affect how another branch is reached.

The fan-out of 29 is high, and in Python the real coupling is likely larger than the number suggests because the node types and helper behavior are resolved at runtime. The preview-flag gating adds a second axis of paths: each flag effectively doubles the states a test needs to cover. The file-level bug-fix rate of 54% and a review comment density of 3.0 are consistent with a function people argue about, but again they are file-level signals.

My recommendation: extract the preview-gated unpacking block into its own function keyed on statement type, and separate the early directive check. Those two moves are mechanical, preserve behavior, and should remove a large share of the 84 paths without touching the main child loop.

visit_power — linegen.py

visit_power
src/black/linegen.py
3.13
critical
CC 32
ND 8
FO 21
touches/yr 28

The number I would look at here is the nesting depth of 8, which is the strong refactoring threshold and the deepest of the five. A cyclomatic complexity of 32 is lower than its neighbors, but depth makes it harder to reason about than the numbers alone suggest.

Max Nesting Depth 8
threshold: 4

From the excerpt, it is a generator method that does two unrelated things: it wraps numeric literals in parentheses before an attribute access (skipping hex, binary, octal and complex literals), and it removes redundant parentheses around generators and subscripts when two preview flags are enabled. The second block is where the depth comes from: a loop over children, a trailer check, an if/elif split between generator and subscript handling, then more conditionals inside each branch for the inner rewrite.

Because it is a visitor method, the surrounding dispatch is implicit and resolved by node type at runtime, so I cannot tell from the data how many paths enter it. What the metrics do show is that the two jobs share no logic.

My recommendation: split it into a numeric-literal wrapping step and two helpers, one for generator trailers and one for subscript trailers, each using early continue to flatten the nesting. Getting depth from 8 to 3 or 4 is achievable with guard clauses alone.

delimiter_split — linegen.py

delimiter_split
src/black/linegen.py
3.12
critical
CC 36
ND 3
FO 23
touches/yr 28

This is one of Black’s line-splitting strategies, splitting on the highest-priority delimiter. A cyclomatic complexity of 36 and fan-out of 23 are both substantial, and the excerpt shows why: a sequence of early-exit checks that raise a ‘cannot split’ exception, several gated by preview features (keeping commented expressions together, hugging comparators, keeping dictionary keys with operators), followed by a nested local helper that appends leaves to the current line and then the main splitting loop.

Two things stand out. First, the number of distinct reasons to abandon a split is a test-coverage burden, since each raise is an exit path that needs its own input. Second, the nested helper closes over mutable state in the enclosing function, so reading the loop means tracking that state across the closure boundary.

My recommendation: move the preflight checks into a single function that either returns or raises, so the splitting body starts after a clean precondition boundary. That leaves the main loop and the helper to be reviewed on their own.

maybe_make_parens_invisible_in_atom — linegen.py

maybe_make_parens_invisible_in_atom
src/black/linegen.py
3.1
critical
CC 47
ND 3
FO 22
touches/yr 28

A cyclomatic complexity of 47 with a nesting depth of only 3 tells me this is wide, not deep. The excerpt shows a single enormous guard condition that chains many or clauses, each testing a different node shape (empty tuples, one-tuples, yields, walrus-containing tuples, star expressions, generators, comparison operands, implicit string concatenation), with feature-flag and Python-version-feature checks folded in. Every or and and adds a path, which is how a function with shallow nesting reaches 47.

The function returns a boolean and takes eight parameters, several of them behavior-toggling booleans added over time (for example the generator-parens and subscript flags). That parameter list is itself a signal: each flag is a mode a caller can request, and visit_power in the same file is one of the callers passing them. The rest of the function appears to handle a walrus-assignment case with a list of parent types.

My recommendation: break the giant guard into named predicate functions such as ‘is this atom’s tuple unremovable’ and ‘is this a protected yield’, then compose them. Naming the clauses makes each independently testable and makes the next preview flag a one-line addition instead of another branch in a 47-path function.

generate_trailers_to_omit — linegen.py

generate_trailers_to_omit
src/black/linegen.py
3.07
critical
CC 29
ND 4
FO 9
touches/yr 28

This is the lowest-ranked of the five and the least coupled: a fan-out of 9 is modest, and the complexity is concentrated in its own control flow, with a cyclomatic complexity of 29 and a nesting depth of 4. It is a generator that walks a line’s leaves in reverse, accumulating a cumulative set of closing-bracket IDs and yielding a new set each time a bracket pair can be omitted for a right-hand split.

The risk here is different from its neighbors. Low fan-out means few external dependencies to mock, but a generator with several break and continue exits, mutable accumulators (the omit set, the inner-bracket set, the tracked opening and closing brackets) and a trailing-comma rule duplicated in two places is hard to follow step by step. The duplicated trailing-comma check in the excerpt is a concrete consolidation target.

My recommendation: extract the repeated trailing-comma check into a small predicate and write table-driven tests that assert the exact sequence of yielded sets for a handful of bracket shapes. With this much implicit state, the yield order is the behavior worth pinning down before anyone refactors the loop.

What the concentration means

Taken together, the five functions share an unusual pattern: nearly all of them combine a large chain of boolean conditions with preview-feature gating. Each new preview feature adds branches to several of them at once, which would explain why a single file shows 28 touches and 14 recent authors. If I were planning refactoring work here, I would not schedule five separate tickets. I would pull the shared predicates, the preview-flag checks and the early-exit preconditions out of linegen.py first, then revisit the functions individually.

Reproduce This Analysis

git clone https://github.com/psf/black
cd black
git checkout 81687b58f7486b29c255055d1205b582dba8e49c
hotspots analyze . --mode snapshot --force

To run the same analysis on your own codebase, run hotspots analyze . --mode snapshot in any local git repo — no configuration required.

I use Hotspots to highlight structural and activity risk — not “bad code.” I treat these findings as a prioritization aid, not a bug predictor. Editorial policy →

Was this useful? Tell me more →

Related Analyses