Skip to content

gh-158585: Optimize bytes.join() for non-empty separator - #158689

Open
vstinner wants to merge 1 commit into
python:mainfrom
vstinner:bytes_join
Open

vstinner wants to merge 1 commit into
python:mainfrom
vstinner:bytes_join

Conversation

@vstinner

@vstinner vstinner commented Oct 3, 2026 •

Copy link
Copy Markdown
Member

@vstinner

vstinner commented Oct 3, 2026

Copy link
Copy Markdown
Member Author

Benchmark:

import pyperf
runner = pyperf.Runner()
for list_size in ('10', '10**3', '10**6'):
    runner.timeit(
        f'short items: {list_size}',
        setup=f"data=[b'x']*{list_size}; sep=b'.'",
        stmt="sep.join(data)")

for list_size in ('10', '10**2', '10**4'):
    runner.timeit(
        f'long items: {list_size}',
        setup=f"data=[b'x' * 100]*{list_size}; sep=b'.'",
        stmt="sep.join(data)")

Results on Fedora 44 with CPU isolation:

Benchmark ref change
short items: 10 215 ns 206 ns: 1.04x faster
short items: 10**3 16.7 us 15.5 us: 1.08x faster
short items: 10**6 63.0 ms 62.6 ms: 1.01x faster
long items: 10 294 ns 282 ns: 1.04x faster
long items: 10**2 1.83 us 1.75 us: 1.05x faster
Geometric mean (ref) 1.03x faster

Benchmark hidden because not significant (1): long items: 10**4

@vstinner

vstinner commented Oct 4, 2026

Copy link
Copy Markdown
Member Author

See also #158693 which optimizes PyUnicode_Join().

@eendebakpt eendebakpt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small change and a nice improvement. While investigating this I found there are larger gains possible in case of exact bytes (the common case?), but also a bug in the FT build (I created #158803).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants