perf: more tokenizer inlining (5% faster tokenizing) by bartveneman · Pull Request #309 · projectwallace/css-parser

bartveneman · 2026-10-15T20:54:08Z

Trims work in the tokenizer's hot paths.

Changes

src/char-types.ts
- Character-type constants are now const instead of let, so engines can inline them.
- Adds a precomputed CHAR_WHITESPACE_OR_NEWLINE mask.
- CHAR_WHITESPACE is no longer exported.
src/tokenize.ts
- Replaces every (CHAR_WHITESPACE | CHAR_NEWLINE) with the combined mask. This covers whitespace skipping, whitespace tokens, hex escapes, comments and strings.
- Moves the ch_type lookup below the single-char token switch, so {, }, :, ; and similar characters skip the table read.

Benchmarks (mean ops/sec, before → after)

	Nerdy.dev	CSS Tricks	Bootstrap	Tailwind
tokenize	+3.5%	+5.3%	+4.8%	+8.6%
parse	+4.8%	+6.2%	+5.0%	+4.2%
parse (no sub-parsing)	+6.6%	+5.0%	+4.0%	+3.8%

Selector, selector-list, at-rule prelude and declaration sub-parsers improved by 3–5%. Value and dimension parsing is flat.
Walk and traverse are unchanged, as expected.
Memory is unchanged. Bundle size grew by 13 B.

bartveneman · 2026-10-15T20:54:19Z

after:

┌─────────┬─────────────────────────────────────┬────────────────┬─────────────┬────────────────┐
│ (index) │ Benchmark                           │ mode (ops/sec) │ mode (µs)   │ mean (ops/sec) │
├─────────┼─────────────────────────────────────┼────────────────┼─────────────┼────────────────┤
│ 0       │ 'tokenize|Nerdy.dev'                │ '710'          │ '1407.783'  │ '705'          │
│ 1       │ 'parse|Nerdy.dev'                   │ '579'          │ '1727.434'  │ '570'          │
│ 2       │ 'parse (no sub-parsing)|Nerdy.dev'  │ '1255'         │ '796.679'   │ '1249'         │
│ 3       │ 'tokenize|CSS Tricks'               │ '80'           │ '12474.752' │ '80'           │
│ 4       │ 'parse|CSS Tricks'                  │ '51'           │ '19468.077' │ '51'           │
│ 5       │ 'parse (no sub-parsing)|CSS Tricks' │ '170'          │ '5883.146'  │ '168'          │
│ 6       │ 'tokenize|Bootstrap'                │ '467'          │ '2142.616'  │ '461'          │
│ 7       │ 'parse|Bootstrap'                   │ '322'          │ '3107.450'  │ '318'          │
│ 8       │ 'parse (no sub-parsing)|Bootstrap'  │ '623'          │ '1605.587'  │ '617'          │
│ 9       │ 'tokenize|Tailwind'                 │ '38'           │ '26465.039' │ '38'           │
│ 10      │ 'parse|Tailwind'                    │ '25'           │ '39679.785' │ '25'           │
│ 11      │ 'parse (no sub-parsing)|Tailwind'   │ '55'           │ '18091.136' │ '54'           │
│ 12      │ 'parse_selector (simple)'           │ '2454771'      │ '0.407'     │ '2040208'      │
│ 13      │ 'parse_selector (complex)'          │ '882706'       │ '1.133'     │ '852535'       │
│ 14      │ 'parse_selector_list (simple)'      │ '1095890'      │ '0.912'     │ '1060964'      │
│ 15      │ 'parse_selector_list (complex)'     │ '351787'       │ '2.843'     │ '331240'       │
│ 16      │ 'parse_atrule_prelude (media)'      │ '799386'       │ '1.251'     │ '773158'       │
│ 17      │ 'parse_atrule_prelude (supports)'   │ '1076878'      │ '0.929'     │ '1035212'      │
│ 18      │ 'parse_declaration'                 │ '830820'       │ '1.204'     │ '800661'       │
│ 19      │ 'parse_value (simple)'              │ '2899559'      │ '0.345'     │ '2508130'      │
│ 20      │ 'parse_value (complex)'             │ '1261098'      │ '0.793'     │ '1196780'      │
│ 21      │ 'parse_dimension'                   │ '12025011'     │ '0.083'     │ '17404295'     │
│ 22      │ 'walk (node.type)|Nerdy.dev'        │ '2475'         │ '404.034'   │ '2430'         │
│ 23      │ 'walk (node.type_name)|Nerdy.dev'   │ '2275'         │ '439.498'   │ '2218'         │
│ 24      │ 'traverse|Nerdy.dev'                │ '3423'         │ '292.164'   │ '3347'         │
│ 25      │ 'walk (node.type)|CSS Tricks'       │ '275'          │ '3637.528'  │ '271'          │
│ 26      │ 'walk (node.type_name)|CSS Tricks'  │ '250'          │ '4007.874'  │ '247'          │
│ 27      │ 'traverse|CSS Tricks'               │ '377'          │ '2649.717'  │ '375'          │
│ 28      │ 'walk (node.type)|Bootstrap'        │ '1269'         │ '787.938'   │ '1259'         │
│ 29      │ 'walk (node.type_name)|Bootstrap'   │ '1165'         │ '858.074'   │ '1163'         │
│ 30      │ 'traverse|Bootstrap'                │ '1812'         │ '551.887'   │ '1775'         │
│ 31      │ 'walk (node.type)|Tailwind'         │ '87'           │ '11464.414' │ '88'           │
│ 32      │ 'walk (node.type_name)|Tailwind'    │ '82'           │ '12221.387' │ '82'           │
│ 33      │ 'traverse|Tailwind'                 │ '125'          │ '7992.658'  │ '127'          │
└─────────┴─────────────────────────────────────┴────────────────┴─────────────┴────────────────┘

── Parse+Walk memory ──

┌─────────┬──────────────┬───────────┬───────────┐
│ (index) │ File         │ Size      │ Memory    │
├─────────┼──────────────┼───────────┼───────────┤
│ 0       │ 'Nerdy.dev'  │ '111 KB'  │ '3.6 MB'  │
│ 1       │ 'CSS Tricks' │ '1308 KB' │ '37.9 MB' │
│ 2       │ 'Bootstrap'  │ '227 KB'  │ '6.9 MB'  │
│ 3       │ 'Tailwind'   │ '2865 KB' │ '86.2 MB' │
└─────────┴──────────────┴───────────┴───────────┘

github-actions · 2026-10-15T20:54:40Z

⚠️ Package Size Increase

📦 Package	📏 Base Size	📏 Source Size	📈 Size Change
@projectwallace/css-parser	46.1 kB	46.1 kB	+13 B

pkg-pr-new · 2026-10-15T20:55:01Z

Open in StackBlitz

npm i https://pkg.pr.new/@projectwallace/css-parser@80197b0

commit: 80197b0

bartveneman · 2026-10-15T20:55:55Z

before:

┌─────────┬─────────────────────────────────────┬────────────────┬─────────────┬────────────────┐
│ (index) │ Benchmark                           │ mode (ops/sec) │ mode (µs)   │ mean (ops/sec) │
├─────────┼─────────────────────────────────────┼────────────────┼─────────────┼────────────────┤
│ 0       │ 'tokenize|Nerdy.dev'                │ '689'          │ '1452.257'  │ '681'          │
│ 1       │ 'parse|Nerdy.dev'                   │ '548'          │ '1823.520'  │ '544'          │
│ 2       │ 'parse (no sub-parsing)|Nerdy.dev'  │ '1188'         │ '841.597'   │ '1172'         │
│ 3       │ 'tokenize|CSS Tricks'               │ '76'           │ '13076.735' │ '76'           │
│ 4       │ 'parse|CSS Tricks'                  │ '48'           │ '20634.975' │ '48'           │
│ 5       │ 'parse (no sub-parsing)|CSS Tricks' │ '162'          │ '6178.260'  │ '160'          │
│ 6       │ 'tokenize|Bootstrap'                │ '447'          │ '2237.639'  │ '440'          │
│ 7       │ 'parse|Bootstrap'                   │ '308'          │ '3248.360'  │ '303'          │
│ 8       │ 'parse (no sub-parsing)|Bootstrap'  │ '599'          │ '1668.656'  │ '593'          │
│ 9       │ 'tokenize|Tailwind'                 │ '35'           │ '28433.282' │ '35'           │
│ 10      │ 'parse|Tailwind'                    │ '24'           │ '42156.749' │ '24'           │
│ 11      │ 'parse (no sub-parsing)|Tailwind'   │ '53'           │ '18729.850' │ '52'           │
│ 12      │ 'parse_selector (simple)'           │ '2454771'      │ '0.407'     │ '2014199'      │
│ 13      │ 'parse_selector (complex)'          │ '848457'       │ '1.179'     │ '814649'       │
│ 14      │ 'parse_selector_list (simple)'      │ '1089372'      │ '0.918'     │ '1027563'      │
│ 15      │ 'parse_selector_list (complex)'     │ '336977'       │ '2.968'     │ '316758'       │
│ 16      │ 'parse_atrule_prelude (media)'      │ '776699'       │ '1.287'     │ '742836'       │
│ 17      │ 'parse_atrule_prelude (supports)'   │ '1030397'      │ '0.970'     │ '992589'       │
│ 18      │ 'parse_declaration'                 │ '799386'       │ '1.251'     │ '776636'       │
│ 19      │ 'parse_value (simple)'              │ '3048223'      │ '0.328'     │ '2512025'      │
│ 20      │ 'parse_value (complex)'             │ '1244555'      │ '0.804'     │ '1182495'      │
│ 21      │ 'parse_dimension'                   │ '12025011'     │ '0.083'     │ '17241241'     │
│ 22      │ 'walk (node.type)|Nerdy.dev'        │ '2433'         │ '411.076'   │ '2398'         │
│ 23      │ 'walk (node.type_name)|Nerdy.dev'   │ '2270'         │ '440.606'   │ '2210'         │
│ 24      │ 'traverse|Nerdy.dev'                │ '3308'         │ '302.300'   │ '3249'         │
│ 25      │ 'walk (node.type)|CSS Tricks'       │ '266'          │ '3761.898'  │ '263'          │
│ 26      │ 'walk (node.type_name)|CSS Tricks'  │ '251'          │ '3985.700'  │ '248'          │
│ 27      │ 'traverse|CSS Tricks'               │ '379'          │ '2638.638'  │ '374'          │
│ 28      │ 'walk (node.type)|Bootstrap'        │ '1279'         │ '781.561'   │ '1279'         │
│ 29      │ 'walk (node.type_name)|Bootstrap'   │ '1167'         │ '857.027'   │ '1162'         │
│ 30      │ 'traverse|Bootstrap'                │ '1805'         │ '553.868'   │ '1781'         │
│ 31      │ 'walk (node.type)|Tailwind'         │ '88'           │ '11400.453' │ '89'           │
│ 32      │ 'walk (node.type_name)|Tailwind'    │ '82'           │ '12163.111' │ '82'           │
│ 33      │ 'traverse|Tailwind'                 │ '124'          │ '8063.065'  │ '126'          │
└─────────┴─────────────────────────────────────┴────────────────┴─────────────┴────────────────┘

── Parse+Walk memory ──

┌─────────┬──────────────┬───────────┬───────────┐
│ (index) │ File         │ Size      │ Memory    │
├─────────┼──────────────┼───────────┼───────────┤
│ 0       │ 'Nerdy.dev'  │ '111 KB'  │ '3.6 MB'  │
│ 1       │ 'CSS Tricks' │ '1308 KB' │ '37.9 MB' │
│ 2       │ 'Bootstrap'  │ '227 KB'  │ '6.9 MB'  │
│ 3       │ 'Tailwind'   │ '2865 KB' │ '86.2 MB' │
└─────────┴──────────────┴───────────┴───────────┘

perf: more tokenizer inlining

80197b0

bartveneman changed the title ~~perf: more tokenizer inlining~~ perf: more tokenizer inlining (5% faster tokenizing) Oct 15, 2026

bartveneman merged commit d221b82 into main Oct 15, 2026
16 checks passed

bartveneman deleted the perf/inline-whitesace-newline-tokenizer branch October 15, 2026 20:59

Navigation Menu

Repository navigation

perf: more tokenizer inlining (5% faster tokenizing) - #309

perf: more tokenizer inlining (5% faster tokenizing)#309
bartveneman merged 1 commit into
mainfrom
perf/inline-whitesace-newline-tokenizer

bartveneman commented Oct 15, 2026 •

edited

Loading

Uh oh!

bartveneman commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

Uh oh!

pkg-pr-new Bot commented Oct 15, 2026

Uh oh!

bartveneman commented Oct 15, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

bartveneman commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Benchmarks (mean ops/sec, before → after)

Uh oh!

bartveneman commented Oct 15, 2026

Uh oh!

github-actions Bot commented Oct 15, 2026

⚠️ Package Size Increase

Uh oh!

pkg-pr-new Bot commented Oct 15, 2026

Uh oh!

bartveneman commented Oct 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

bartveneman commented Oct 15, 2026 •

edited

Loading

bartveneman commented Oct 15, 2026 •

edited

Loading