[TIKA-4943] add tika table and list handling

[henry-lindeman-glean]

Thanks for your contribution to Apache Tika! Your help is appreciated!

Before opening the pull request, please verify that

• there is an open issue on the Tika issue tracker which describes the problem or the improvement. We cannot accept pull requests without an issue because the change wouldn't be listed in the release notes.
• the issue ID (TIKA-XXXX)
  • is referenced in the title of the pull request
  • and placed in front of your commit messages surrounded by square brackets ([TIKA-XXXX] Issue or pull request title)
• commits are squashed into a single one (or few commits for larger changes)
• Tika builds and unit tests pass with ./mvnw clean install (clean test alone cannot resolve the pipes plugin zips)
• if you used a generative AI tool: follow the ASF Generative Tooling Guidance (Generated-by: <tool> in the commit message), and consider running the pre-flight in .skills/devs/pr-review/SKILL.md — fix what it finds; don't paste its report here
• there should be no conflicts when merging the pull request branch into the recent main branch. If there are conflicts, please try to rebase the pull request branch on top of a freshly pulled main branch
• if you add new module that downstream users will depend upon add it to relevant group in tika-bom/pom.xml.

We will be able to faster integrate your pull request if these conditions are met. If you have any questions how to fix your problem or about using Tika in general, please sign up for the Tika mailing list. Thanks!

❯ java -jar tika-app/target/tika-app-4.1.1-SNAPSHOT.jar -x ~/glean/tika/Downloadme/parsing-test-2/Section-1.one
INFO  [main] 11:26:22,159 org.apache.tika.cli.TikaCLI As a convenience, TikaCLI has turned on several non-default features
as specified in tika-app/src/main/resources/tika-config-default-single-file.json.
See: TIKA-2374, TIKA-4017, TIKA-4354 and TIKA-4472).
This is not the default behavior in Tika generally or in tika-server.
INFO  [main] 11:26:22,247 org.apache.tika.config.loader.TikaLoader temporary files go to java.io.tmpdir=/var/folders/r3/0ktdq26562d324x6wxwj10y40000gn/T
INFO  [main] 11:26:24,518 org.apache.tika.config.loader.ParserLoader text recognizers: none found among the loaded parsers; images and rendered pages are not enriched
<?xml version="1.0" encoding="UTF-8"?><html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta name="tk:content-type-magic-detected" content="application/onenote; format=one"/>
<meta name="tk:resource-name" content="Section-1.one"/>
<meta name="Content-Length" content="86561"/>
<meta name="tk:parsed-by" content="org.apache.tika.parser.CompositeParser"/>
<meta name="tk:parsed-by" content="org.apache.tika.parser.DefaultParser"/>
<meta name="tk:parsed-by" content="org.apache.tika.parser.microsoft.onenote.OneNoteParser"/>
<meta name="Content-Type" content="application/onenote; format=one"/>
<title/>
</head>
<body><div class="page"><p>Page 1</p>
<p>Wednesday, August 12, 2026</p>
<p>10:24 AM</p>
<p>Text</p>
<p>More text</p>
<p>Bold Text</p>
<p>Italic Text</p>
<p>Underlined test</p>
<p>Red text</p>
<p>Highlighted text</p>
<p>ddd</p>
</div>
<div class="page"><p>Page sdskgjhsdlf</p>
<p>Wednesday, August 12, 2026</p>
<p>10:26 AM</p>
<ul>	<li><p>Bullet 1</p>
</li>
	<li><p>Bullet 2</p>
<ul>	<li><p>Indented bullet</p>
<ul>	<li><p>Indented square</p>
</li>
</ul>
</li>
</ul>
</li>
</ul>
<ol type="1">	<li><p>Number 1</p>
</li>
	<li><p>Number 2</p>
<ol type="a">	<li><p>Letter a</p>
<ol type="i">	<li><p>Numeral I</p>
</li>
</ol>
</li>
</ol>
</li>
</ol>
<ol type="1">	<li><p>Outdented number</p>
</li>
</ol>
<p>More text</p>
<table><tr>	<td><p>Table cell 1a</p>
</td>	<td/>	<td/>	<td><p>Table cell 4a</p>
</td></tr>
<tr>	<td/>	<td><p>Table cell 2b</p>
</td>	<td><p>Table cell 3b</p>
</td>	<td><p>Table cell 4b</p>
</td></tr>
</table>
<p>Comic sans</p>
<p>Dd</p>
<p>D</p>
</div>
</body></html>%                                                                                                      

[tballison]

Let me know what you think of this from my 🤖

 Findings

  1. Page-node GUIDs end up in entityGuids. I logged which object type supplied each GUID in the classic walker. Every entityGuids value on the fixtures came from JCID 0x0B. The PR defines that type as
     OneNoteJcid.PAGE_NODE but neither switch handles it, so it falls through to the default case (OneNoteTreeWalker.addClassicEntityGuid, MSOneStorePackage.recordEntityGuid). As a result, entityGuids in
     practice is undocumented page-node IDs. testOneNote1 and testOneNote2 return the same two values. Either map page nodes to their own key or drop the catch-all key.
  2. The PR doesn't show that these are the IDs OneNote itself uses. The section-node (0x07) GUIDs are dropped and the file header's GUID is used as the section GUID instead. The page GUID comes from the
     page-metadata object (0x30), not the page node (0x0B), and the two differ in every fixture. The tests only check the GUID format, not the values. Since these keys freeze once released, ask the author
     to match them against OneNote's "Copy Link to Page" section-id/page-id for one fixture, then pin those exact values in tests on testOneNoteFromOffice365.one (newer format) and testOneNote2.one
     (classic).
  3. Classic and newer formats may disagree on old page versions. test-tika-3970-dupetext.one (a version-history file) yields 2 page GUIDs but only 1 page-series GUID. The classic walker records GUIDs from
     every revision, while the newer-format path skips old page versions (it has a test for that). Check whether the classic path is picking up version-history pages.
  4. The output change isn't in CHANGES. Newer-format files now get <div class="page" id="{GUID}">; classic files still have no page divs. Add a CHANGES line, or keep the change to metadata only.

  Maintainer decisions
  - SECTION_GUIDS is a bag, but a .one file is one section. A single onenote:sectionGuid may be the better key to freeze.
  - OneNoteJcid and GUID.fromMicrosoftBytes become new public API.

  Hygiene
  - The 100,000-value cap and the type-to-key switch are written twice: once in OneNoteGuidCollector, once inline in MSOneStorePackage with its own MAX_GUID_COUNT. Share one collector.
  - The newer-format bags are in insertion order while the classic ones are sorted (the existing author keys are sorted in both).

[tballison]

Just some 🤖 nits. Some look reasonable.

 Edge cases

  1. No real file pins the new output. All 14 new tests are hand-built object graphs. Of the 16 .one fixtures only
     testOneNote2007OrEarlier.one changes (two <ul>, six <li>, verified by diffing main vs PR), and nothing asserts it. No
     fixture has a table or a numbered list. Ask the author to contribute the Section-1.one from the PR description (their
     own content, 86 KB, covers table, nested bullets, 1/a/i, restart) with an OneNoteParserTest asserting the shape, plus
     one assertion on the 2007 fixture.
  2. Fallback pass order. walkObjectGroupRoots walks deferred tables before deferred list containers. A table nested in an
     unreached outline element is emitted as a bare root, then the element finds it visited and loses it. One pass over
     deferredObjects in group order gives the same completeness and keeps nesting; only true cycles stay order-arbitrary.
     Degraded path only, low.

  Hygiene
  - OneNoteStructureJcid duplicates OneNoteJcid down to the javadoc line. Move the six constants there.
  - listStyleComparisonCache memoizes Arrays.equals on 4–6-byte arrays via nested IdentityHashMap; costs more than it saves.
    Delete it and the reflection test on the private field; ListStyle becomes static.
  - indent from RgOutlineIndentDistance on outline elements: the spec puts that property on jcidOutlineNode; elements carry
    OutlineElementChildLevel. I probed the fixture: null on every item. Drop the field.
  - testFallbackRecoversDepthSkippedTableDescendants reflects into two private methods; a 999-object chain reaches the depth
    cap through the public walk.

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

5 open findings

• walkRootObjectcan be invoked withobject == nullfromwalkObjectGroupRoots` (it iterates raw… · New
• walkTree()clearsdepthSkippedObjects, but the newly added caches (numberListInfoCache` and… · New
• This unit test allocates multiple 1MB arrays and builds 2001 list items; in constrained CI… · New
• This test depends on private method names/signatures and even the binary name of a nested type… · New
• This test reaches into a private field by name, which tightly couples it to an implementation… · New

What changed in this PR

Adds XHTML structure preservation for OneNote (FSSHTTPB) table and list content, and documents the enhancement for the upcoming release.

Changes:

• Emit <table>/<tr>/<td> and <ol>/<ul>/<li> markup based on OneNote JCID structure and list metadata during tree walking.
• Add list-style parsing, grouping, and caching to keep sibling list items within shared containers.
• Add extensive unit tests for tables/lists, fallback traversal, depth limits, and list-style edge cases; update release notes.

File	Description
.../​MSOneStorePackage.java	Adds table/list XHTML emission, list-style parsing/grouping, and depth-skip fallback behavior.
.../​OneNoteStructureJcid.java	Introduces JCID index constants for structure nodes used by the walker.
.../​MSOneStorePackageTest.java	Adds new tests covering table/list structure and fallback traversal behavior.
CHANGES.txt	Documents the new OneNote table/list structure preservation feature (TIKA-4943).

🧠 Review effort: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[henry-lindeman-glean]

here's the first three lines of walkObject:

    private void walkObject(RevisionStoreObject object,
                             Map<ExGuid, RevisionStoreObject> objectsById, Set<ExGuid> visited,
                             AuthorRole authorRole, OneNoteTreeWalkerOptions options,
                             Metadata metadata, XHTMLContentHandler xhtml, int depth,
                             EmbeddedResourceInfo inheritedResourceInfo)
            throws SAXException, TikaException, IOException {
        if (object == null) {
            return;
        }

[tballison]

I was getting stackoverflow with jacoco. I moved the limit to something smaller, but still huge compared with what we'd expect in a real file.

[tballison]

Will merge once green. Thank you!