Analysis templates suggested updates

.cspell.yml

@@ -46,3 +46,4 @@ words:
   - Uchechukwu
   - webdev
   - Welsch
+  - llms

docs/analysis/criteria.md

@@ -15,24 +15,25 @@ referred to in the criteria.
 
 How your technical information is organized, laid out, and presented. This
 includes structure (pages/subpages/sections/subsections), information types
-(conceptual/instructional/reference), and matching documentation content to user
+(conceptual/instructional/reference), and content that matches user
 expectations.
 
 We evaluate on the following:
 
 - Is there high level conceptual content?
 - Is every product feature documented?
 - Does the documentation define all user roles (personas) for the product?
-- Are there instructions (tasks, tutorials) documented for features?
+- Do instructions (tasks, tutorials) cover the intended purposes of all
+  user-facing features?
 - Are there instructions for all major use cases for each user role?
 - Are tasks organized by user role and use case?
 - Does instructional content demonstrate atomicity — are individual tasks
   clearly named according to their goals?
 - Are tasks written as numbered step-by-step instructions?
 - Do task descriptions in headings and the TOC describe the task with a verb
   phrase?
-- Is the documentation free of any key features which are documented but missing
-  task documentation?
+- Are there any features that are documented (for example, described
+  conceptually) but are missing task documentation?
 - Is the “happy path” — the most common use case — documented?
 - If the documentation does not suffice, is there a clear escalation path for
   users needing more help?
@@ -60,7 +61,8 @@ We evaluate on the following:
 - Are different types of installation documented (development, test, production)
   if necessary?
 - If needed, are multiple OSes documented?
-- Do users know where to go after reading the getting started guide?
+- Is there guidance for what to read next after reading the getting started
+  guide?
 - Is your new user content clearly signposted on your site’s homepage or at the
   top of your information architecture?
 - Is there easily copy-pastable sample code or other example content?
@@ -126,6 +128,23 @@ We evaluate on the following:
   [Inclusive Naming Initiative](https://inclusivenaming.org) website?
 - Does the project use language like "simple", "easy", etc.?
 
+### AI Optimization and Discoverability
+
+Tools and techniques are emerging that enable AI agents to use documentation as
+input. For example, many repositories now include "AI-friendly" Markdown files
+that enable LLMs to ingest content. We encourage projects to optimize for
+AI-based documentation use.
+
+We evaluate on the following:
+
+- Is there an llms.txt file in the root of the website or documentation
+  repository?
+
+  The leading proposal for an AI-enabling file for websites is at
+  [llms-txt](https://llmstxt.org/ llms.txt).
+
+  Task can be accomplished manually or by tools.
+
 ## Contributor documentation
 
 ### Communication methods documented
@@ -150,10 +169,14 @@ Example:
 
 We evaluate on the following:
 
-- Are docs issues well-triaged?
+- Are docs issues triaged with respect to effort level and required expertise?
 - Is there a clearly marked way for new contributors to make code or
   documentation contributions (i.e. a “good first issue” label)?
-- Are issues well-documented (i.e., more than just a title)?
+- Are issues documented with:
+  - What is missing, incorrect, or improperly presented?
+  - What impact the issue has on documentation users?
+  - A suggested remedy?
+  - An estimate of the effort required to address the issue?
 - Are issues maintained for staleness?
 
 Example:
@@ -172,7 +195,7 @@ We evaluate on the following:
 - Do you have a community repository or section on your website?
 - Is there a document specifically welcoming new contributors and documenting a
   first contribution process?
-- Can new users find where to get help?
+- Are there prominently displayed resources for new users to find help?
 
 Example:
 
@@ -184,7 +207,7 @@ One of the CNCF’s core project values is open governance.
 
 We evaluate on the following:
 
-- Is project governance clearly documented?
+- Does the project governance align with CNCF guidelines and values?
 
 Example:
 
@@ -239,8 +262,8 @@ requirements for sandbox projects._
     Incubating status.
 - Is there rudimentary **project documentation**, or a placeholder or
   substitute?
-  - It is acceptable at this maturity level to link out to documentation that
-    hasn't yet been integrated into the website.
+  - It is acceptable at this maturity level to link to documentation that hasn't
+    yet been integrated into the website.
   - _Example_: website with a single homepage, without any documentation or, as
     was mentioned above, linking out to an external (preexisting) source for
     docs.
@@ -266,15 +289,15 @@ requirements for sandbox projects._
 #### Graduated
 
 - Are follow-through actions from the [Docs assessment][] complete?
-- Does **project documentation** fully addresses the needs of key stakeholders?
+- Does **project documentation** fully addresses the needs of stakeholders?
 
 #### Archived
 
 - Is the website repo in an [archived state][]?
 - Is the archived status of the project obvious to those visiting the website,
   such as through the use of a prominent banner?
 - If a successor project exists, are there links to its website and/or migration
-  documentation.
+  documentation?
 
 [archived state]:
   https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories
@@ -295,12 +318,8 @@ We evaluate on the following:
 
 - Is the website usable from mobile?
 - Are doc pages readable?
-- Are all or most website features accessible from mobile -- such as the
-  top-nav, site search, and in-page table of contents?
 - Might a [mobile-first][] design make sense for your project?
-- Are color contrasts significant enough for color-impaired readers?
 - Are most website features usable using a keyboard only?
-- Does text-to-speech offer listeners a good experience?
 
 [mobile-first]:
   https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first
@@ -310,8 +329,8 @@ Plan for suitable [accessibility][] measures for your website.
 We evaluate on the following:
 
 - Are color contrasts significant enough for color-impaired readers?
-- Are most website features usable using a keyboard only?
-- Does text-to-speech offer listeners a good experience?
+- Are all vital website features usable using a keyboard only?
+- Does text-to-speech offer listeners a usable experience?
 
 It is up to each project to set their own guidelines.
 

docs/analysis/howto.md

@@ -218,7 +218,7 @@ Create issues in the project documentation GitHub repository for:
 - An umbrella issue that provides a context for the previously created
   individual issues.
 
-[analyses]: ../../analyses/
+[analyses]: ../../analyses
 [criteria]: ./criteria.md
 [project maturity level]: https://www.cncf.io/project-metrics
 [templates]: ./templates/index.md

docs/analysis/templates/analysis.md

@@ -23,6 +23,9 @@ TO USE THIS TEMPLATE, search and replace the named IDs:
 - `_PROJECT-DOC-REPO_`: repository where the project technical documentation is
   stored; this might be its own repo or a directory in the project main repo
 
+Replace link placeholders in brackets, such as [_PROJECT_][project-website],
+with a Markdown-style link.
+
 For the analysis procedure, see [Analysis how-to](../howto.md).
 
 > Note: delete this "About this template" section after you have customized this
@@ -40,6 +43,17 @@ prerequisite for program graduation. The documentation analysis is the first
 step of a CNCF process aimed at assisting projects with their documentation
 efforts.
 
+### AI assisted analysis
+
+> Author's note: Remove if not applicable, otherwise edit as needed.
+
+The following parts of this analysis were generated by AI:
+
+- Overall comments, following the section heading, on the areas in that section.
+- Answers to the criteria questions for an area.
+- Comments, including strengths and weaknesses, for an area.
+- Recommendations for improvements in each area.
+
 ### Purpose
 
 This document was written to analyze the current state of _PROJECT_
@@ -86,11 +100,14 @@ This document is divided into three sections that represent three major areas of
 concern:
 
 - **Project documentation:** concerns documentation for users of the _PROJECT_
-  software, aimed at people who intend to use the project software
+  software, aimed at people who intend to use the project software.
 - **Contributor documentation:** concerns documentation for new and existing
-  contributors to the _PROJECT_ OSS project
+  contributors to the _PROJECT_ OSS project.
 - **Website:** concerns the mechanics of publishing the documentation, and
-  includes branding, website structure, and maintainability
+  includes branding, website structure, and maintainability.
+
+A fourth area of concern, AI optimization and discovery, might be included in
+this analysis. If so, it is discussed in a separate document, `ai-guidance.md`.
 
 Each section begins with summary ratings based on a rubric with appropriate
 [criteria] for the section, then proceeds to:
@@ -100,6 +117,12 @@ Each section begins with summary ratings based on a rubric with appropriate
 - **Recommendations**: suggested changes that would improve the effectiveness of
   the documentation.
 
+> Author's note: We recommend writing implementation notes only for large
+> projects. An implementation plan can bridge the gap between the high-level
+> recommendations in this document and individual issues entered in the doc
+> repo. Remove the sentences about the implementation doc if you didn't write
+> one.
+
 The accompanying [implementation] document breaks the recommendations down into
 concrete actions that can be implemented by project contributors. Its focus is
 on drilling down to specific, achievable work that can be completed in
@@ -168,15 +191,15 @@ Documentation rubric.
 
 #### Information architecture
 
-The overall structure (pages/subpages/sections/subsections) of your project
+The overall structure (pages/subpages/sections/subsections) of the project
 documentation. We evaluate on the following:
 
 - Is there high level conceptual/“About” content? Is the documentation feature
   complete? (i.e., each product feature is documented)
 - Are there step-by-step instructions (tasks, tutorials) documented for
   features?
-- Are there any key features which are documented but missing task
-  documentation?
+- Are there any features that are documented (for example, described
+  conceptually) but are missing task documentation?
 - Is the “happy path”/most common use case documented? Does task and tutorial
   content demonstrate atomicity and isolation of concerns? (Are tasks clearly
   named according to user goals?)
@@ -195,8 +218,8 @@ specifically for them. We evaluate on the following:
 - Is installation documented step-by-step?
 - If needed, are multiple OSes documented?
 - Do users know where to go after reading the getting started guide?
-- Is your new user content clearly signposted on your site’s homepage or at the
-  top of your information architecture?
+- Is the new user content clearly signposted on the site’s homepage or at the
+  top of the information architecture?
 - Is there sample code or other example content that can easily be copy-pasted?
 
 #### Content maintainability & site mechanics
@@ -206,10 +229,10 @@ become large maintenance burdens, particularly if you don’t plan for them.
 
 We evaluate on the following:
 
-- Is your documentation searchable?
+- Is the documentation searchable?
 - Are you planning for localization/internationalization with regards to site
   directory structure? Is a localization framework present?
-- Do you have a clearly documented method for versioning your content?
+- Do you have a clearly documented method for versioning the content?
 
 #### Content creation processes
 
@@ -220,7 +243,7 @@ We evaluate on the following:
 
 - Is there a clearly documented (ongoing) contribution process for
   documentation?
-- Does your code release process account for documentation creation & updates?
+- Does the code release process account for documentation creation & updates?
 - Who reviews and approves documentation pull requests?
 - Does the website have a clear owner/maintainer?
 
@@ -291,8 +314,8 @@ to reach you.
 We evaluate on the following:
 
 - Is there a Slack/Discord/Discourse/etc. community and is it prominently linked
-  from your website?
-- Is there a direct link to your GitHub organization/repository?
+  from the website?
+- Is there a direct link to the GitHub organization/repository?
 - Are weekly/monthly project meetings documented? Is it clear how someone can
   join those meetings?
 - Are mailing lists documented?
@@ -315,8 +338,8 @@ in easily?
 
 We evaluate on the following:
 
-- Do you have a community repository or section on your website?
-- Is there a document specifically for new contributors/your first contribution?
+- Do you have a community repository or section on the website?
+- Is there a document specifically for new contributors/the first contribution?
 - Do new users know where to get help?
 
 #### Project governance documentation
@@ -402,6 +425,11 @@ submodules][git-submodules].
 If a project chooses to keep source files in multiple repos, they need a clearly
 documented strategy for managing mirrored files and new contributions.
 
+We evaluate on the following:
+
+- Does the project have a single source for its documentation? If not, is there
+  a reason?
+
 #### Minimal website requirements
 
 Listed here are the minimal website requirements for projects based on their
@@ -425,22 +453,27 @@ only two levels for which a tech docs analysis can be requested.)
   https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations
 [cncf-servicedesk]: https://servicedesk.cncf.io
 
+We evaluate on the following:
+
+- Are most of the applicable guidelines satisfied as described in the Website
+  guidelines and checklist?
+
 #### Usability, accessibility and devices
 
 Most CNCF websites are accessed from mobile and other non-desktop devices at
-least 10-20% of the time. Planning for this early in your website's design will
+least 10-20% of the time. Planning for this early in the website's design will
 be much less effort than retrofitting a desktop-first design.
 
 - Is the website usable from mobile?
 - Are doc pages readable?
 - Are all / most website features accessible from mobile -- such as the top-nav,
   site search and in-page table of contents?
-- Might a [mobile-first] design make sense for your project?
+- Might a [mobile-first] design make sense for the project?
 
 [mobile-first]:
   https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first
 
-Plan for suitable [accessibility][] measures for your website. For example:
+Plan for suitable [accessibility][] measures for the website. For example:
 
 - Are color contrasts significant enough for color-impaired readers?
 - Are most website features usable using a keyboard only?
@@ -478,16 +511,16 @@ We evaluate on the following:
 
 #### SEO, Analytics and site-local search
 
-SEO helps users find your project and it's documentation, and analytics helps
-you monitor site traffic and diagnose issues like page 404s. Intra-site search,
-while optional, can offer your readers a site-focused search results.
+SEO helps users find the project and it's documentation, and analytics helps you
+monitor site traffic and diagnose issues like page 404s. Intra-site search,
+while optional, can offer the readers a site-focused search results.
 
 We evaluate on the following:
 
 - Analytics:
   - Is analytics enabled for the production server?
   - Is analytics disabled for all other deploys?
-  - If your project used Google Analytics, have you migrated to GA4?
+  - If the project used Google Analytics, have you migrated to GA4?
   - Can Page-not-found (404) reports easily be generated from you site
     analytics? Provide a sample of the site's current top-10 404s.
 - Is site indexing supported for the production server, while disabled for
@@ -503,15 +536,15 @@ project maintainers aren’t web developers.
 
 We evaluate on the following:
 
-- Is your website tooling well supported by the community (i.e., Hugo with the
+- Is the website tooling well supported by the community (i.e., Hugo with the
   Docsy theme) or commonly used by CNCF projects (our recommended tech stack?)
 - Are you actively cultivating website maintainers from within the community?
 - Are site build times reasonable?
 - Do site maintainers have adequate permissions?
 
 #### Other
 
-- Is your website accessible via HTTPS?
+- Is the website accessible via HTTPS?
 - Does HTTP access, if any, redirect to HTTPS?
 
 ### Recommendations