diff --git a/.cspell.yml b/.cspell.yml index 9da23446..997f8c89 100644 --- a/.cspell.yml +++ b/.cspell.yml @@ -46,6 +46,7 @@ words: - Uchechukwu - webdev - Welsch + - llms - KubeVirt - kubevirt - Quickstart diff --git a/docs/analysis/criteria.md b/docs/analysis/criteria.md index 9045bb01..b71d9610 100644 --- a/docs/analysis/criteria.md +++ b/docs/analysis/criteria.md @@ -15,7 +15,7 @@ referred to in the criteria. How your technical information is organized, laid out, and presented. This includes structure (pages/subpages/sections/subsections), information types -(conceptual/instructional/reference), and matching documentation content to user +(conceptual/instructional/reference), and content that matches user expectations. We evaluate on the following: @@ -23,7 +23,8 @@ We evaluate on the following: - Is there high level conceptual content? - Is every product feature documented? - Does the documentation define all user roles (personas) for the product? -- Are there instructions (tasks, tutorials) documented for features? +- Do instructions (tasks, tutorials) cover the intended purposes of all + user-facing features? - Are there instructions for all major use cases for each user role? - Are tasks organized by user role and use case? - Does instructional content demonstrate atomicity — are individual tasks @@ -31,8 +32,8 @@ We evaluate on the following: - Are tasks written as numbered step-by-step instructions? - Do task descriptions in headings and the TOC describe the task with a verb phrase? -- Is the documentation free of any key features which are documented but missing - task documentation? +- Are there any features that are documented (for example, described + conceptually) but are missing task documentation? - Is the “happy path” — the most common use case — documented? - If the documentation does not suffice, is there a clear escalation path for users needing more help? @@ -60,7 +61,8 @@ We evaluate on the following: - Are different types of installation documented (development, test, production) if necessary? - If needed, are multiple OSes documented? -- Do users know where to go after reading the getting started guide? +- Is there guidance for what to read next after reading the getting started + guide? - Is your new user content clearly signposted on your site’s homepage or at the top of your information architecture? - Is there easily copy-pastable sample code or other example content? @@ -126,6 +128,23 @@ We evaluate on the following: [Inclusive Naming Initiative](https://inclusivenaming.org) website? - Does the project use language like "simple", "easy", etc.? +### AI Optimization and Discoverability + +Tools and techniques are emerging that enable AI agents to use documentation as +input. For example, many repositories now include "AI-friendly" Markdown files +that enable LLMs to ingest content. We encourage projects to optimize for +AI-based documentation use. + +We evaluate on the following: + +- Is there an llms.txt file in the root of the website or documentation + repository? + + The leading proposal for an AI-enabling file for websites is at + [llms-txt](https://llmstxt.org/ llms.txt). + + Task can be accomplished manually or by tools. + ## Contributor documentation ### Communication methods documented @@ -150,10 +169,14 @@ Example: We evaluate on the following: -- Are docs issues well-triaged? +- Are docs issues triaged with respect to effort level and required expertise? - Is there a clearly marked way for new contributors to make code or documentation contributions (i.e. a “good first issue” label)? -- Are issues well-documented (i.e., more than just a title)? +- Are issues documented with: + - What is missing, incorrect, or improperly presented? + - What impact the issue has on documentation users? + - A suggested remedy? + - An estimate of the effort required to address the issue? - Are issues maintained for staleness? Example: @@ -172,7 +195,7 @@ We evaluate on the following: - Do you have a community repository or section on your website? - Is there a document specifically welcoming new contributors and documenting a first contribution process? -- Can new users find where to get help? +- Are there prominently displayed resources for new users to find help? Example: @@ -184,7 +207,7 @@ One of the CNCF’s core project values is open governance. We evaluate on the following: -- Is project governance clearly documented? +- Does the project governance align with CNCF guidelines and values? Example: @@ -239,8 +262,8 @@ requirements for sandbox projects._ Incubating status. - Is there rudimentary **project documentation**, or a placeholder or substitute? - - It is acceptable at this maturity level to link out to documentation that - hasn't yet been integrated into the website. + - It is acceptable at this maturity level to link to documentation that hasn't + yet been integrated into the website. - _Example_: website with a single homepage, without any documentation or, as was mentioned above, linking out to an external (preexisting) source for docs. @@ -266,7 +289,7 @@ requirements for sandbox projects._ #### Graduated - Are follow-through actions from the [Docs assessment][] complete? -- Does **project documentation** fully addresses the needs of key stakeholders? +- Does **project documentation** fully addresses the needs of stakeholders? #### Archived @@ -274,7 +297,7 @@ requirements for sandbox projects._ - Is the archived status of the project obvious to those visiting the website, such as through the use of a prominent banner? - If a successor project exists, are there links to its website and/or migration - documentation. + documentation? [archived state]: https://docs.github.com/en/repositories/archiving-a-github-repository/archiving-repositories @@ -295,12 +318,8 @@ We evaluate on the following: - Is the website usable from mobile? - Are doc pages readable? -- Are all or most website features accessible from mobile -- such as the - top-nav, site search, and in-page table of contents? - Might a [mobile-first][] design make sense for your project? -- Are color contrasts significant enough for color-impaired readers? - Are most website features usable using a keyboard only? -- Does text-to-speech offer listeners a good experience? [mobile-first]: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first @@ -310,8 +329,8 @@ Plan for suitable [accessibility][] measures for your website. We evaluate on the following: - Are color contrasts significant enough for color-impaired readers? -- Are most website features usable using a keyboard only? -- Does text-to-speech offer listeners a good experience? +- Are all vital website features usable using a keyboard only? +- Does text-to-speech offer listeners a usable experience? It is up to each project to set their own guidelines. diff --git a/docs/analysis/howto.md b/docs/analysis/howto.md index 19a71825..1a00c5ec 100644 --- a/docs/analysis/howto.md +++ b/docs/analysis/howto.md @@ -218,7 +218,7 @@ Create issues in the project documentation GitHub repository for: - An umbrella issue that provides a context for the previously created individual issues. -[analyses]: ../../analyses/ +[analyses]: ../../analyses [criteria]: ./criteria.md [project maturity level]: https://www.cncf.io/project-metrics [templates]: ./templates/index.md diff --git a/docs/analysis/templates/analysis.md b/docs/analysis/templates/analysis.md index 2551255f..5df43f8a 100644 --- a/docs/analysis/templates/analysis.md +++ b/docs/analysis/templates/analysis.md @@ -3,6 +3,7 @@ title: _PROJECT_ Documentation Analysis tags: [_PROJECT_] created: YYYY-MM-DD modified: YYYY-MM-DD +toc_max_heading_level: 4 author: _NAME_ (@_HANDLE_) --- @@ -23,6 +24,9 @@ TO USE THIS TEMPLATE, search and replace the named IDs: - `_PROJECT-DOC-REPO_`: repository where the project technical documentation is stored; this might be its own repo or a directory in the project main repo +Replace link placeholders in brackets, such as [_PROJECT_][project-website], +with a Markdown-style link. + For the analysis procedure, see [Analysis how-to](../howto.md). > Note: delete this "About this template" section after you have customized this @@ -40,6 +44,17 @@ prerequisite for program graduation. The documentation analysis is the first step of a CNCF process aimed at assisting projects with their documentation efforts. +### AI assisted analysis + +> Author's note: Remove if not applicable, otherwise edit as needed. + +The following parts of this analysis were generated by AI: + +- Overall comments, following the section heading, on the areas in that section. +- Answers to the criteria questions for an area. +- Comments, including strengths and weaknesses, for an area. +- Recommendations for improvements in each area. + ### Purpose This document was written to analyze the current state of _PROJECT_ @@ -86,11 +101,14 @@ This document is divided into three sections that represent three major areas of concern: - **Project documentation:** concerns documentation for users of the _PROJECT_ - software, aimed at people who intend to use the project software + software, aimed at people who intend to use the project software. - **Contributor documentation:** concerns documentation for new and existing - contributors to the _PROJECT_ OSS project + contributors to the _PROJECT_ OSS project. - **Website:** concerns the mechanics of publishing the documentation, and - includes branding, website structure, and maintainability + includes branding, website structure, and maintainability. + +A fourth area of concern, AI optimization and discovery, might be included in +this analysis. If so, it is discussed in a separate document, `ai-guidance.md`. Each section begins with summary ratings based on a rubric with appropriate [criteria] for the section, then proceeds to: @@ -100,6 +118,12 @@ Each section begins with summary ratings based on a rubric with appropriate - **Recommendations**: suggested changes that would improve the effectiveness of the documentation. +> Author's note: We recommend writing implementation notes only for large +> projects. An implementation plan can bridge the gap between the high-level +> recommendations in this document and individual issues entered in the doc +> repo. Remove the sentences about the implementation doc if you didn't write +> one. + The accompanying [implementation] document breaks the recommendations down into concrete actions that can be implemented by project contributors. Its focus is on drilling down to specific, achievable work that can be completed in @@ -168,15 +192,15 @@ Documentation rubric. #### Information architecture -The overall structure (pages/subpages/sections/subsections) of your project +The overall structure (pages/subpages/sections/subsections) of the project documentation. We evaluate on the following: - Is there high level conceptual/“About” content? Is the documentation feature complete? (i.e., each product feature is documented) - Are there step-by-step instructions (tasks, tutorials) documented for features? -- Are there any key features which are documented but missing task - documentation? +- Are there any features that are documented (for example, described + conceptually) but are missing task documentation? - Is the “happy path”/most common use case documented? Does task and tutorial content demonstrate atomicity and isolation of concerns? (Are tasks clearly named according to user goals?) @@ -195,8 +219,8 @@ specifically for them. We evaluate on the following: - Is installation documented step-by-step? - If needed, are multiple OSes documented? - Do users know where to go after reading the getting started guide? -- Is your new user content clearly signposted on your site’s homepage or at the - top of your information architecture? +- Is the new user content clearly signposted on the site’s homepage or at the + top of the information architecture? - Is there sample code or other example content that can easily be copy-pasted? #### Content maintainability & site mechanics @@ -206,10 +230,10 @@ become large maintenance burdens, particularly if you don’t plan for them. We evaluate on the following: -- Is your documentation searchable? +- Is the documentation searchable? - Are you planning for localization/internationalization with regards to site directory structure? Is a localization framework present? -- Do you have a clearly documented method for versioning your content? +- Do you have a clearly documented method for versioning the content? #### Content creation processes @@ -220,7 +244,7 @@ We evaluate on the following: - Is there a clearly documented (ongoing) contribution process for documentation? -- Does your code release process account for documentation creation & updates? +- Does the code release process account for documentation creation & updates? - Who reviews and approves documentation pull requests? - Does the website have a clear owner/maintainer? @@ -291,8 +315,8 @@ to reach you. We evaluate on the following: - Is there a Slack/Discord/Discourse/etc. community and is it prominently linked - from your website? -- Is there a direct link to your GitHub organization/repository? + from the website? +- Is there a direct link to the GitHub organization/repository? - Are weekly/monthly project meetings documented? Is it clear how someone can join those meetings? - Are mailing lists documented? @@ -315,8 +339,8 @@ in easily? We evaluate on the following: -- Do you have a community repository or section on your website? -- Is there a document specifically for new contributors/your first contribution? +- Do you have a community repository or section on the website? +- Is there a document specifically for new contributors/the first contribution? - Do new users know where to get help? #### Project governance documentation @@ -402,6 +426,11 @@ submodules][git-submodules]. If a project chooses to keep source files in multiple repos, they need a clearly documented strategy for managing mirrored files and new contributions. +We evaluate on the following: + +- Does the project have a single source for its documentation? If not, is there + a reason? + #### Minimal website requirements Listed here are the minimal website requirements for projects based on their @@ -425,22 +454,27 @@ only two levels for which a tech docs analysis can be requested.) https://github.com/cncf/toc/tree/main/process#ii-stages---definitions--expectations [cncf-servicedesk]: https://servicedesk.cncf.io +We evaluate on the following: + +- Are most of the applicable guidelines satisfied as described in the Website + guidelines and checklist? + #### Usability, accessibility and devices Most CNCF websites are accessed from mobile and other non-desktop devices at -least 10-20% of the time. Planning for this early in your website's design will +least 10-20% of the time. Planning for this early in the website's design will be much less effort than retrofitting a desktop-first design. - Is the website usable from mobile? - Are doc pages readable? - Are all / most website features accessible from mobile -- such as the top-nav, site search and in-page table of contents? -- Might a [mobile-first] design make sense for your project? +- Might a [mobile-first] design make sense for the project? [mobile-first]: https://developer.mozilla.org/en-US/docs/Web/Progressive_web_apps/Responsive/Mobile_first -Plan for suitable [accessibility][] measures for your website. For example: +Plan for suitable [accessibility][] measures for the website. For example: - Are color contrasts significant enough for color-impaired readers? - Are most website features usable using a keyboard only? @@ -478,16 +512,16 @@ We evaluate on the following: #### SEO, Analytics and site-local search -SEO helps users find your project and it's documentation, and analytics helps -you monitor site traffic and diagnose issues like page 404s. Intra-site search, -while optional, can offer your readers a site-focused search results. +SEO helps users find the project and it's documentation, and analytics helps you +monitor site traffic and diagnose issues like page 404s. Intra-site search, +while optional, can offer the readers a site-focused search results. We evaluate on the following: - Analytics: - Is analytics enabled for the production server? - Is analytics disabled for all other deploys? - - If your project used Google Analytics, have you migrated to GA4? + - If the project used Google Analytics, have you migrated to GA4? - Can Page-not-found (404) reports easily be generated from you site analytics? Provide a sample of the site's current top-10 404s. - Is site indexing supported for the production server, while disabled for @@ -503,7 +537,7 @@ project maintainers aren’t web developers. We evaluate on the following: -- Is your website tooling well supported by the community (i.e., Hugo with the +- Is the website tooling well supported by the community (i.e., Hugo with the Docsy theme) or commonly used by CNCF projects (our recommended tech stack?) - Are you actively cultivating website maintainers from within the community? - Are site build times reasonable? @@ -511,7 +545,7 @@ We evaluate on the following: #### Other -- Is your website accessible via HTTPS? +- Is the website accessible via HTTPS? - Does HTTP access, if any, redirect to HTTPS? ### Recommendations