build(docx): Add Sitemap #2382

ZakaryH · 2025-02-18T23:32:36Z

Motivations

having a sitemap is a good practice in general. in addition to that, we'd like to help tools like Cursor be able to better parse our documentation.

Changes

Added

added a step in the build script to generate a sitemap for our docs site. this is NOT including all the storybook stories, though perhaps we want to consider adding those too for even more context later.

at the end of the build script we will parse our routes and component list to generate the xml ourselves. the output will be placed in the storybook-static directory which is roughly equivalent to a public folder where assets would go.

Changed

Deprecated

Removed

Fixed

Security

Testing

to see the output yourself, run the npm run build command from the top level package.json, or the packages/site and then the sitemap.xml file will be in storybook-static

testing the changes with Cursor is a bit un-scientific but what I've been doing is validating the Cloudflare preview url's sitemap.xml, and then using Cursor's @web command with the top level docs url then asking it questions about Atlantis components

if you use prod, I was getting back a message effectively saying it wasn't sure.

if you use the preview url, I'm getting back useful and (mostly) correct details about the Button. it's definitely hallucinating some things like a medium size

Changes can be
tested via Pre-release

In Atlantis we use Github's built in pull request reviews.

cloudflare-workers-and-pages · 2025-02-18T23:37:19Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`2499d78`
Status:	✅ Deploy successful!
Preview URL:	https://50a0ac56.atlantis.pages.dev
Branch Preview URL:	https://sitemap-please-and-thank-you.atlantis.pages.dev

View logs

scotttjob · 2025-02-19T20:56:21Z

.circleci/config.yml

+      - run:
+          name: Create sitemap generator script
+          command: |
+            cat \<< 'EOF' > generate-sitemap.js


Is it possible to roll this script up into just 'generate-sitemap.js' and call it from here?

Also does this generate a sitemap for just storybook or the whole site?

oh sorry this isn't actually ready for review. I was testing some github labels.

also, I was seeing what Cursor would come up with as a first solution which is what all this is.

I'm now considering not doing this in CI at all, and instead using Github Actions to generate the sitemap on merges to master since there's not really much value in generating a sitemap on every commit/CI run.

Okay! I just saw it was Open while clearing out my email for the day. Curious to see where this lands.

right now I think it's generating a sitemap for storybook only. the first output it had was.... empty, so I told it to check for storybook content and this is as far as it got with that.

…thank-you

cloudflare-workers-and-pages · 2025-02-20T21:28:58Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`9057c30`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-20T21:33:51Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`bb2e298`
Status:	🚫 Build failed.

View logs

cloudflare-workers-and-pages · 2025-02-20T21:33:57Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`2da29e9`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-20T21:50:38Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`758b732`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-20T22:01:13Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`758b732`
Status:	✅ Deploy successful!
Preview URL:	https://bd435216.atlantis.pages.dev
Branch Preview URL:	https://sitemap-please-and-thank-you.atlantis.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-21T17:31:58Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`49be2aa`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-21T17:37:30Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`49be2aa`
Status:	✅ Deploy successful!
Preview URL:	https://d7b129d9.atlantis.pages.dev
Branch Preview URL:	https://sitemap-please-and-thank-you.atlantis.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-21T17:43:37Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`1345672`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-21T17:46:07Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`9960c11`
Status:	⚡️ Build in progress...

View logs

cloudflare-workers-and-pages · 2025-02-21T17:48:51Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`1345672`
Status:	✅ Deploy successful!
Preview URL:	https://d02fdf26.atlantis.pages.dev
Branch Preview URL:	https://sitemap-please-and-thank-you.atlantis.pages.dev

View logs

cloudflare-workers-and-pages · 2025-02-21T17:51:35Z

Deploying atlantis with Cloudflare Pages

Latest commit:	`9960c11`
Status:	✅ Deploy successful!
Preview URL:	https://b8e8c549.atlantis.pages.dev
Branch Preview URL:	https://sitemap-please-and-thank-you.atlantis.pages.dev

View logs

package.json

sitemap-generator.js

nad182 · 2025-02-24T18:18:13Z

sitemap-generator.test.js

+      "/",
+      "/components",
+      "/design",
+      "/hooks",
+      "/guides",
+      "/patterns",
+      "/content",
+      "/changelog",


This is not a huge deal, since this is not something that would change frequently, but I was wondering whether this list should come from a shared constant or be generated, so that this test doesn't fail in case some of the routes get added, deleted or updated?

that's a fair point. we don't have any existing asset that we could refer to, so it would have to be something we generate which is pretty similar to the function of the sitemap generation script itself.

I also think this is something that wouldn't change very often, and the cost of updating a test is low enough that I'm in favor of that over creating another script to generate a file.

so if it's alright with you, let's wait until it becomes an annoyance and deal with it then.

nad182

The sitemap generation script looks good! I left a few comments with suggestions to improve readability and testing.

I tried testing it with Cursor, and unfortunately, I'm still not able to access Atlantis documentation (either via @Docs handle, where I added the link to the preview URL or via @Web + preview link):

Query 1 (@Atlantis) is the prod doc site link.
Query 2(@AtlantisDocs) is the preview docs site link.

Querying the web with the preview URL.

It does somewhat work if I specifically mention that I'm asking about the Atlantis component.
Querying the web with preview URL + specifying that I'm looking for the Atlantis component:

Co-authored-by: Nazarii L <[email protected]>

nad182 · 2025-02-24T20:40:33Z

package.json

@@ -30,7 +30,7 @@
    "lint:fix": "npm run lint:js -- --fix && npm run lint:css -- --fix && npm run lint:markdown -- --write",
    "lint:js": "eslint --ext .js,.jsx,.ts,.tsx,.mjs,.cjs . --report-unused-disable-directives",
    "lint:markdown": "prettier --check '**/*.{md,mdx}' --write",
-    "merge:dist": "mv ../../storybook-static dist/storybook && mv dist ../../storybook-static",
+    "merge:dist": "mv storybook-static packages/site/dist/storybook && mv packages/site/dist storybook-static",


nad182

This PR looks good! I tested generating sitemap locally with the build script and it works as expected.

Just wanted to note that my comment/issue about prompting with@Docs and @Web still stands. However, I don't think it's related to anything in this PR.

add sitemap generation step to CI

0dd30f4

ZakaryH added 8 commits February 18, 2025 15:47

install locally to avoid permission errors

58f4847

persist storybook build to create sitemap from

dd93c3e

use custom script instead of library to generate sitemap

ffcb457

fix syntax for CircleCI config

7e60b7d

adjust script paths, add error handling and logging

292769f

ensure double slashes after https:

30e2523

update custom script to find storybook format data

c1a13df

fix syntax for CI

dd9372c

ZakaryH added the Team UX Foundations label Feb 19, 2025

ZakaryH marked this pull request as ready for review February 19, 2025 20:27

ZakaryH requested a review from a team as a code owner February 19, 2025 20:27

jobbiebot bot added the review requested label Feb 19, 2025

ZakaryH added Team Atlantis review requested and removed review requested labels Feb 19, 2025

scotttjob reviewed Feb 19, 2025

View reviewed changes

ZakaryH added 4 commits February 19, 2025 16:28

add script to existing build script to generate sitemap

dddf61c

add generated file warning comment to sitemap template

6fd5323

remove ci generation

bb2e298

Merge remote-tracking branch 'origin/master' into sitemap-please-and-…

9057c30

…thank-you

move back to top level before calling generate script

2da29e9

fix cd command

758b732

remove warning that no one would see

49be2aa

include individual hook doc pages

1345672

add test for the sitemap generator

9960c11

nad182 reviewed Feb 24, 2025

View reviewed changes

package.json Outdated Show resolved Hide resolved

nad182 reviewed Feb 24, 2025

View reviewed changes

sitemap-generator.js Outdated Show resolved Hide resolved

nad182 reviewed Feb 24, 2025

View reviewed changes

sitemap-generator.js Outdated Show resolved Hide resolved

nad182 reviewed Feb 24, 2025

View reviewed changes

sitemap-generator.js Outdated Show resolved Hide resolved

nad182 reviewed Feb 24, 2025

View reviewed changes

nad182 requested changes Feb 24, 2025

View reviewed changes

jobbiebot bot added changes requested and removed review requested labels Feb 24, 2025

ZakaryH and others added 3 commits February 24, 2025 11:45

split build commands up for improved readability

36891b6

Co-authored-by: Nazarii L <[email protected]>

fix formatting of package.json

721af9c

fix script directory navigation

eb53158

nad182 reviewed Feb 24, 2025

View reviewed changes

improve readability of regex operation and path collection

2499d78

ZakaryH requested a review from nad182 February 24, 2025 21:50

jobbiebot bot added review requested and removed changes requested labels Feb 24, 2025

nad182 approved these changes Feb 25, 2025

View reviewed changes

jobbiebot bot added approved and removed review requested labels Feb 25, 2025

ZakaryH merged commit f579b67 into master Feb 25, 2025
12 checks passed

ZakaryH deleted the sitemap-please-and-thank-you branch February 25, 2025 17:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build(docx): Add Sitemap #2382

build(docx): Add Sitemap #2382

ZakaryH commented Feb 18, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 18, 2025 •

edited

Loading

scotttjob Feb 19, 2025

scotttjob Feb 19, 2025

ZakaryH Feb 19, 2025

scotttjob Feb 19, 2025

ZakaryH Feb 19, 2025

cloudflare-workers-and-pages bot commented Feb 20, 2025

cloudflare-workers-and-pages bot commented Feb 20, 2025

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 20, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

cloudflare-workers-and-pages bot commented Feb 21, 2025

nad182 Feb 24, 2025

ZakaryH Feb 24, 2025

nad182 left a comment

nad182 Feb 24, 2025

nad182 left a comment

build(docx): Add Sitemap #2382

build(docx): Add Sitemap #2382

Conversation

ZakaryH commented Feb 18, 2025 • edited Loading

Motivations

Changes

Added

Changed

Deprecated

Removed

Fixed

Security

Testing

cloudflare-workers-and-pages bot commented Feb 18, 2025 • edited Loading

Deploying atlantis with Cloudflare Pages

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cloudflare-workers-and-pages bot commented Feb 20, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 20, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 20, 2025 • edited Loading

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 20, 2025 • edited Loading

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 20, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying atlantis with Cloudflare Pages

cloudflare-workers-and-pages bot commented Feb 21, 2025

Deploying atlantis with Cloudflare Pages

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nad182 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nad182 left a comment

Choose a reason for hiding this comment

ZakaryH commented Feb 18, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 18, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading

cloudflare-workers-and-pages bot commented Feb 20, 2025 •

edited

Loading