Best practice for indexing and filtering different contents

[codejanovic]

Hi everyone,

after trying out Pagefind with AstroJS and reading throught the documentation (especially Node-API, Filtering and Multisite), I am struggling on how to index & filter different contents independently and bundle them all together on one site.

Heres what I want to index:

• indexing basic site content pages like /company, /aboutme, /gdpr and so on from generated *.html files
• indexing products from Node-API
• indexing blog posts from Node-API
• indexing articles from Node-API

Now I would like to implement different searches on my site, searching only for the relevant indexes like:

• a global search (the site header) should offer a search across all indexes
• /products/ site should offer a search that only uses the product index
• /blog/ site should offer a search that only uses the blogindex
• /articles/ site should offer a search that only uses the articles index

What I've tried so far:

• indexing content via pagefind cli and data-pagefind-body attributes, generating dist/pagefind (optional)
• indexing products via Node-API, writing the index to a custom dist/pagefind-product folder (required)
• indexing articlesvia Node-API, writing the index to a custom dist/pagefind-article folder (required)
• indexing blog via Node-API, writing the index to a custom dist/pagefind-blog folder (required)

Then for example on the /product page, I am initializing the PagefindUI by defining mergeFilters:

 new PagefindUI({
          element: "#search",
          mergeFilter: {
              resource: "Content"
          },
          mergeIndex: [{
              bundlePath: "/pagefind-product",
              mergeFilter: {
                  resource: "Product"
              }
          }],
          showSubResults: true,
          showImages: true,
          processResult: function (result: any) {
              console.log("pagefind result", result);
              return result;
          }
      });

But I get the following error:

pagefind.js:1 Failed to load Pagefind metadata:
SyntaxError: Unexpected end of JSON input
loadEntry @ pagefind.js:1
await in loadEntry
init @ pagefind.js:1
mergeIndex @ pagefind.js:9
await in mergeIndex
mergeIndex @ pagefind.js:9
an @ pagefind-ui.js:2
pagefind.js:2 Uncaught (in promise) Error: Failed to load Pagefind metadata
    at PagefindInstance.loadEntry (pagefind.js:2:26)
    at async PagefindInstance.init (pagefind.js:1:19619)
    at async Pagefind.mergeIndex (pagefind.js:9:1860)
    at async Module.mergeIndex (pagefind.js:9:3826)
    at async HTMLInputElement.an (pagefind-ui.js:2:210)

What also confused me is that using the Node-API only will generate empty files for pagefind-ui.css and pagefind-ui.js. It seems only the pagefind CLI will generate them properly.

Would love to hear your feedback or even sample code/projects of people that may already implemented such a usecase.

Cheers,
Aleks

[bglw]

Hi @codejanovic 👋

For this use-case I would usually just generate a single Pagefind instance with filtering to demarcate each specific search. Searching as one combined index has better (network) performance than the merged-index style, so when it's possible to do it's always preferred. The NodeJS API can be used for it all, both ingesting current content and merging it with the standard directory-crawl behavior.

What also confused me is that using the Node-API only will generate empty files for pagefind-ui.css and pagefind-ui.js. It seems only the pagefind CLI will generate them properly.

Oop — if true that's definitely a bug 🤔 I'll add an integration test for the node api / ui crossover and see if it smokes anything out.