add documentation

README.md

@@ -23,6 +23,9 @@ This extension is still pretty raw but it works for us [tm]. Bug reports & contr
 - [VSCode Marketplace](https://marketplace.visualstudio.com/items?itemName=Zazuko.sparql-notebook)
 -  [Open VSX Registry](https://open-vsx.org/extension/zazuko/sparql-notebook)
 
+## Manual
+- [Manual](./doc/00_intro.md)
+
 ## Features
 
 - Open any `.sparqlbook` file as a Notebook.

doc/00_intro.md

@@ -0,0 +1,209 @@
+# Zazuko SPARQL Notebook 
+
+## Overview
+
+The **SPARQL Notebook for VSCode** extension enables users to run SPARQL queries directly within VSCode using a notebook-like interface with **Markdown** and **Code cells**. This versatile setup supports querying both remote HTTP/HTTPS SPARQL endpoints and local RDF files. The extension includes a **side panel for managing endpoint connections**, allowing you to seamlessly set up and switch between different SPARQL endpoints.
+
+### Key Features
+- **Query Execution on HTTP/HTTPS Endpoints**: Run SPARQL queries directly against remote SPARQL endpoints with HTTP/HTTPS protocols.
+- **Binding Local Query Files to Code Cells**: Bind local `.sparql` or `.rq` files directly to code cells, allowing the contents of these files to be embedded within the notebook cell itself. This approach enables you to develop and document queries in a dedicated file while keeping the query available within the notebook for easy execution. Markdown cells can be used alongside these code cells to add explanations or documentation.
+- **RDF File Querying**: Execute SPARQL queries on local RDF files (e.g., Turtle, RDF-XML) by providing a file path or pattern. This is especially useful when transforming data to RDF format and verifying the output with SPARQL before committing it to a triple store.
+- **Markdown and Code Cells**: Use Markdown cells for explanations and Code cells for SPARQL queries. This format is ideal for creating a rich, interactive documentation environment around your queries.
+
+While it’s a powerful tool for working with SPARQL and RDF data, the SPARQL Notebook is adaptable to a range of use cases beyond data science. Whether you’re building transformation workflows or simply documenting and testing queries. If there are any features missing for your specific use case, feel free to request them.
+
+
+## HTTP Endpoint
+To configure connections to SPARQL endpoints, click on the **SPARQL Notebook** icon on the left sidebar to open the **Connections** panel. This panel allows you to manage your SPARQL endpoint connections with ease.
+
+To add a new connection, click the **+** icon in the top right corner. Both HTTP and HTTPS endpoints are supported.
+
+![Connections](./img/endpoint-connection.png)
+
+To delete an endpoint connection, simply use the context menu (right-click on the connection) to remove it when it’s no longer needed.
+
+> **Note:**
+> 
+> Using these connections is not "portable". It another user opens the notebook, they will not have the same connections.
+> The connections are stored in the user settings of VSCode.
+
+> **Note:**
+> 
+> Currently not supported:
+> - SSL client certificates
+> - Proxy
+
+## Endpoints from Cells
+You can also define endpoints directly in a cell. This is useful if you want to share the notebook with others and don't want to rely on the connections. This is very useful for sharing the notebook with others. But for it is limited to HTTP/HTTPS SPARQL endpoints without authentication or files (we come to that later). 
+
+This is an example how you provide a SPARQL endpoint in a cell:
+
+```sparql
+#
+# Test [endpoint=https://query.wikidata.org/bigdata/namespace/wdq/sparql]
+# 
+PREFIX wd: <http://www.wikidata.org/entity/>
+PREFIX wdt: <http://www.wikidata.org/prop/direct/>
+PREFIX wikibase: <http://wikiba.se/ontology#>
+PREFIX bd: <http://www.bigdata.com/rdf#>
+
+SELECT DISTINCT ?euMemberCountry ?euMemberCountryLabel ?headOfState ?headOfStateLabel
+ 
+WHERE {
+  ?euMemberCountry wdt:P463 wd:Q458;
+      wdt:P35 ?headOfState .
+  SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en". }
+}
+```
+The `[endpoint=https://query.wikidata.org/bigdata/namespace/wdq/sparql]` configures the endpoint for this cell.
+
+## Local RDF Files
+You can also query local files. This is useful if you want to query RDF files on your local machine. This is an example how you provide a file in a cell:
+
+```sparql
+#
+# Test [endpoint=./data/00_intro.ttl]
+
+SELECT ?s ?p ?o
+WHERE {
+  ?s ?p ?o
+} LIMIT 10
+```
+
+You can provide a file path or a glob pattern.
+
+Here are a few patterns:
+```sparql
+# [endpoint=./rdf/curia.nt]
+```
+or
+```sparql
+# [endpoint=./rdf/*]
+```
+or
+```sparql
+# [endpoint=rdf/*]
+```
+or
+```sparql
+# [endpoint=./rdf/{a,b}.ttl]
+```
+or
+```sparql
+# [endpoint=rdf/a.ttl]
+# [endpoint=./rdf/b.ttl]
+```
+
+You can go here to see the [Glob Patterns Reference](https://code.visualstudio.com/docs/editor/glob-patterns).
+
+
+There is another option to provide a file. You can navigate in the vscode file explorer to the file you want to query and then right-click on the file and select `SPARQL Notebook: Use File as Store`.
+
+![Connections](./img/use-file.png)
+
+> **Note:**
+> 
+> File patterns in the comment are more flexible than the right-click option. But it is not doing the same thing. 
+> If you use the flexible comment option then every time you execute the query it will create a new store and load the
+> data from the file. If you use the right-click option then it will load the data only once and you can execute the query. But if you change the file then you have to reload the data in the same way.
+> The right-click option is more efficient if you have a large file and you want to execute the query multiple times.
+
+
+**FAQ:**
+
+**Q:** Can I use a file with a different extension than `.ttl`?
+
+YES
+
+| File Extension | MIME Type          |
+|----------------|--------------------|
+| .ttl           | text/turtle        |
+| .nt            | application/n-triples|
+| .rdf           | application/rdf+xml|
+| .trig          | application/trig   |
+| .nq            | application/n-quads|
+
+**Q:** Do you ship with a SPARQL endpoint?
+
+YES. The notebook is using the amazing [Oxigraph](https://github.com/oxigraph/oxigraph) engine.
+
+**Q:** It there a limit for local rdf files?
+
+YES. The limit is currently approx 1G per file. But you can load multiple files. Just try it out.
+
+
+## Local Query Files
+You can also bind local `.sparql` or `.rq` files to code cells. This is useful if you want to develop and document queries in a dedicated file while keeping the query available within the notebook for easy execution. This is an example how you provide a file in a cell:
+
+![Include query from file](./img/external-query.png)
+
+> **Note:**
+> 
+> If you change the cell content and save the notebook then the content of the file will be updated.
+
+## Export to MARKDOWN
+
+A notebook can be exported to a markdown file. Find the file in the vscode file explorer and right-click on the file and select `SPARQL Notebook: Export to Markdown`.
+
+
+## Result Cells
+Query results are displayed within dedicated result cells, with display formats tailored to different query types.
+
+- **SELECT and ASK Queries**: Results from these queries are shown in a table format, making it easy to view and analyze tabular data.
+- **CONSTRUCT and DESCRIBE Queries**: Results from these queries are rendered as a graph, providing a visual representation of RDF triples.
+
+The notebook offers flexible display options through **renderers** specific to different MIME types. Some extensions also provide additional renderers for specific MIME types, enabling a more customized visualization experience for various data formats.
+This extension is shipped with a few renderers. We include the extension ["Data Table Renderers from Random Fractals Inc."](https://marketplace.visualstudio.com/items?itemName=RandomFractalsInc.vscode-data-table) which provides better table rendering for large tables. 
+
+You can change the result renderer like this: 
+![Change Result Renderer](./img/change-renderer.png)
+
+Then the top middle menu show the available renderers. Just choose the one you like.
+
+![The new renderer](./img/another-renderer.png)
+
+> **Note:**
+> 
+> How to set the current renderer as default?
+> To set the current renderer as default, you can use the VSCode command palette:
+> 
+> 1. Open the Command Palette (Ctrl+Shift+P or Cmd+Shift+P on Mac),
+> 2. Type and select `Notebook: Save Mimetype Display Order`.
+
+> **Note:**
+>
+> Our renderers are not designed for a large amount of data. Use the `Data Table Renderers from Random Fractals Inc.` extension for large data.
+
+## SPARQL Anything
+
+This has nothing todo with this notebook but i will show you how to use SPARQL Anything. We don't ship it with this extension but you can get it from https://sparql-anything.cc/.
+You can run it in server mode and connect the notebook to it. 
+
+Run sparql-anything in server mode:
+```bash
+java -jar sparql-anything-server-<version>.jar 
+```
+
+Then you can use it in the notebook like this:
+
+```sparql
+# [endpoint=http://localhost:3000/sparql.anything]
+
+PREFIX fx: <http://sparql.xyz/facade-x/ns/>
+PREFIX schema: <http://schema.org/>
+PREFIX xyz: <http://sparql.xyz/facade-x/data/>
+PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
+PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
+PREFIX myns: <http://example.org/myns/>
+
+CONSTRUCT {
+   ?s ?p ?o .
+} WHERE {
+     # file relative to the SPARQL anything server
+     SERVICE<x-sparql-anything:>{
+        fx:properties fx:location "../input/2024-03-08 AKN4ZH ACT A0.xml";  
+                      fx:media-type "application/xml".  
+            ?s ?p ?o .
+    }
+}
+```

doc/development/README.md

@@ -0,0 +1,106 @@
+# Development Notes
+
+This document contains some notes about the development of the extension.
+
+## Publishing the extension
+
+The extension is published to vscode marketplace and open-vsx. Both versions are the same except the dependencies to other extensions. These dependencies are not really dependencies but recommendations to have a better experience with the extension. These extensions are not available in both market places. This is why we list them here. And if we deploy a new version for vscode marketplace or open-vsx we need to update the dependencies in the `package.json` file.
+
+### vscode
+
+These extensions are currently used in our vscode version:
+
+```json
+ "extensionDependencies": [
+    "stardog-union.stardog-rdf-grammars",
+    "stardog-union.vscode-langserver-sparql",
+    "RandomFractalsInc.vscode-data-table"
+  ]
+```
+What are they for?
+
+**vscode-langserver-sparql**
+
+"A Visual Studio Code extension providing language intelligence (autocomplete, diagnostics, hover tooltips, etc.) for the SPARQL query language, including both W3C standard SPARQL and Stardog extensions (e.g., PATHS queries), via the Language Server Protocol."
+
+This means we have all these features in Notebook Cells with SPARQL language.
+
+**Stardog RDF Grammars**
+
+"Visual Studio Code syntax highlighting for all your favorite RDF languages, and a few Stardog-specific ones, too!"
+
+This means we have syntax highlighting for all RDF languages.This is not really necessary but it is nice to have.
+
+**RandomFractalsInc.vscode-data-table**
+
+"These Data Table Renderers were created to enhance raw data views in Jupyter and custom VSCode Notebooks 📚."
+
+This means extension provides more advanced rendering for data tables in Notebook Cells and can handle larger amount of data. If you have big results then this is the only way to see an output. Because our renderer is not optimized for big results.
+
+
+### open-vsx
+
+The stardog extensions are not available in vsx. 
+
+```json
+   "extensionDependencies": [
+    "vemonet.stardog-rdf-grammars"
+  ]
+```
+
+What are they for?
+
+**vemonet.stardog-rdf-grammars** 
+"Visual Studio Code syntax highlighting for all your favorite RDF languages, and a few Stardog-specific ones, too!"
+
+They are the same as the vscode version. But released by another entity than stardog-union.
+
+
+### Publishing to both
+
+#### vscode marketplace
+
+Just run the following command:
+
+You can create a bundle and install the bundle locally to test it before publishing it to the marketplace.
+
+```bash
+ npm run package 
+```
+
+This will create a file like `sparql-notebook-0.0.39.vsix`. You can right click on it in vscode and install it. 
+
+To publish it to the marketplace you need to run the following command:
+```bash
+npm run deploy
+```
+This runs the `npx vsce publish` command. You need the credentials to do this.
+
+#### open-vsx
+
+First of all we have to change the dependencies in the `package.json` file. Then we can run the following command:
+
+```bash
+cd vsx-utils
+node replace-deps.js
+```
+
+This will replace the `extensionDependencies` in the `package.json` file. The old `package.json` file is saved as `package.json.bak`. 
+
+Then we can package it with the same command as for vscode:
+
+```bash
+ npm run package 
+```
+
+This will create a file like `sparql-notebook-0.0.39.vsix`. You can right click on it in vscodium and install it.
+
+
+After this you can upload the extension here [open-vsx](https://open-vsx.org/user-settings/extensions). You need credentials for this.
+
+After this restore the old `package.json` file with the following command:
+
+```bash
+cd vsx-utils
+node restore-deps.js
+```