Merge pull request #61 from valkey-io/main

[Move to Prod] Caps fix, docs urls, modules 101 blog post

.gitignore

@@ -6,4 +6,5 @@ vendor
 .DS_Store
 /_data/commands/
 /_includes/commands/
-/_includes/docs/
+/_includes/docs/
+tmp/*

Gemfile.lock

@@ -79,6 +79,8 @@ GEM
     sass-embedded (1.69.5)
       google-protobuf (~> 3.23)
       rake (>= 13.0.0)
+    sass-embedded (1.69.5-x86_64-darwin)
+      google-protobuf (~> 3.23)
     terminal-table (3.0.2)
       unicode-display_width (>= 1.1.1, < 3)
     typhoeus (1.4.1)
@@ -90,6 +92,7 @@ PLATFORMS
   arm64-darwin-22
   x86_64-darwin-19
   x86_64-darwin-21
+  x86_64-darwin-23
   x86_64-linux
 
 DEPENDENCIES

README.md

@@ -19,8 +19,9 @@ This site is built using Jekyll 4.3.2 with Ruby 3.3.0. Other versions may work,
 2. Install [Ruby](https://www.ruby-lang.org/en/) and [RVM](https://rvm.io/) (or some other Ruby version switcher, e.g. [chruby](https://github.com/postmodern/chruby))
 3. Install [Jekyll](https://jekyllrb.com/)
 4. Install dependencies: `bundle install`
-5. Build: `bundle exec jekyll serve` for the local server, `bundle exec jekyll build` for one off builds. Either way, the HTML of the site is generated to `/_site`
-6. Point your browser at `http://127.0.0.1:4000/`
+5. Run `git submodule update --init --recursive`
+6. Build: `bundle exec jekyll serve` for the local server, `bundle exec jekyll build` for one off builds. Either way, the HTML of the site is generated to `/_site`
+7. Point your browser at `http://127.0.0.1:4000/`
 
 ## Build with Docker
 

_authors/dmitrypol.markdown

@@ -0,0 +1,12 @@
+---
+name: Dmitry Polyakovsky
+short_name: dmitrypol
+photo: '/assets/media/authors/dmitrypol.jpeg'
+github: dmitrypol
+---
+
+Dmitry is a Consulting Member of Technical Staff at Oracle Cloud.  
+
+Dmitry has used Redis for over 10 years for a variety of purposes.  He has written numerous blog articles, been interviewed by podcasts, presented at various conferences and built open-source libraries on top of Redis.  Currently Dmitry is working at Oracle Cloud as Tech Lead for OCI Cache service.  He is very excited to be part of the Valkey project.  
+
+In his spare time Dmitry enjoys hiking in the Seattle area and spending time with his family.  

_includes/topics

@@ -0,0 +1 @@
+../_submodules/valkey-doc/topics

_posts/2024-04-26-modules-101.markdown

@@ -0,0 +1,175 @@
+---
+layout: post
+title:  "Valkey Modules 101"
+authors: 
+  - dmitrypol
+date: 2024-05-01 01:01:01 -0700
+categories: modules
+---
+
+## What are Valkey modules?  
+
+The idea of modules is to allow adding extra features (such as new commands and data types) to Valkey without making changes to the core code.
+Modules are a special type of code distribution called a shared library, which can be loaded by other programs at runtime and executed.
+Modules can be written in C or other languages that have C bindings.
+In this article we will go through the process of building simple modules in C and Rust (using the Valkey Module Rust SDK).
+This article expects the audience to be at least somewhat familiar with git, C, Rust and Valkey.
+
+## Hello World module in C
+
+If we clone the Valkey repo by running `git clone [email protected]:valkey-io/valkey.git` we will find numerous examples in `src/modules`.
+Let's create a new file `module1.c` in the same folder.
+
+```c
+#include "../valkeymodule.h"
+
+int hello(ValkeyModuleCtx *ctx, ValkeyModuleString **argv, int argc) {
+    VALKEYMODULE_NOT_USED(argv);
+    VALKEYMODULE_NOT_USED(argc);
+    return ValkeyModule_ReplyWithSimpleString(ctx, "world1");
+}
+
+int ValkeyModule_OnLoad(ValkeyModuleCtx *ctx, ValkeyModuleString **argv, int argc) {
+    VALKEYMODULE_NOT_USED(argv);
+    VALKEYMODULE_NOT_USED(argc);
+    if (ValkeyModule_Init(ctx,"module1",1,VALKEYMODULE_APIVER_1) 
+        == VALKEYMODULE_ERR) return VALKEYMODULE_ERR;
+    if (ValkeyModule_CreateCommand(ctx,"module1.hello", hello,"",0,0,0) 
+        == VALKEYMODULE_ERR) return VALKEYMODULE_ERR;
+    return VALKEYMODULE_OK;
+}
+```
+
+Here we are calling `ValkeyModule_OnLoad` C function (required by Valkey) to initialize `module1` using `ValkeyModule_Init`.
+Then we use `ValkeyModule_CreateCommand` to create a Valkey command `hello` which uses C function `hello` and returns `world1` string.
+In future blog posts we will expore these areas at greater depth.
+
+Now we need to update `src/modules/Makefile`
+
+```makefile
+all: ... module1.so
+
+module1.xo: ../valkeymodule.h
+
+module1.so: module1.xo
+	$(LD) -o $@ $^ $(SHOBJ_LDFLAGS) $(LIBS) -lc
+```
+
+Run `make module1.so` inside `src/modules` folder.
+This will compile our module in the `src/modules` folder.
+
+## Hello World module in Rust
+
+We will create a new Rust package by running `cargo new --lib module2` in bash.
+Inside the `module2` folder we will have `Cargo.toml` and `src/lib.rs` files.
+To install the valkey-module SDK run `cargo add valkey-module` inside `module2` folder.
+Alternativley we can add `valkey-module = "0.1.0` in `Cargo.toml` under `[dependencies]`.
+Run `cargo build` and it will create or update the `Cargo.lock` file.
+
+Modify `Cargo.toml` to specify the crate-type to be "cdylib", which will tell cargo to build the target as a shared library.
+Read [Rust docs](https://doc.rust-lang.org/reference/linkage.html) to understand more about `crate-type`.
+
+```
+[lib]
+crate-type = ["cdylib"]
+```
+
+Now in `src/lib.rs` replace the existing code with the following:
+
+```rust
+#[macro_use]
+extern crate valkey_module;
+
+use valkey_module::{Context, ValkeyResult, ValkeyString, ValkeyValue};
+
+fn hello(_ctx: &Context, _args: Vec<ValkeyString>) -> ValkeyResult {
+    Ok(ValkeyValue::SimpleStringStatic("world2"))
+}
+
+valkey_module! {
+    name: "module2",
+    version: 1,
+    allocator: (valkey_module::alloc::ValkeyAlloc, valkey_module::alloc::ValkeyAlloc),
+    data_types: [],
+    commands: [
+        ["module2.hello", hello, "", 0, 0, 0],
+    ]
+}
+```
+
+Rust syntax is a bit different than C but we are creating `module2` with command `hello` that returns `world2` string.
+We are using the external crate `valkey_module` with [Rust macros](https://doc.rust-lang.org/book/ch19-06-macros.html) and passing it variables like `name` and `version`.
+Some variables like `data_types` and `commands` are arrays and we can pass zero, one or many values.
+Since we are not using ctx or args we prefix them with `_` (Rust convention) instead of `VALKEYMODULE_NOT_USED` as we did in C.
+
+Run `cargo build` in the root folder.
+We will now see `target/debug/libmodule2.dylib` (on macOS).
+The build will produce *.so files on Linux and *.dll files on Windows.
+
+
+## Run Valkey server with both modules
+
+Go back into the Valkey repo folder and run `make` to compile the Valkey code.
+Then add these lines to the bottom of the `valkey.conf` file.
+
+```
+loadmodule UPDATE_PATH_TO_VALKEY/src/modules/module1.so
+loadmodule UPDATE_PATH_TO_MODULE2/target/debug/libmodule2.dylib
+```
+
+and run `src/valkey-server valkey.conf`.
+You will see these messages in the log output.
+
+```
+Module 'module1' loaded from UPDATE_PATH_TO_VALKEY/src/modules/module1.so
+...
+Module 'module2' loaded from UPDATE_PATH_TO_MODULE2/target/debug/libmodule2.dylib
+```
+
+Then use `src/valkey-cli` to connect.
+
+```bash
+src/valkey-cli -3
+127.0.0.1:6379> module list
+1) 1# "name" => "module2"
+   2# "ver" => (integer) 1
+   3# "path" => "UPDATE_PATH_TO_MODULE2/target/debug/libmodule2.dylib"
+   4# "args" => (empty array)
+2) 1# "name" => "module1"
+   2# "ver" => (integer) 1
+   3# "path" => "UPDATE_PATH_TO_VALKEY/src/modules/module1.so"
+   4# "args" => (empty array)
+127.0.0.1:6379> module1.hello
+world1
+127.0.0.1:6379> module2.hello
+world2
+```
+
+We can now run both modules side by side and if we modify either C or RS file, recompile the code and restart `valkey-server` we will get the new functionality.
+
+As an alternative to specifying modules in `valkey.conf` file, we can use `MODULE LOAD` and `UNLOAD` from `valkey-cli` to update the server.
+First specify `enable-module-command yes` in `valkey.conf` and restart `valkey-server`.
+This enables us to update our module code, recompile it and reload it at runtime.
+
+```
+127.0.0.1:6379> module load UPDATE_PATH_TO_VALKEY/src/modules/module1.so
+OK
+127.0.0.1:6379> module list
+1) 1# "name" => "module1"
+   2# "ver" => (integer) 1
+   3# "path" => "UPDATE_PATH_TO_VALKEY/src/modules/module1.so"
+   4# "args" => (empty array)
+127.0.0.1:6379> module unload module1
+OK
+127.0.0.1:6379> module list
+(empty array)
+127.0.0.1:6379> 
+```
+
+Please stay tuned for more articles in the future as we explore the possibilities of Valkey modules and where using C or Rust makes sense.
+
+## Usefull links
+
+* [Valkey repo](https://github.com/valkey-io/valkey)
+* [Valkey Rust SDK](https://github.com/valkey-io/valkeymodule-rs)
+* [Rust in VS Code](https://code.visualstudio.com/docs/languages/rust)

docs/index.md

@@ -6,8 +6,8 @@ primary_title: Documentation
 
 * [Command Reference](../commands/)
 * Management
-  * [Persistence](/docs/management/persistence.html)
+  * [Persistence](/docs/topics/persistence.html)
   * Security
-    * [ACL](/docs/management/security/acl.html)
+    * [ACL](/docs/topics/acl.html)
 * Valkey Manual
-  * [Keyspace Notifications](/docs/manual/keyspace-notifications.html)
+  * [Keyspace Notifications](/docs/topics/keyspace.html)

docs/management/security/acl.md → docs/topics/acl.md

@@ -1,5 +1,5 @@
 ---
 layout: doc-import
 primary_title: ACL
-source: "/docs/management/security/acl.md"
+source: "/topics/acl.md"
 ---

docs/manual/keyspace-notifications.md → docs/topics/keyspace.md

@@ -1,5 +1,5 @@
 ---
 layout: doc-import
 primary_title: Keyspace Notifications
-source: /docs/manual/keyspace-notifications.md
+source: "/topics/keyspace.md"
 ---

docs/management/persistence.md → docs/topics/persistence.md

@@ -1,5 +1,5 @@
 ---
 layout: doc-import
 primary_title: Persistence
-source: "/docs/management/persistence.md"
+source: "/topics/persistence.md"
 ---

index.markdown

@@ -37,7 +37,7 @@ Valkey can run as either a **standalone** daemon or in a **cluster**, with optio
 
 Valkey natively supports a rich collection of datatypes, including **strings**, **numbers**, **hashes**, **lists**, **sets**, **sorted sets**, **bitmaps**, **hyperloglogs** and more.
 You can operate on data structures in-place with a expressive collection of commands.
-Valkey also supports native extensibility with built-in scripting support for **LUA** and supports **module** plugins to create new commands, data types, and more.
+Valkey also supports native extensibility with built-in scripting support for **Lua** and supports **module** plugins to create new commands, data types, and more.
 
 
 ---