Chat synchronization via WebHooks

CHANGELOG.md

@@ -1,3 +1,9 @@
+# indev
+- Introduce webhook chat sync, allowing for user imitation
+
+  **IMPORTANT INFO**: If you don't want to manually set the webhook URL, the
+  bot needs to have the MANAGE_WEBHOOKS permission
+
 # 1.4.0
 - Port to 1.20.4 and NeoForge
 - Removed Forge as a supported Platform for Minecraft > 1.20.1

README.md

@@ -12,7 +12,7 @@ while allowing true complexity for power users.
 These features are the core of what this bot will do. See [the Status section](#status)
 to see how much is already implemented.
 - Discord and Minecraft account linking, optionally requiring this to be whitelisted
-- Discord and Minecraft chat synchronization
+- Discord and Minecraft chat synchronization, optionally via webhook for improved readability
 - FAQ commands using Markdown files without needing to restart the server
 
 ## Dependencies
@@ -35,7 +35,7 @@ The following things will be configurable:
   2. (If possible) Minecraft user imitation (The Minecraft usernames and heads will be used for messages)
   3. (Only available if requiring linking for whitelist) Discord user imitation (The messages will be written under the linked Discord name)
 
-  Default: No user imitation
+  Default: Minecraft user imitation (if webhook URL is set, otherwise no imitation)
 - Managing FAQs through a command interface (default: `off`)
 - Database connection. Uses an SQLite database by default
 
@@ -45,20 +45,20 @@ The following things will be configurable:
   - [x] Reading Markdown files
   - [x] Updating suggestions without restart
   - [ ] Management/Creation via commands
-- [ ] Chat synchronization
+- [x] Chat synchronization
   - [x] Minecraft to Discord
   - [x] Discord to Minecraft
-  - [ ] User imitation on Discord
+  - [x] User imitation on Discord. Either uses Minecraft avatars or Discord avatars
 
 ## Running
 There is no public instance of this bot/mod available. To use it, create a new Application
 on the [Discord Developer Portal](https://discord.com/developers/applications) and configure it
 to have the three privileged gateway intents: `PRESENCE`, `SERVER MEMBERS` and `MESSAGE CONTENT`.
 
 Copy the bot token and store it somewhere safe (like a Password Manager) and never show it to
-anybody else. To make sure the token gets read by the bot, it has to be in an [Environment Variable](https://en.wikipedia.org/wiki/Environment_variable)
-`DISCORD_TOKEN` where the running Minecraft server can access it. You could for example modify a `start.sh` script
-on a Unix-like system to `export` it or start the shell script with it set directly:
+anybody else. To make sure the token gets read by the bot, it has to be in either an [Environment Variable](https://en.wikipedia.org/wiki/Environment_variable)
+`DISCORD_TOKEN` where the running Minecraft server can access it or in the config file under the key `token`.
+You could for example modify a `start.sh` script on a Unix-like system to `export` it or start the shell script with it set directly:
 
 `startmc.sh`:
 ```shell
@@ -75,4 +75,9 @@ DISCORD_TOKEN=<place token here> startmc.sh # Start the script with the env vari
 
 After starting the server, you can go into the OAuth2 URL builder on the Discord
 Developer Portal and generate a URL with the `bot` and `applications.command` permissions.
-Use the generated URL to have the bot join your server.
+Use the generated URL to have the bot join your server.
+
+To be able to use user imitation, which makes the chat synchronization more readable, the bot will try and create a webhook
+in the configured chat synchronization channel. This means it will need to have the permission to manage webhooks. If you
+don't want that you can manually create a webhook in the channel where you want the messages synchronized and set it in the
+`webhook.url` option in the config for the mod.

build.gradle.kts

@@ -1,3 +1,4 @@
+import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar
 import org.jetbrains.kotlin.gradle.utils.extendsFrom
 import java.nio.charset.StandardCharsets
 import java.text.SimpleDateFormat
@@ -71,6 +72,7 @@ subprojects {
 
     // Bot dependencies
     val jdaVersion: String by project
+    val dcWebhooksVersion: String by project
     val exposedVersion: String by project
     val sqliteJDBCVersion: String by project
     val commonmarkVersion: String by project
@@ -102,13 +104,19 @@ subprojects {
         arrayOf(
             // Library used to communicate with Discord, see https://jda.wiki
             "net.dv8tion:JDA:$jdaVersion",
+            // Library used for sending messages via Discord Webhooks
+            "club.minnced:discord-webhooks:$dcWebhooksVersion",
 
             // Library to interact with the SQLite database,
             // see: https://github.com/JetBrains/Exposed
             "org.jetbrains.exposed:exposed-core:$exposedVersion",
             "org.jetbrains.exposed:exposed-dao:$exposedVersion",
             "org.jetbrains.exposed:exposed-jdbc:$exposedVersion",
         ).forEach {
+            implementation(it) {
+                exclude(module = "opus-java")
+                exclude(group = "org.slf4j")
+            }
             runtimeLib(it) {
                 exclude(module = "opus-java")
                 exclude(group = "org.slf4j")
@@ -122,6 +130,7 @@ subprojects {
                 // on JDA and Exposed, but is already provided by the
                 // respective Kotlin implementation of the mod loaders
                 exclude(group = "org.jetbrains.kotlin")
+                exclude(group = "org.jetbrains.kotlinx")
                 // Minecraft already ships with a logging system
                 exclude(group = "org.slf4j")
             }
@@ -194,9 +203,48 @@ subprojects {
     }
 
     if (!isCommon) {
+        apply(plugin = "io.github.goooler.shadow")
+
         dependencies {
             implementation(project(":common"))
         }
+
+        tasks.named<ShadowJar>("shadowJar") {
+            // The shadowBotDep configuration was explicitly made to be shaded in, this is where that happens
+            configurations.clear()
+            configurations = listOf(shadowBotDep)
+
+            // This transforms the service files to make relocated Exposed work (see: https://github.com/JetBrains/Exposed/issues/1353)
+            mergeServiceFiles()
+
+            // Forge restricts loading certain classes for security reasons.
+            // Luckily, shadow can relocate them to a different package.
+            relocate("org.apache.commons.collections4", "dev.erdragh.shadowed.org.apache.commons.collections4")
+
+            // Relocating Exposed somewhere different so other mods not doing that don't run into issues (e.g. Ledger)
+            relocate("org.jetbrains.exposed", "dev.erdragh.shadowed.org.jetbrains.exposed")
+
+            // Relocating jackson to prevent incompatibilities with other mods also bundling it (e.g. GroovyModLoader on Forge)
+            relocate("com.fasterxml.jackson", "dev.erdragh.shadowed.com.fasterxml.jackson")
+
+            // relocate discord interaction stuff to maybe allow other discord integrations mods to work
+            relocate("club.minnced.discord", "dev.erdragh.shadowed.club.minnced.discord")
+            relocate("net.dv8tion.jda", "dev.erdragh.shadowed.net.dv8tion.jda")
+
+            // relocate dependencies of discord stuff
+            relocate("okhttp3", "dev.erdragh.shadowed.okhttp3")
+            relocate("okio", "dev.erdragh.shadowed.okio")
+            relocate("gnu.trove", "dev.erdragh.shadowed.gnu.trove")
+            relocate("com.iwebpp.crypto", "dev.erdragh.shadowed.com.iwebpp.crypto")
+            relocate("com.neovisionaries.ws", "dev.erdragh.shadowed.com.neovisionaries.ws")
+            relocate("org.json", "dev.erdragh.shadowed.org.json")
+            relocate("net.bytebuddy", "dev.erdragh.net.bytebuddy")
+
+            exclude("**/org/slf4j/**")
+
+            exclude("**/org/jetbrains/annotations/*")
+            exclude("**/org/intellij/**")
+        }
     }
 
     // Disables Gradle's custom module metadata from being published to maven. The

common/build.gradle.kts

@@ -1,3 +1,5 @@
+import org.spongepowered.gradle.vanilla.repository.MinecraftPlatform
+
 plugins {
     idea
     java
@@ -10,6 +12,7 @@ val modId: String by project
 
 minecraft {
     version(minecraftVersion)
+    platform(MinecraftPlatform.SERVER)
     if (file("src/main/resources/${modId}.accesswidener").exists())
         accessWideners(file("src/main/resources/${modId}.accesswidener"))
 }

common/src/main/java/dev/erdragh/astralbot/mixins/PlayerListMixin.java

@@ -20,6 +20,6 @@ public class PlayerListMixin {
 
     @Inject(method = "canPlayerLogin", at = @At(value = "INVOKE", target = "Lnet/minecraft/network/chat/Component;translatable(Ljava/lang/String;)Lnet/minecraft/network/chat/MutableComponent;"), cancellable = true)
     private void astralbot$returnWhiteListMessage(SocketAddress socketAddress, GameProfile gameProfile, CallbackInfoReturnable<Component> cir) {
-        cir.setReturnValue(WhitelistHandler.INSTANCE.writeWhitelistMessage(gameProfile));
+        WhitelistHandler.INSTANCE.writeWhitelistMessage(gameProfile).ifPresent(cir::setReturnValue);
     }
 }

common/src/main/kotlin/dev/erdragh/astralbot/Bot.kt

@@ -28,7 +28,7 @@ var minecraftHandler: MinecraftHandler? = null
 
 var textChannel: TextChannel? = null
 var guild: Guild? = null
-private var jda: JDA? = null
+var jda: JDA? = null
 var applicationId by Delegates.notNull<Long>()
 
 var baseDirectory: File? = null
@@ -97,6 +97,9 @@ private fun setupFromJDA(api: JDA) {
     textChannel = ch
     guild = g
 
+    // Text channel was fetched, now the minecraftHandler can fetch its webhook stuff
+    minecraftHandler?.updateWebhookClient()
+
     ch.sendMessage("Server Started!").queue()
 }
 

common/src/main/kotlin/dev/erdragh/astralbot/commands/minecraft/MinecraftCommands.kt

@@ -18,6 +18,7 @@ object LinkCommand : Command<CommandSourceStack> {
         val caller = context?.source?.playerOrException!!
         if (WhitelistHandler.checkWhitelist(caller.uuid) != null) {
             context.source.sendFailure(Component.literal(AstralBotTextConfig.LINK_COMMAND_ALREADY_LINKED.get()))
+            return 1
         }
         val whitelistCode = WhitelistHandler.getOrGenerateWhitelistCode(caller.uuid)
         context.source.sendSuccess({

common/src/main/kotlin/dev/erdragh/astralbot/config/AstralBotConfig.kt

@@ -3,7 +3,10 @@ package dev.erdragh.astralbot.config
 import dev.erdragh.astralbot.LOGGER
 import dev.erdragh.astralbot.commands.discord.allCommands
 import net.neoforged.neoforge.common.ModConfigSpec
+import java.net.URI
 import java.net.URL
+import java.net.URLDecoder
+import java.nio.charset.StandardCharsets
 
 /**
  * Config for the AstralBot mod. This uses Forge's config system
@@ -20,6 +23,30 @@ object AstralBotConfig {
      */
     val DISCORD_TOKEN: ModConfigSpec.ConfigValue<String>
 
+    /**
+     * Used for sending more appealing messages
+     */
+    val WEBHOOK_URL: ModConfigSpec.ConfigValue<String>
+    /**
+     * Whether the configured [WEBHOOK_URL] should actually be used,
+     * useful in case somebody wants to temporarily disable using
+     * the webhook without removing the URL
+     */
+    val WEBHOOK_ENABLED: ModConfigSpec.BooleanValue
+
+    /**
+     * URL template for getting avatars from Minecraft users
+     */
+    val WEBHOOK_MC_AVATAR_URL: ModConfigSpec.ConfigValue<String>
+
+    /**
+     * Whether the chat messages sent via the webhook should
+     * imitate the sender's Discord account or their Minecraft
+     * account. If this is on, the linked Discord account will
+     * be used.
+     */
+    val WEBHOOK_USE_LINKED: ModConfigSpec.BooleanValue
+
     /**
      * Whether the default whitelisting process is respected or ignored.
      * Setting this to `true` will *force* every user who wants to join
@@ -101,6 +128,15 @@ object AstralBotConfig {
         DISCORD_TOKEN = builder.comment("Discord token for the bot. Can also be supplied via DISCORD_TOKEN environment variable")
             .define("token", "")
 
+        WEBHOOK_URL = builder.comment("URL to the webhook where the messages will be sent from")
+            .define(listOf("webhook", "url"), "")
+        WEBHOOK_ENABLED = builder.comment("Whether to use the configured webhook for sending messages")
+            .define(listOf("webhook", "enabled"), true)
+        WEBHOOK_USE_LINKED = builder.comment("Whether to imitate user's linked Discord accounts when sending messages from MC to DC")
+            .define(listOf("webhook", "useLinked"), false)
+        WEBHOOK_MC_AVATAR_URL = builder.comment("API that returns images based on Minecraft users. {{uuid}} and {{name}} can be used")
+            .define(listOf("webhook", "mcAvatarUrl"), "https://mc-heads.net/head/{{uuid}}")
+
         REQUIRE_LINK_FOR_WHITELIST = builder.comment("Whether to require being linked to be whitelisted")
             .define("requireLinkForWhitelist", false)
         DISCORD_LINK = builder.comment("Link to the discord where your users can run the /link command")
@@ -136,15 +172,15 @@ object AstralBotConfig {
                 )
             ) {
                 if (it !is String) {
-                    LOGGER.warn("$it in URL blocklist is not a String")
+                    LOGGER.warn("$it in URI blocklist is not a String")
                     return@defineList false
                 }
                 // TODO: Replace with better way to check for URL
                 try {
-                    URL(it)
+                    URI(it)
                     return@defineList true
                 } catch (e: Exception) {
-                    LOGGER.warn("Failed to parse URL on blocklist: $it", e)
+                    LOGGER.warn("Failed to parse URI on blocklist: $it", e)
                     return@defineList false
                 }
             }
@@ -180,12 +216,12 @@ object AstralBotConfig {
     fun urlAllowed(url: String?): Boolean {
         if (url == null) return true
         try {
-            val parsedURL = URL(url)
+            val parsedURL = URI(url)
             for (blockedURL in URL_BLOCKLIST.get()) {
-                if (parsedURL.host == URL(blockedURL).host) return false
+                if (parsedURL.host == URI(blockedURL).host) return false
             }
         } catch (e: Exception) {
-            LOGGER.warn("URL $url", e)
+            LOGGER.warn("URI $url", e)
             return false
         }
         return true

common/src/main/kotlin/dev/erdragh/astralbot/config/AstralBotTextConfig.kt

@@ -16,13 +16,17 @@ object AstralBotTextConfig {
 
     val PLAYER_MESSAGE: ModConfigSpec.ConfigValue<String>
 
+    val WEBHOOK_NAME_TEMPLATE: ModConfigSpec.ConfigValue<String>
+
     val DISCORD_MESSAGE: ModConfigSpec.ConfigValue<String>
     val DISCORD_REPLY: ModConfigSpec.ConfigValue<String>
     val DISCORD_EMBEDS: ModConfigSpec.ConfigValue<String>
 
     val RELOAD_ERROR: ModConfigSpec.ConfigValue<String>
     val RELOAD_SUCCESS: ModConfigSpec.ConfigValue<String>
 
+    val WHITELIST_LINKED_NOT_ALLOWED: ModConfigSpec.ConfigValue<String>
+
     val LINK_NO_MINECRAFT: ModConfigSpec.ConfigValue<String>
     val LINK_MINECRAFT_TAKEN: ModConfigSpec.ConfigValue<String>
     val LINK_DISCORD_TAKEN: ModConfigSpec.ConfigValue<String>
@@ -63,12 +67,19 @@ object AstralBotTextConfig {
                 .define("tickReport", "Average Tick Time: {{mspt}} MSPT (TPS: {{tps}})")
 
         PLAYER_MESSAGE =
-            builder.comment("""Template for how Minecraft chat messages are sent to Discord.
+            builder.comment("""Template for how Minecraft chat messages are sent to Discord if webhooks aren't used
                 The player's name can be accessed via {{name}} and its name with pre- and suffix
                 via {{fullName}}. The message itself is accessed via {{message}}.
             """.replace(whitespaceRegex, "\n"))
                 .define(mutableListOf("messages", "minecraft"), "<{{fullName}}> {{message}}")
 
+        WEBHOOK_NAME_TEMPLATE =
+            builder.comment("""Template for how chat synchronization using Webhooks formats
+                the message author's name.
+                The player's primary name can be accessed via {{primary}} and the secondary name via {{secondary}}.
+            """.replace(whitespaceRegex, "\n"))
+                .define(mutableListOf("webhook", "name"), "{{primary}} ({{secondary}})")
+
         DISCORD_MESSAGE =
             builder.comment("""Template for how Discord messages are synchronized to Minecraft.
                 The sender is referenced by {{user}}. The optional response is accessed by {{reply}}.
@@ -92,6 +103,13 @@ object AstralBotTextConfig {
         RELOAD_SUCCESS = builder.comment("Message sent to Discord after a successful reload")
             .define(mutableListOf("reload", "success"), "Reloaded commands for guild")
 
+        WHITELIST_LINKED_NOT_ALLOWED = builder.comment("""
+            The message the user gets shown if they are already linked, but have not
+            yet been whitelisted by another means (i.e. being an operator, being on the vanilla whitelist, etc.)
+            The Minecraft username is accessible via {{name}}
+            """.replace(whitespaceRegex, "\n"))
+            .define(listOf("whitelist", "linkedNotAllowed"), "Hi {{mc}}! You're already linked, but not yet whitelisted.")
+
         LINK_NO_MINECRAFT =
             builder.comment("""
                 Message for when there's no Minecraft account associated with the given Link code.