Compilation error because of collision between external dependency and transitive IDE dependencies

[Undin]

What happened?

If the IDE artifact contains a dependency with version X and you add the same dependency with version Y, it may break compilation because X version is taken instead if Y. At the same time, plugin will use Y lib version in runtime. So the compilation classpath is wrong
As a particular example, if you use com.github.ajalt.clikt:clikt-core:5.0.0 dependency with CLion as IDE dependency, compilation fails because CLion contains an older version of the same lib with incompatible API

Relevant log output or stack trace

No response

Steps to reproduce

Checkout https://github.com/Undin/intellij-platform-gradle-plugins-bugs/tree/library-conflict and try to compile it

Gradle IntelliJ Plugin version

2.1.0

Gradle version

8.7

Operating System

macOS

Link to build, i.e. failing GitHub Action job

No response

[AlexanderBartash]

Try this:

plugins {
    kotlin("jvm") version "1.9.23"
    id("org.jetbrains.intellij.platform") version "2.1.0"
}

group = "org.example"
version = "1.0-SNAPSHOT"

repositories {
    mavenCentral()

    intellijPlatform {
        defaultRepositories()
        jetbrainsRuntime()
    }
}

val pluginClasspath: Configuration by project.configurations.creating {
    isCanBeConsumed = false
    isCanBeResolved = true
}

sourceSets {
    main.configure {
        compileClasspath = pluginClasspath + compileClasspath
    }
}

intellijPlatform {
    projectName = "IntelliJPlatformPluginExample"
    pluginConfiguration {
        id = "org.jetbrains.intellij.platform.example"
        name = "IntelliJ Platform Plugin Example"
        version = project.version.toString()
    }

  instrumentCode = false
}


dependencies {
    pluginClasspath("com.github.ajalt.clikt:clikt-core:5.0.0")
    intellijPlatform {
        clion("2024.2", useInstaller = false)
        jetbrainsRuntime()
    }
    testImplementation(kotlin("test"))
}

[AlexanderBartash]

But if you really want to do something like that you need to customize prepareSandbox task so that libs from that custom configuration are included & then also packaged into the distribution zip. Otherwise you have to duplicate the dependency as implementation(...).

[Undin]

@AlexanderBartash Thanks for the suggested workaround! In my particular case I will just avoid using CLion for the corresponding part of the plugin and will use IDEA instead.
So this issue is mostly about support for such cases by intellij-platform gradle plugin out of the box because it's a quite unexpected thing

[AlexanderBartash]

Something needs to be changed here:

intellij-platform-gradle-plugin/src/main/kotlin/org/jetbrains/intellij/platform/gradle/plugins/project/IntelliJPlatformBasePlugin.kt

Line 301 in a4e556c

this@configurations[Configurations.External.COMPILE_ONLY].extendsFrom(

https://docs.gradle.org/current/userguide/java_plugin.html#resolvable_configurations

[AlexanderBartash]

Can you try this https://plugins.jetbrains.com/docs/intellij/plugin-class-loaders.html#overriding-ide-dependencies ? I just found this accidentally, thought, may help here.

[AlexanderBartash]

It seems to me that, what ☝️ document suggests does the opposite of the expected.

[AlexanderBartash]

configurations {
    // It may be that doing it for 'compileOnly' here, or some other compile* configurations would be enough.
    all {
        resolutionStrategy.sortArtifacts(ResolutionStrategy.SortOrder.DEPENDENCY_FIRST)
    }
}

Actually helps, tried with 2.1.0

[AlexanderBartash]

In my plugin if I do that to all configurations it breaks the build because it affects some plugin. I would recommend to do it only for specific configurations like compileClasspath

[AlexanderBartash]

https://youtrack.jetbrains.com/issue/IDEA-242532

[AlexanderBartash]

It looks like dependencies order in Gradle is a big can of worms gradle/gradle#29275 but in our case it is made even worse because we also care about dependencies order in the IDE, which gets it from Gradle. Also to make it worse have a look at

https://github.com/gradle/gradle/blob/72c708412450a7674c4a2907a3f1aaae8997d01e/platforms/ide/ide/src/main/java/org/gradle/plugins/ide/idea/model/internal/IdeaDependenciesOptimizer.java#L51

Plus there is some logic on the IDE side.
How dependencies will be ordered after all this is unclear.

I tried to test what is recommended in https://plugins.jetbrains.com/docs/intellij/plugin-class-loaders.html#overriding-ide-dependencies and firstly DEPENDENCY_FIRST breaks my build at all, because it looks like in my case it does the opposite of expected. So I checked Gradle docs and was surprised that DEPENDENCY_FIRST : Artifacts for a consuming component should appear afte artifacts for its dependencies. which is indeed the opposite of what we want. CONSUMER_FIRST works for me but it does not change anything. With DEPENDENCY_FIRST I do see that libs from random plugins are being used instead of my declared dependencies.

https://github.com/gradle/gradle/blob/72c708412450a7674c4a2907a3f1aaae8997d01e/subprojects/core-api/src/main/java/org/gradle/api/artifacts/ResolutionStrategy.java#L398

Specifies the ordering for resolved artifacts. Options are:
<ul>
<li>{@link SortOrder#DEFAULT} : Don't specify the sort order. Gradle will provide artifacts in the default order.</li>
<li>{@link SortOrder#CONSUMER_FIRST} : Artifacts for a consuming component should appear <em>before</em> artifacts for its dependencies.</li>
<li>{@link SortOrder#DEPENDENCY_FIRST} : Artifacts for a consuming component should appear <em>after</em> artifacts for its dependencies.</li>
</ul>
A best attempt will be made to sort artifacts according the supplied {@link SortOrder}, but no guarantees will be made in the presence of dependency cycles.
NOTE: For a particular Gradle version, artifact ordering will be consistent. Multiple resolves for the same inputs will result in the
same outputs in the same order.

It works for me like this because it just happens that in my IDE ideaIU dependency is not the first in the list of dependencies. But in the example provided in this ticket the IDE is first. Why it works like this is a mystery to me. I tried to use different SortOrder in both projects, it does not influence IDE's dependencies order whatsoever, it only breaks or fixes Gradle's build.

[AlexanderBartash]

I've created a PR which improves this a bit but for tests only https://github.com/JetBrains/intellij-platform-gradle-plugin/pull/1818/files

[AlexanderBartash]

To fix this issue either something similar to what was done in the above PR needs to be done to the compile classpath of the main source set. But in that case we ca not rely on whatever happens to be in the sandbox, because we compile the code before the sandbox is present. We probably have to combine configurations instead.

Or all IDE's dependencies should be enriched with the missing metadata so that Gradle could use its build-in dependencies conflict resolution https://docs.gradle.org/current/userguide/component_metadata_rules.html#adding_missing_capabilities_to_detect_conflicts