Auto merge of #16092 - kilpkonn:term_search_1, r=Veykril

feat: Introduce term search to rust-analyzer

# Introduce term search to `rust-analyzer`
_I've marked this as draft as there might be some shortcomings, please point them out so I can fix them. Otherwise I think it is kind of ready as I think I'll rather introduce extra functionality in follow up PRs._

Term search (or I guess expression search for rust) is a technique to generate code by basically making the types match.
Consider the following program
```rust
fn wrap(arg: i32) -> Option<i32> {
    todo!();
}
```
From the types of values in scope and constructors of `Option`, we can produce the expected result of wrapping the argument in `Option`

Dependently typed languages such as `Idris2` and `Agda` have similar tools to help with proofs, but this can be also used in everyday development as a "auto-complete".

# Demo videos

https://github.com/rust-lang/rust-analyzer/assets/19900308/7b68a1b7-7dba-4e31-9221-6c7485e77d88

https://github.com/rust-lang/rust-analyzer/assets/19900308/0fae530a-aabb-4b28-af71-e19f8d3d64b2

# What does it currently do
- It works well with locals, free functions, type constructors and non-static impl methods that take items by value.
- Works with functions/methods that take shared references, but not with unique references (very conservative).
- Can handle projections to struct fields (eg. `foo.bar.baz`) but this might me more conservative than it has to be to avoid conflicting with borrow checker
- Should create only valid programs (no type / borrow checking errors). Tested with `rust-analyzer analysis-stats /path/to/ripgrep/Cargo.toml --run-term-search --validate-term-search` (basically running `cargo check` on all of the generated programs and only error seems to be due to type inference which is more of issue of testing method.

# Performace / fitness
```txt
ripgrep (latest)
Tail Expr syntactic hits: 130/1692 (7%)
Tail Exprs found: 523/1692 (30%)
Term search avg time: 9ms
Term search:         15.64s, 97ginstr, 8mb

rust-analyzer (on this branch)
Tail Expr syntactic hits: 804/13860 (5%)
Tail Exprs found: 6757/13860 (48%)
Term search avg time: 78ms
Term search:         1088.23s, 6765ginstr, 98mb
```
Highly generic code seems to blow up the search space so currently the amount of generics allowed is functions/methods is limited down to 0 (1 didn't give much improvement and 2 is already like 0.5+s search time)

# Plans for the future (not in this PR)
- ``~~Add impl methods that do not take `self` type (should be quite straight forward)~~ Done
- Be smarter (aka less restrictive) about borrow checking - this seems quite hard but since the current approach is rather naive I think some easy improvement is available.
- ``~~See if it works as a autocomplete while typing~~ Done

_Feel free to ask questions / point of shortcoming either here or on Zulip, I'll be happy to address them. I'm doing this as part of my MSc thesis so I'll be working on it till summer anyway 😄_

crates/hir-def/src/attr.rs

@@ -377,27 +377,39 @@ impl AttrsWithOwner {
             AttrDefId::GenericParamId(it) => match it {
                 GenericParamId::ConstParamId(it) => {
                     let src = it.parent().child_source(db);
-                    RawAttrs::from_attrs_owner(
-                        db.upcast(),
-                        src.with_value(&src.value[it.local_id()]),
-                        db.span_map(src.file_id).as_ref(),
-                    )
+                    // FIXME: We should be never getting `None` here.
+                    match src.value.get(it.local_id()) {
+                        Some(val) => RawAttrs::from_attrs_owner(
+                            db.upcast(),
+                            src.with_value(val),
+                            db.span_map(src.file_id).as_ref(),
+                        ),
+                        None => RawAttrs::EMPTY,
+                    }
                 }
                 GenericParamId::TypeParamId(it) => {
                     let src = it.parent().child_source(db);
-                    RawAttrs::from_attrs_owner(
-                        db.upcast(),
-                        src.with_value(&src.value[it.local_id()]),
-                        db.span_map(src.file_id).as_ref(),
-                    )
+                    // FIXME: We should be never getting `None` here.
+                    match src.value.get(it.local_id()) {
+                        Some(val) => RawAttrs::from_attrs_owner(
+                            db.upcast(),
+                            src.with_value(val),
+                            db.span_map(src.file_id).as_ref(),
+                        ),
+                        None => RawAttrs::EMPTY,
+                    }
                 }
                 GenericParamId::LifetimeParamId(it) => {
                     let src = it.parent.child_source(db);
-                    RawAttrs::from_attrs_owner(
-                        db.upcast(),
-                        src.with_value(&src.value[it.local_id]),
-                        db.span_map(src.file_id).as_ref(),
-                    )
+                    // FIXME: We should be never getting `None` here.
+                    match src.value.get(it.local_id) {
+                        Some(val) => RawAttrs::from_attrs_owner(
+                            db.upcast(),
+                            src.with_value(val),
+                            db.span_map(src.file_id).as_ref(),
+                        ),
+                        None => RawAttrs::EMPTY,
+                    }
                 }
             },
             AttrDefId::ExternBlockId(it) => attrs_from_item_tree_loc(db, it),

crates/hir-ty/src/infer.rs

@@ -68,7 +68,7 @@ use crate::{
 #[allow(unreachable_pub)]
 pub use coerce::could_coerce;
 #[allow(unreachable_pub)]
-pub use unify::could_unify;
+pub use unify::{could_unify, could_unify_deeply};
 
 use cast::CastCheck;
 pub(crate) use closure::{CaptureKind, CapturedItem, CapturedItemWithoutTy};

crates/hir-ty/src/infer/unify.rs

@@ -74,6 +74,12 @@ impl<T: HasInterner<Interner = Interner>> Canonicalized<T> {
     }
 }
 
+/// Check if types unify.
+///
+/// Note that we consider placeholder types to unify with everything.
+/// This means that there may be some unresolved goals that actually set bounds for the placeholder
+/// type for the types to unify. For example `Option<T>` and `Option<U>` unify although there is
+/// unresolved goal `T = U`.
 pub fn could_unify(
     db: &dyn HirDatabase,
     env: Arc<TraitEnvironment>,
@@ -82,21 +88,35 @@ pub fn could_unify(
     unify(db, env, tys).is_some()
 }
 
+/// Check if types unify eagerly making sure there are no unresolved goals.
+///
+/// This means that placeholder types are not considered to unify if there are any bounds set on
+/// them. For example `Option<T>` and `Option<U>` do not unify as we cannot show that `T = U`
+pub fn could_unify_deeply(
+    db: &dyn HirDatabase,
+    env: Arc<TraitEnvironment>,
+    tys: &Canonical<(Ty, Ty)>,
+) -> bool {
+    let mut table = InferenceTable::new(db, env);
+    let vars = make_substitutions(tys, &mut table);
+    let ty1_with_vars = vars.apply(tys.value.0.clone(), Interner);
+    let ty2_with_vars = vars.apply(tys.value.1.clone(), Interner);
+    let ty1_with_vars = table.normalize_associated_types_in(ty1_with_vars);
+    let ty2_with_vars = table.normalize_associated_types_in(ty2_with_vars);
+    table.resolve_obligations_as_possible();
+    table.propagate_diverging_flag();
+    let ty1_with_vars = table.resolve_completely(ty1_with_vars);
+    let ty2_with_vars = table.resolve_completely(ty2_with_vars);
+    table.unify_deeply(&ty1_with_vars, &ty2_with_vars)
+}
+
 pub(crate) fn unify(
     db: &dyn HirDatabase,
     env: Arc<TraitEnvironment>,
     tys: &Canonical<(Ty, Ty)>,
 ) -> Option<Substitution> {
     let mut table = InferenceTable::new(db, env);
-    let vars = Substitution::from_iter(
-        Interner,
-        tys.binders.iter(Interner).map(|it| match &it.kind {
-            chalk_ir::VariableKind::Ty(_) => table.new_type_var().cast(Interner),
-            // FIXME: maybe wrong?
-            chalk_ir::VariableKind::Lifetime => table.new_type_var().cast(Interner),
-            chalk_ir::VariableKind::Const(ty) => table.new_const_var(ty.clone()).cast(Interner),
-        }),
-    );
+    let vars = make_substitutions(tys, &mut table);
     let ty1_with_vars = vars.apply(tys.value.0.clone(), Interner);
     let ty2_with_vars = vars.apply(tys.value.1.clone(), Interner);
     if !table.unify(&ty1_with_vars, &ty2_with_vars) {
@@ -125,6 +145,21 @@ pub(crate) fn unify(
     ))
 }
 
+fn make_substitutions(
+    tys: &chalk_ir::Canonical<(chalk_ir::Ty<Interner>, chalk_ir::Ty<Interner>)>,
+    table: &mut InferenceTable<'_>,
+) -> chalk_ir::Substitution<Interner> {
+    Substitution::from_iter(
+        Interner,
+        tys.binders.iter(Interner).map(|it| match &it.kind {
+            chalk_ir::VariableKind::Ty(_) => table.new_type_var().cast(Interner),
+            // FIXME: maybe wrong?
+            chalk_ir::VariableKind::Lifetime => table.new_type_var().cast(Interner),
+            chalk_ir::VariableKind::Const(ty) => table.new_const_var(ty.clone()).cast(Interner),
+        }),
+    )
+}
+
 bitflags::bitflags! {
     #[derive(Default, Clone, Copy)]
     pub(crate) struct TypeVariableFlags: u8 {
@@ -431,6 +466,18 @@ impl<'a> InferenceTable<'a> {
         true
     }
 
+    /// Unify two relatable values (e.g. `Ty`) and check whether trait goals which arise from that could be fulfilled
+    pub(crate) fn unify_deeply<T: ?Sized + Zip<Interner>>(&mut self, ty1: &T, ty2: &T) -> bool {
+        let result = match self.try_unify(ty1, ty2) {
+            Ok(r) => r,
+            Err(_) => return false,
+        };
+        result.goals.iter().all(|goal| {
+            let canonicalized = self.canonicalize(goal.clone());
+            self.try_resolve_obligation(&canonicalized).is_some()
+        })
+    }
+
     /// Unify two relatable values (e.g. `Ty`) and return new trait goals arising from it, so the
     /// caller needs to deal with them.
     pub(crate) fn try_unify<T: ?Sized + Zip<Interner>>(
@@ -501,7 +548,8 @@ impl<'a> InferenceTable<'a> {
 
     fn register_obligation_in_env(&mut self, goal: InEnvironment<Goal>) {
         let canonicalized = self.canonicalize(goal);
-        if !self.try_resolve_obligation(&canonicalized) {
+        let solution = self.try_resolve_obligation(&canonicalized);
+        if matches!(solution, Some(Solution::Ambig(_))) {
             self.pending_obligations.push(canonicalized);
         }
     }
@@ -627,38 +675,35 @@ impl<'a> InferenceTable<'a> {
     fn try_resolve_obligation(
         &mut self,
         canonicalized: &Canonicalized<InEnvironment<Goal>>,
-    ) -> bool {
+    ) -> Option<chalk_solve::Solution<Interner>> {
         let solution = self.db.trait_solve(
             self.trait_env.krate,
             self.trait_env.block,
             canonicalized.value.clone(),
         );
 
-        match solution {
+        match &solution {
             Some(Solution::Unique(canonical_subst)) => {
                 canonicalized.apply_solution(
                     self,
                     Canonical {
-                        binders: canonical_subst.binders,
+                        binders: canonical_subst.binders.clone(),
                         // FIXME: handle constraints
-                        value: canonical_subst.value.subst,
+                        value: canonical_subst.value.subst.clone(),
                     },
                 );
-                true
             }
             Some(Solution::Ambig(Guidance::Definite(substs))) => {
-                canonicalized.apply_solution(self, substs);
-                false
+                canonicalized.apply_solution(self, substs.clone());
             }
             Some(_) => {
                 // FIXME use this when trying to resolve everything at the end
-                false
             }
             None => {
                 // FIXME obligation cannot be fulfilled => diagnostic
-                true
             }
         }
+        solution
     }
 
     pub(crate) fn callable_sig(

crates/hir-ty/src/lib.rs

@@ -79,8 +79,8 @@ pub use builder::{ParamKind, TyBuilder};
 pub use chalk_ext::*;
 pub use infer::{
     closure::{CaptureKind, CapturedItem},
-    could_coerce, could_unify, Adjust, Adjustment, AutoBorrow, BindingMode, InferenceDiagnostic,
-    InferenceResult, OverloadedDeref, PointerCast,
+    could_coerce, could_unify, could_unify_deeply, Adjust, Adjustment, AutoBorrow, BindingMode,
+    InferenceDiagnostic, InferenceResult, OverloadedDeref, PointerCast,
 };
 pub use interner::Interner;
 pub use lower::{