Allowing users to add LINQ order by operators with literal SQL. Closes …

…GH-2843

docs/configuration/cli.md

@@ -1,106 +0,0 @@
-# Command Line Tooling
-
-::: warning
-The usage of Marten.CommandLine shown in this document is only valid for applications bootstrapped with the
-[generic host builder](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/host/generic-host) with Marten registered in the application's IoC container.
-:::
-
-There is a separate NuGet package called _Marten.CommandLine_ that can be used to quickly add command-line tooling directly to
-your .Net Core application that uses Marten. _Marten.CommandLine_ is an extension library to [Oakton](https://jasperfx.github.io/oakton) that
-is the actual command line parser in this case.
-
-To use the expanded command line options to a .NET application, add a reference to the _Marten.CommandLine_ Nuget and add this line of code to your `Program.cs`:
-
-<!-- snippet: sample_using_WebApplication_1 -->
-<a id='snippet-sample_using_webapplication_1'></a>
-```cs
-var builder = WebApplication.CreateBuilder(args);
-
-// Easiest to just do this right after creating builder
-// Must be done before calling builder.Build() at least
-builder.Host.ApplyOaktonExtensions();
-```
-<sup><a href='https://github.com/JasperFx/marten/blob/master/src/samples/MinimalAPI/Program.cs#L9-L17' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_using_webapplication_1' title='Start of snippet'>anchor</a></sup>
-<!-- endSnippet -->
-
-And finally, use Oakton as the command line parser and executor by replacing `App.Run()` as the last line of code in your
-`Program.cs` file:
-
-<!-- snippet: sample_using_WebApplication_2 -->
-<a id='snippet-sample_using_webapplication_2'></a>
-```cs
-// Instead of App.Run(), use the app.RunOaktonCommands(args)
-// as the last line of your Program.cs file
-return await app.RunOaktonCommands(args);
-```
-<sup><a href='https://github.com/JasperFx/marten/blob/master/src/samples/MinimalAPI/Program.cs#L51-L57' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_using_webapplication_2' title='Start of snippet'>anchor</a></sup>
-<!-- endSnippet -->
-
-Once the _Marten.CommandLine_ Nuget is installed and Oakton is handling your command line parsing, you should be able to see the Marten commands by typing `dotnet run -- help` in the command line terminal of your choice at the root of your project:
-
-```bash
-  ----------------------------------------------------------------------------------------------------------
-    Available commands:
-  ----------------------------------------------------------------------------------------------------------
-        check-env -> Execute all environment checks against the application
-         describe -> Writes out a description of your running application to either the console or a file
-             help -> list all the available commands
-     marten-apply -> Applies all outstanding changes to the database based on the current configuration
-    marten-assert -> Assert that the existing database matches the current Marten configuration
-      marten-dump -> Dumps the entire DDL for the configured Marten database
-     marten-patch -> Evaluates the current configuration against the database and writes a patch and drop file if there are any differences
-      projections -> Rebuilds all projections of specified kind
-              run -> Start and run this .Net application
-  ----------------------------------------------------------------------------------------------------------
-```
-
-If you're not using the dotnet CLI yet, you'd just need to compile your new console application like you've always done and call the exe directly. If you're familiar with the *nix style of command-line interfaces ala Git, you should feel right at home with the command line usage in Marten.
-
-For the sake of usability, let's say that you stick a file named "marten.cmd" (or the *nix shell file equivalent) at the root of your codebase like so:
-
-```bash
-dotnet run --project src/MyConsoleApp %*
-```
-
-All the example above does is delegate any arguments to your console application. Once you have that file, some sample usages are shown below:
-
-Assert that the database matches the current database. This command will fail if there are differences
-
-```bash
-marten marten-assert --log log.txt
-```
-
-This command tries to update the database to reflect the application configuration
-
-```bash
-marten marten-apply --log log.txt
-```
-
-This dumps a single file named "database.sql" with all the DDL necessary to build the database to
-match the application configuration
-
-```bash
-marten marten-dump database.sql
-```
-
-This dumps the DDL to separate files per document
-type to a folder named "scripts"
-
-```bash
-marten marten-dump scripts --by-type
-```
-
-Create a patch file called "patch1.sql" and
-the corresponding rollback file "patch.drop.sql" if any
-differences are found between the application configuration
-and the database
-
-```bash
-marten marten-patch patch1.sql --drop patch1.drop.sql
-```
-
-In all cases, the commands expose usage help through "marten help [command]." Each of the commands also exposes a "--conn" (or "-c" if you prefer) flag to override the database connection string and a "--log" flag to record all the command output to a file.
-
-## Projections Support
-
-See [the Async Daemon documentation](/events/projections/async-daemon.md) for more information about the newly improved `projections` command.

docs/configuration/ioc.md

@@ -1,59 +0,0 @@
-# Custom IoC Integration
-
-::: tip
-The Marten team recommends using the `IServiceCollection.AddMarten()` extension method
-for IoC integration out of the box.
-:::
-
-The Marten team has striven to make the library perfectly usable without the usage of an IoC container, but you may still want to
-use an IoC container specifically to manage dependencies and the life cycle of Marten objects.
-While the `IServiceCollection.AddMarten()` method is the recommended way to integrate Marten
-into an IoC container, you can certainly recreate that functionality in the IoC container
-of your choice.
-
-::: tip INFO
-Lamar supports the .Net Core abstractions for IoC service registrations, so you *could* happily
-use the `AddMarten()` method directly with Lamar as well.
-:::
-
-Using [Lamar](https://jasperfx.github.io/lamar) as the example container, we recommend registering Marten something like this:
-
-<!-- snippet: sample_MartenServices -->
-<a id='snippet-sample_martenservices'></a>
-```cs
-public class MartenServices : ServiceRegistry
-{
-    public MartenServices()
-    {
-        ForSingletonOf<IDocumentStore>().Use(c =>
-        {
-            return DocumentStore.For(options =>
-            {
-                options.Connection("your connection string");
-                options.AutoCreateSchemaObjects = AutoCreate.None;
-
-                // other Marten configuration options
-            });
-        });
-
-        // Register IDocumentSession as Scoped
-        For<IDocumentSession>()
-            .Use(c => c.GetInstance<IDocumentStore>().LightweightSession())
-            .Scoped();
-
-        // Register IQuerySession as Scoped
-        For<IQuerySession>()
-            .Use(c => c.GetInstance<IDocumentStore>().QuerySession())
-            .Scoped();
-    }
-}
-```
-<sup><a href='https://github.com/JasperFx/marten/blob/master/src/Marten.Testing/DevelopmentModeRegistry.cs#L7-L34' title='Snippet source file'>snippet source</a> | <a href='#snippet-sample_martenservices' title='Start of snippet'>anchor</a></sup>
-<!-- endSnippet -->
-
-There are really only two key points here:
-
-1. There should only be one `IDocumentStore` object instance created in your application, so I scoped it as a "Singleton" in the StructureMap container
-1. The `IDocumentSession` service that you use to read and write documents should be scoped as "one per transaction." In typical usage, this
-   ends up meaning that an `IDocumentSession` should be scoped to a single HTTP request in web applications or a single message being handled in service
-   bus applications.

docs/documents/plv8.md

@@ -303,8 +303,7 @@ The `Patch.Remove()` operation removes the given item from a child collection:
 <!-- snippet: sample_remove_primitive_element -->
 <a id='snippet-sample_remove_primitive_element'></a>
 ```cs
-[Fact]
-public void remove_primitive_element()
+[Fact]public void remove_primitive_element()
 {
     var random = new Random();
     var target = Target.Random();

src/LinqTests/Acceptance/order_by_clauses.cs

@@ -22,4 +22,5 @@ static order_by_clauses()
         ordered(t => t.OrderBy(x => x.String).Skip(2));
         ordered(t => t.OrderBy(x => x.String).Take(2).Skip(2));
     }
+
 }

src/LinqTests/Acceptance/order_by_sql.cs

@@ -0,0 +1,58 @@
+using System.Linq;
+using System.Threading.Tasks;
+using Marten;
+using Marten.Testing.Documents;
+using Marten.Testing.Harness;
+using Xunit.Abstractions;
+
+namespace LinqTests.Acceptance;
+
+public class order_by_sql : OneOffConfigurationsContext
+{
+    private readonly ITestOutputHelper _output;
+
+    public order_by_sql(ITestOutputHelper output)
+    {
+        _output = output;
+    }
+
+    [Fact]
+    public async Task sort_by_literal_sql()
+    {
+        StoreOptions(x =>
+        {
+            x.Schema.For<Target>()
+                .Duplicate(x => x.String)
+                .Duplicate(x => x.AnotherString);
+        });
+
+        var targets = Target.GenerateRandomData(100).ToArray();
+        await theStore.BulkInsertAsync(targets);
+
+        theSession.Logger = new TestOutputMartenLogger(_output);
+
+        var expected = await theSession
+            .Query<Target>()
+            .OrderBy(x => x.String)
+            .ThenByDescending(x => x.AnotherString)
+            .Select(x => x.Id)
+            .ToListAsync();
+
+        var command = theSession
+            .Query<Target>()
+            .OrderBySql("string")
+            .ThenBySql("another_string desc")
+            .Select(x => x.Id).ToCommand();
+
+        _output.WriteLine(command.CommandText);
+
+        var actual = await theSession
+            .Query<Target>()
+            .OrderBySql("string")
+            .ThenBySql("another_string desc")
+            .Select(x => x.Id)
+            .ToListAsync();
+
+        actual.ShouldHaveTheSameElementsAs(expected);
+    }
+}

src/LinqTests/Bugs/Bug_2563_sub_collection_queries_and_OR.cs

@@ -19,34 +19,34 @@ public Bug_2563_sub_collection_queries_and_OR(ITestOutputHelper output)
     [Fact]
     public async Task get_distinct_number()
     {
-        theStore.Options.Schema.For<Target>()
+        theStore.Options.Schema.For<Bug2563Target>()
             .Duplicate(x => x.UserIds);
 
-        theSession.Store(new Target {Id = 1, IsPublic = false, UserIds = new [] { 1, 2, 3, 4, 5, 6 }});
-        theSession.Store(new Target {Id = 2, IsPublic = false, UserIds = new int[] { }});
-        theSession.Store(new Target {Id = 3, IsPublic = true, UserIds = new [] { 1, 2, 3 }});
-        theSession.Store(new Target {Id = 4, IsPublic = true, UserIds = new [] { 1, 2, 6 }});
-        theSession.Store(new Target {Id = 5, IsPublic = false, UserIds = new [] { 4, 5, 6 }});
-        theSession.Store(new Target {Id = 6, IsPublic = true, UserIds = new [] { 10 }});
+        theSession.Store(new Bug2563Target {Id = 1, IsPublic = false, UserIds = new [] { 1, 2, 3, 4, 5, 6 }});
+        theSession.Store(new Bug2563Target {Id = 2, IsPublic = false, UserIds = new int[] { }});
+        theSession.Store(new Bug2563Target {Id = 3, IsPublic = true, UserIds = new [] { 1, 2, 3 }});
+        theSession.Store(new Bug2563Target {Id = 4, IsPublic = true, UserIds = new [] { 1, 2, 6 }});
+        theSession.Store(new Bug2563Target {Id = 5, IsPublic = false, UserIds = new [] { 4, 5, 6 }});
+        theSession.Store(new Bug2563Target {Id = 6, IsPublic = true, UserIds = new [] { 10 }});
 
         await theSession.SaveChangesAsync();
         theSession.Logger = new TestOutputMartenLogger(_output);
 
 
-        var result1 = await theSession.Query<Target>()
+        var result1 = await theSession.Query<Bug2563Target>()
             .Where(x => x.IsPublic == false || x.UserIds.Contains(10))
             .ToListAsync();
 
         result1.Count.ShouldBeEquivalentTo(4);
 
         // This should pass without any error as the query will return results
-        var result2 = await theSession.Query<Target>().Where(x => x.IsPublic || x.UserIds.Contains(5)).ToListAsync();
+        var result2 = await theSession.Query<Bug2563Target>().Where(x => x.IsPublic || x.UserIds.Contains(5)).ToListAsync();
 
         result2.ShouldContain(x => x.Id == 1);
         result2.ShouldContain(x => x.Id == 5);
     }
 
-    public class Target
+    public class Bug2563Target
     {
         public int Id { get; set; }
 

src/Marten.Testing/Examples/LinqExamples.cs

@@ -77,6 +77,7 @@ public void case_insensitive_string_fields(IDocumentSession session)
         session.Query<Target>().Where(x => x.String.Contains("soMeThiNg", StringComparison.OrdinalIgnoreCase));
 
         session.Query<Target>().Where(x => x.String.Equals("ThE SaMe ThInG", StringComparison.OrdinalIgnoreCase));
+
     }
 
     #endregion
@@ -92,6 +93,9 @@ public void order_by(IDocumentSession session)
 
         // You can use multiple order by's
         session.Query<Target>().OrderBy(x => x.Date).ThenBy(x => x.Number);
+
+        // If you're brave, you can even use raw SQL literals as of Marten v7!
+        session.Query<Target>().OrderBySql("substring(d.data -> 'String', 1, 2)");
     }
 
     #endregion
@@ -199,4 +203,4 @@ public async Task sample_aggregation_operations(IQuerySession session)
     }
 
     #endregion
-}
+}

src/Marten/Linq/MartenLinqQueryable.cs

@@ -13,6 +13,7 @@
 using Marten.Internal.Storage;
 using Marten.Linq.Includes;
 using Marten.Linq.Parsing;
+using Marten.Linq.Parsing.Operators;
 using Marten.Linq.QueryHandlers;
 using Marten.Services;
 using Npgsql;

src/Marten/Linq/Members/DuplicatedField.cs

@@ -99,8 +99,7 @@ public DuplicatedField(EnumStorage enumStorage, QueryableMember innerMember,
     public string JsonPathSegment => throw new NotSupportedException();
     public string BuildOrderingExpression(Ordering ordering, CasingRule casingRule)
     {
-        // TODO -- memoize or intern this. Watch if this is a string!!!
-        if (ordering.Direction == OrderingDirection.Desc) return "d.id desc";
+        if (ordering.Direction == OrderingDirection.Desc) return $"{TypedLocator} desc";
 
         return TypedLocator;
     }

src/Marten/Linq/Parsing/Operators/OperatorLibrary.cs

@@ -16,18 +16,21 @@ public OperatorLibrary()
         Add<LastOperator>();
         Add<LastOrDefaultOperator>();
 
-        AddOrdering("OrderBy", OrderingDirection.Asc);
-        AddOrdering("ThenBy", OrderingDirection.Asc);
+        AddOrdering(nameof(QueryableExtensions.OrderBy), OrderingDirection.Asc);
+        AddOrdering(nameof(QueryableExtensions.ThenBy), OrderingDirection.Asc);
 
-        AddOrdering("OrderByDescending", OrderingDirection.Desc);
-        AddOrdering("ThenByDescending", OrderingDirection.Desc);
+        AddOrdering(nameof(QueryableExtensions.OrderByDescending), OrderingDirection.Desc);
+        AddOrdering(nameof(QueryableExtensions.ThenByDescending), OrderingDirection.Desc);
 
         Add<SelectManyOperator>();
         Add<SelectOperator>();
         Add<AnyOperator>();
         Add<DistinctOperator>();
         Add<IncludeOperator>();
 
+        Add<OrderBySqlOperator>();
+        Add<ThenBySqlOperator>();
+
         foreach (var mode in Enum.GetValues<SingleValueMode>()) addSingleValueMode(mode);
     }
 

src/Marten/Linq/Parsing/Operators/OrderBySqlOperator.cs

@@ -0,0 +1,36 @@
+using System.Linq;
+using System.Linq.Expressions;
+
+namespace Marten.Linq.Parsing.Operators;
+
+internal class OrderBySqlOperator : LinqOperator
+{
+    public OrderBySqlOperator() : base(nameof(QueryableExtensions.OrderBySql))
+    {
+    }
+
+    public override void Apply(ILinqQuery query, MethodCallExpression expression)
+    {
+        var sql = expression.Arguments.Last().ReduceToConstant();
+        var usage = query.CollectionUsageFor(expression);
+        var ordering = new Ordering((string)sql.Value);
+
+        usage.OrderingExpressions.Insert(0, ordering);
+    }
+}
+
+internal class ThenBySqlOperator : LinqOperator
+{
+    public ThenBySqlOperator() : base(nameof(QueryableExtensions.ThenBySql))
+    {
+    }
+
+    public override void Apply(ILinqQuery query, MethodCallExpression expression)
+    {
+        var sql = expression.Arguments.Last().ReduceToConstant();
+        var usage = query.CollectionUsageFor(expression);
+        var ordering = new Ordering((string)sql.Value);
+
+        usage.OrderingExpressions.Insert(0, ordering);
+    }
+}