Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transferhistory updates #888

Merged
merged 63 commits into from
Feb 5, 2025
Merged
Changes from all commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
d7ef0f2
new DB fields and shorter fields
Seifert69 Jan 8, 2025
41f3c96
added the fields Todd wanted
Seifert69 Jan 8, 2025
27a3bf6
Adding a public interface to TransferHistory
Seifert69 Jan 9, 2025
fe3a883
making public interface for the local tag index
Seifert69 Jan 9, 2025
7d1516c
Merge branch 'main' into newfields
Jan 9, 2025
eaf256f
starting process of saving data to history table instead of file
Jan 13, 2025
4e96a71
working on storgage
Jan 13, 2025
239ebe2
merged main
Jan 13, 2025
eb979bc
working on migration script
Jan 15, 2025
92eb933
renaming
Jan 16, 2025
314314f
working on summary
Jan 16, 2025
5f5c8ae
saving progress
Jan 17, 2025
7bde3f2
wip
Jan 17, 2025
84ebfd5
removed incorrect feature
Jan 18, 2025
1e2ae92
working on summary and tests
Jan 20, 2025
015e17f
wip
Jan 20, 2025
00a43d5
tests - wip
Jan 20, 2025
7337436
wip - test
Jan 20, 2025
c8433ec
tests wip
Jan 20, 2025
65d53dc
Merge branch 'main' into transferhistory-updates
Jan 20, 2025
d872ad8
changed for swagger
Jan 20, 2025
e01f190
upgraded from sql-lite generator and fixed exception
Jan 21, 2025
1614e92
wip
Jan 22, 2025
8985f09
tests -wip
Jan 23, 2025
752b5d6
tests wip
Jan 23, 2025
bcdbe49
Merge branch 'main' into transferhistory-updates
Jan 23, 2025
ec6620f
merged in main
Jan 23, 2025
82c89e9
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 23, 2025
f7b39f5
working on tests
Jan 23, 2025
219b26f
added delay in test; thx seb for the research
Jan 24, 2025
51f9646
fixed ordering of db table per seb
Jan 24, 2025
af9cbcf
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 24, 2025
f235bc0
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 28, 2025
0db551f
fixed migration issues
Jan 28, 2025
cc06a6c
fixed migration structure issue
Jan 28, 2025
1f5b400
changed to support postress types
Jan 28, 2025
e963bcc
fixed table
Jan 28, 2025
801b345
addressing postgress issues
Jan 29, 2025
64727af
testing for postgress and sql support
Jan 29, 2025
d04a16e
fixed modified time in fileModified event when transfer history is up…
Jan 29, 2025
35e6259
Fix DB null portability
sebbarg Jan 30, 2025
22b3466
Adding UpdateTransferHistoryRecordAsync
Seifert69 Jan 30, 2025
46e33b9
Adding UpdateTransferHistoryRecordAsync
Seifert69 Jan 30, 2025
d62ac07
adding a reference to the CRUD file GetAsync
Seifert69 Jan 30, 2025
30855d0
adding the auto generated files and adjustments to the TransferHistor…
Seifert69 Jan 30, 2025
b318481
CRUD with INSERT et al
Seifert69 Jan 30, 2025
6945f14
updated table strucutre and refactored to kill extra class
Jan 30, 2025
3f36f12
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 30, 2025
8de3366
fixed warning
Jan 30, 2025
cb15e71
fixes
Jan 30, 2025
711ce43
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 31, 2025
94d11fd
Added tests for DriveTransferHistory - especially for the hand coded …
Seifert69 Jan 31, 2025
1e91160
Merge branch 'transferhistory-updates' of github.com:homebase-id/odin…
Seifert69 Jan 31, 2025
62536fe
trying insert for new outbox them with fallback to update
Jan 31, 2025
46e0b2c
Merge branch 'transferhistory-updates' of github.com:homebase-id/odin…
Jan 31, 2025
9dfd77e
added exception
Jan 31, 2025
f231c79
crossing changes
Seifert69 Jan 31, 2025
74580ea
Merge branch 'transferhistory-updates' of github.com:homebase-id/odin…
Seifert69 Jan 31, 2025
066c41a
remove local tag operations to a diff class
Jan 31, 2025
5ed746a
fixed migraiton table
Feb 3, 2025
b3562e6
Merge branch 'main' into transferhistory-updates
toddmitchell Feb 3, 2025
416459c
Merge branch 'main' into transferhistory-updates
toddmitchell Feb 4, 2025
7fcbb29
fixed bug for sending file between two auto-connected identities
Feb 4, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 20 additions & 0 deletions src/apps/Odin.Hosting/Cli/CommandLine.cs
Original file line number Diff line number Diff line change
@@ -3,6 +3,7 @@
using System.Net.Sockets;
using Odin.Core.Configuration;
using Odin.Core.Storage.SQLite.Migrations;
using Odin.Core.Storage.SQLite.Migrations.TransferHistory;

namespace Odin.Hosting.Cli;

@@ -180,6 +181,25 @@ public static (bool didHandle, int exitCode) HandleCommandLineArgs(string[] args
// return (true, 0);
// }

//
// Command line: convert header files to database
//
// examples:
//
// dotnet run -- --header2database
//
// Note: arg[1] is path to registrations root (i.e. /identity-host/data/tenants)
// ASPNETCORE_ENVIRONMENT=Production ./Odin.Hosting --localapptags /identity-host/data/tenants
//
// launchSettings.json :"commandLineArgs": "--transferhistory /Users/taud/tmp/dotyou/tenants/"
//

if (args.Length == 2 && args[0] == "--transferhistory")
{
TransferHistoryMigration.Execute(args[1]);
return (true, 0);
}

return (false, 0);
}
}
Original file line number Diff line number Diff line change
@@ -83,6 +83,22 @@ protected async Task<IActionResult> GetFileHeaderByGlobalTransitId(GlobalTransit
return new JsonResult(result);
}

protected async Task<FileTransferHistoryResponse> GetFileTransferHistory(ExternalFileIdentifier file)
{
var storage = GetHttpFileSystemResolver().ResolveFileSystem().Storage;
var (count, history) = await storage.GetTransferHistory(this.MapToInternalFile(file), WebOdinContext);
if (history == null)
{
return null;
}

return new FileTransferHistoryResponse()
{
OriginalRecipientCount = count,
History = history
};
}

/// <summary>
/// Returns the payload for a given file
/// </summary>
@@ -195,7 +211,8 @@ protected async Task<SendReadReceiptResult> SendReadReceipt(SendReadReceiptReque
}

var internalFiles = request.Files.Select(MapToInternalFile).ToList();
return await peerOutgoingTransferService.SendReadReceipt(internalFiles, WebOdinContext, this.GetHttpFileSystemResolver().GetFileSystemType());
return await peerOutgoingTransferService.SendReadReceipt(internalFiles, WebOdinContext,
this.GetHttpFileSystemResolver().GetFileSystemType());
}

/// <summary>
@@ -396,4 +413,4 @@ private async Task<DeleteFileResult> PerformFileDelete(DeleteFileRequest request
return result;
}
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
using System.Linq;
using Odin.Core;
using Odin.Core.Identity;
using Odin.Services.Drives.DriveCore.Storage;

namespace Odin.Hosting.Controllers.Base.Drive;

public class FileTransferHistoryResponse
{
public int OriginalRecipientCount { get; set; }
public PagedResult<RecipientTransferHistoryItem> History { get; set; }

public RecipientTransferHistoryItem GetHistoryItem(OdinId recipient)
{
return this.History?.Results.SingleOrDefault(h => h.Recipient == recipient);
}
}
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
using System;
using System.Net;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
@@ -54,15 +55,36 @@ public async Task<IActionResult> GetFileHeaderAsGetRequest([FromQuery] Guid file
});
}

[HttpGet("files/transfer-history")]
public async Task<FileTransferHistoryResponse> GetFileTransferHistory([FromQuery] Guid fileId, [FromQuery] Guid alias,
[FromQuery] Guid type)
{
var result = await base.GetFileTransferHistory(new ExternalFileIdentifier()
{
FileId = fileId,
TargetDrive = new TargetDrive()
{
Alias = alias,
Type = type
}
});

if (null == result)
{
Response.StatusCode = (int)HttpStatusCode.NotFound;
return null;
}

return result;
}

/// <summary>
/// Retrieves a file's header and metadata by globalTransitId
/// </summary>
[HttpGet("files/header_byglobaltransitid")]
public async Task<IActionResult> GetFileHeaderByGlobalTransitId([FromQuery] Guid globalTransitId, [FromQuery] Guid alias,
[FromQuery] Guid type)
{


return await base.GetFileHeaderByGlobalTransitId(
new GlobalTransitIdFileIdentifier()
{
@@ -84,7 +106,6 @@ public async Task<IActionResult> GetFileHeaderByGlobalTransitId([FromQuery] Guid
[HttpPost("files/payload")]
public new async Task<IActionResult> GetPayloadStream([FromBody] GetPayloadRequest request)
{

return await base.GetPayloadStream(request);
}

@@ -95,7 +116,7 @@ public async Task<IActionResult> GetPayloadAsGetRequest([FromQuery] Guid fileId,
[FromQuery] int? chunkStart, [FromQuery] int? chunkLength)
{
FileChunk chunk = this.GetChunk(chunkStart, chunkLength);

return await base.GetPayloadStream(
new GetPayloadRequest()
{
@@ -120,7 +141,6 @@ public async Task<IActionResult> GetPayloadAsGetRequest([FromQuery] Guid fileId,
[HttpPost("files/thumb")]
public new async Task<IActionResult> GetThumbnail([FromBody] GetThumbnailRequest request)
{

return await base.GetThumbnail(request);
}

@@ -155,31 +175,27 @@ public async Task<IActionResult> GetThumbnailAsGetRequest([FromQuery] Guid fileI
[HttpPost("files/delete")]
public new async Task<IActionResult> DeleteFile([FromBody] DeleteFileRequest request)
{

return await base.DeleteFile(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.ClientTokenDrive })]
[HttpPost("files/deletefileidbatch")]
public new async Task<IActionResult> DeleteFileIdBatch([FromBody] DeleteFileIdBatchRequest request)
{

return await base.DeleteFileIdBatch(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.ClientTokenDrive })]
[HttpPost("files/deletegroupidbatch")]
public new async Task<IActionResult> DeleteFilesByGroupIdBatch([FromBody] DeleteFilesByGroupIdBatchRequest request)
{

return await base.DeleteFilesByGroupIdBatch(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.ClientTokenDrive })]
[HttpPost("files/deletepayload")]
public async Task<DeletePayloadResult> DeletePayloadC(DeletePayloadRequest request)
{

return await base.DeletePayload(request);
}

@@ -190,17 +206,15 @@ public async Task<DeletePayloadResult> DeletePayloadC(DeletePayloadRequest reque
[HttpPost("files/harddelete")]
public async Task<IActionResult> HardDeleteFileC([FromBody] DeleteFileRequest request)
{

return await base.HardDeleteFile(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.ClientTokenDrive })]
[HttpPost("files/send-read-receipt")]
public new async Task<IActionResult> SendReadReceipt(SendReadReceiptRequest request)
{

var result = await base.SendReadReceipt(request);
return new JsonResult(result);
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
using System;
using System.Linq;
using System.Net;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;
@@ -33,7 +34,6 @@ public class OwnerDriveStorageController(
[HttpPost("header")]
public new async Task<IActionResult> GetFileHeader([FromBody] ExternalFileIdentifier request)
{

return await base.GetFileHeader(request);
}

@@ -43,7 +43,6 @@ public class OwnerDriveStorageController(
[HttpGet("header")]
public async Task<IActionResult> GetFileHeaderAsGetRequest([FromQuery] Guid fileId, [FromQuery] Guid alias, [FromQuery] Guid type)
{

return await base.GetFileHeader(
new ExternalFileIdentifier()
{
@@ -56,15 +55,36 @@ public async Task<IActionResult> GetFileHeaderAsGetRequest([FromQuery] Guid file
});
}

[HttpGet("transfer-history")]
public async Task<FileTransferHistoryResponse> GetFileTransferHistory([FromQuery] Guid fileId, [FromQuery] Guid alias,
[FromQuery] Guid type)
{
var result = await base.GetFileTransferHistory(new ExternalFileIdentifier()
{
FileId = fileId,
TargetDrive = new TargetDrive()
{
Alias = alias,
Type = type
}
});

if (null == result)
{
Response.StatusCode = (int)HttpStatusCode.NotFound;
return null;
}

return result;
}

/// <summary>
/// Retrieves a file's header and metadata by globalTransitId
/// </summary>
[HttpGet("header_byglobaltransitid")]
public async Task<IActionResult> GetFileHeaderByGlobalTransitId([FromQuery] Guid globalTransitId, [FromQuery] Guid alias,
[FromQuery] Guid type)
{


return await base.GetFileHeaderByGlobalTransitId(
new GlobalTransitIdFileIdentifier()
{
@@ -84,7 +104,6 @@ public async Task<IActionResult> GetFileHeaderByGlobalTransitId([FromQuery] Guid
[HttpPost("payload")]
public new async Task<IActionResult> GetPayloadStream([FromBody] GetPayloadRequest request)
{

return await base.GetPayloadStream(request);
}

@@ -94,10 +113,11 @@ public async Task<IActionResult> GetFileHeaderByGlobalTransitId([FromQuery] Guid
/// </summary>
[SwaggerOperation(Tags = new[] { ControllerConstants.ClientTokenDrive })]
[HttpGet("payload")]
public async Task<IActionResult> GetPayloadAsGetRequest([FromQuery] Guid fileId, [FromQuery] Guid alias, [FromQuery] Guid type, [FromQuery] string key)
public async Task<IActionResult> GetPayloadAsGetRequest([FromQuery] Guid fileId, [FromQuery] Guid alias, [FromQuery] Guid type,
[FromQuery] string key)
{
FileChunk chunk = this.GetChunk(null, null);

return await base.GetPayloadStream(
new GetPayloadRequest()
{
@@ -124,7 +144,6 @@ public async Task<IActionResult> GetPayloadAsGetRequest([FromQuery] Guid fileId,
[HttpPost("thumb")]
public new async Task<IActionResult> GetThumbnail([FromBody] GetThumbnailRequest request)
{

return await base.GetThumbnail(request);
}

@@ -134,11 +153,11 @@ public async Task<IActionResult> GetPayloadAsGetRequest([FromQuery] Guid fileId,
/// See GET files/header
/// </summary>
[HttpGet("thumb")]
public async Task<IActionResult> GetThumbnailAsGetRequest([FromQuery] Guid fileId, [FromQuery] string payloadKey, [FromQuery] Guid alias,
public async Task<IActionResult> GetThumbnailAsGetRequest([FromQuery] Guid fileId, [FromQuery] string payloadKey,
[FromQuery] Guid alias,
[FromQuery] Guid type, [FromQuery] int width,
[FromQuery] int height)
{

return await base.GetThumbnail(new GetThumbnailRequest()
{
File = new ExternalFileIdentifier()
@@ -164,31 +183,27 @@ public async Task<IActionResult> GetThumbnailAsGetRequest([FromQuery] Guid fileI
[HttpPost("delete")]
public new async Task<IActionResult> DeleteFile([FromBody] DeleteFileRequest request)
{

return await base.DeleteFile(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.OwnerDrive })]
[HttpPost("deletefileidbatch")]
public new async Task<IActionResult> DeleteFileIdBatch([FromBody] DeleteFileIdBatchRequest request)
{

return await base.DeleteFileIdBatch(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.OwnerDrive })]
[HttpPost("deletegroupidbatch")]
public new async Task<IActionResult> DeleteFilesByGroupIdBatch([FromBody] DeleteFilesByGroupIdBatchRequest request)
{

return await base.DeleteFilesByGroupIdBatch(request);
}

[SwaggerOperation(Tags = new[] { ControllerConstants.OwnerDrive })]
[HttpPost("deletepayload")]
public async Task<DeletePayloadResult> DeletePayloadC(DeletePayloadRequest request)
{

return await base.DeletePayload(request);
}

@@ -199,16 +214,14 @@ public async Task<DeletePayloadResult> DeletePayloadC(DeletePayloadRequest reque
[HttpPost("harddelete")]
public async Task<IActionResult> HardDeleteFileC([FromBody] DeleteFileRequest request)
{

return await base.HardDeleteFile(request);
}

[HttpPost("send-read-receipt")]
public new async Task<IActionResult> SendReadReceipt(SendReadReceiptRequest request)
{

var result = await base.SendReadReceipt(request);
return new JsonResult(result);
}
}
}
}
2 changes: 1 addition & 1 deletion src/apps/Odin.Hosting/Properties/launchSettings.json
Original file line number Diff line number Diff line change
@@ -7,7 +7,7 @@
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"commandLineArgs": ""
"commandLineArgs": "--transferhistory /Users/todd/tmp/dotyou/tenants"
}
}
}
Loading