Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Transferhistory updates #888

Merged
merged 63 commits into from
Feb 5, 2025
Merged
Changes from 1 commit
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
d7ef0f2
new DB fields and shorter fields
Seifert69 Jan 8, 2025
41f3c96
added the fields Todd wanted
Seifert69 Jan 8, 2025
27a3bf6
Adding a public interface to TransferHistory
Seifert69 Jan 9, 2025
fe3a883
making public interface for the local tag index
Seifert69 Jan 9, 2025
7d1516c
Merge branch 'main' into newfields
Jan 9, 2025
eaf256f
starting process of saving data to history table instead of file
Jan 13, 2025
4e96a71
working on storgage
Jan 13, 2025
239ebe2
merged main
Jan 13, 2025
eb979bc
working on migration script
Jan 15, 2025
92eb933
renaming
Jan 16, 2025
314314f
working on summary
Jan 16, 2025
5f5c8ae
saving progress
Jan 17, 2025
7bde3f2
wip
Jan 17, 2025
84ebfd5
removed incorrect feature
Jan 18, 2025
1e2ae92
working on summary and tests
Jan 20, 2025
015e17f
wip
Jan 20, 2025
00a43d5
tests - wip
Jan 20, 2025
7337436
wip - test
Jan 20, 2025
c8433ec
tests wip
Jan 20, 2025
65d53dc
Merge branch 'main' into transferhistory-updates
Jan 20, 2025
d872ad8
changed for swagger
Jan 20, 2025
e01f190
upgraded from sql-lite generator and fixed exception
Jan 21, 2025
1614e92
wip
Jan 22, 2025
8985f09
tests -wip
Jan 23, 2025
752b5d6
tests wip
Jan 23, 2025
bcdbe49
Merge branch 'main' into transferhistory-updates
Jan 23, 2025
ec6620f
merged in main
Jan 23, 2025
82c89e9
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 23, 2025
f7b39f5
working on tests
Jan 23, 2025
219b26f
added delay in test; thx seb for the research
Jan 24, 2025
51f9646
fixed ordering of db table per seb
Jan 24, 2025
af9cbcf
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 24, 2025
f235bc0
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 28, 2025
0db551f
fixed migration issues
Jan 28, 2025
cc06a6c
fixed migration structure issue
Jan 28, 2025
1f5b400
changed to support postress types
Jan 28, 2025
e963bcc
fixed table
Jan 28, 2025
801b345
addressing postgress issues
Jan 29, 2025
64727af
testing for postgress and sql support
Jan 29, 2025
d04a16e
fixed modified time in fileModified event when transfer history is up…
Jan 29, 2025
35e6259
Fix DB null portability
sebbarg Jan 30, 2025
22b3466
Adding UpdateTransferHistoryRecordAsync
Seifert69 Jan 30, 2025
46e33b9
Adding UpdateTransferHistoryRecordAsync
Seifert69 Jan 30, 2025
d62ac07
adding a reference to the CRUD file GetAsync
Seifert69 Jan 30, 2025
30855d0
adding the auto generated files and adjustments to the TransferHistor…
Seifert69 Jan 30, 2025
b318481
CRUD with INSERT et al
Seifert69 Jan 30, 2025
6945f14
updated table strucutre and refactored to kill extra class
Jan 30, 2025
3f36f12
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 30, 2025
8de3366
fixed warning
Jan 30, 2025
cb15e71
fixes
Jan 30, 2025
711ce43
Merge branch 'main' into transferhistory-updates
toddmitchell Jan 31, 2025
94d11fd
Added tests for DriveTransferHistory - especially for the hand coded …
Seifert69 Jan 31, 2025
1e91160
Merge branch 'transferhistory-updates' of github.com:homebase-id/odin…
Seifert69 Jan 31, 2025
62536fe
trying insert for new outbox them with fallback to update
Jan 31, 2025
46e0b2c
Merge branch 'transferhistory-updates' of github.com:homebase-id/odin…
Jan 31, 2025
9dfd77e
added exception
Jan 31, 2025
f231c79
crossing changes
Seifert69 Jan 31, 2025
74580ea
Merge branch 'transferhistory-updates' of github.com:homebase-id/odin…
Seifert69 Jan 31, 2025
066c41a
remove local tag operations to a diff class
Jan 31, 2025
5ed746a
fixed migraiton table
Feb 3, 2025
b3562e6
Merge branch 'main' into transferhistory-updates
toddmitchell Feb 3, 2025
416459c
Merge branch 'main' into transferhistory-updates
toddmitchell Feb 4, 2025
7fcbb29
fixed bug for sending file between two auto-connected identities
Feb 4, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove local tag operations to a diff class
toddmitchell committed Jan 31, 2025
commit 066c41a2ac658cd818eca8a3b8a81652f78917e5

This file was deleted.

Original file line number Diff line number Diff line change
@@ -107,9 +107,6 @@ public class IdentityDatabase(ILifetimeScope lifetimeScope) : AbstractDatabase<I
private Lazy<MainIndexMeta> _mainIndexMeta;
public MainIndexMeta MainIndexMeta => LazyResolve(ref _mainIndexMeta);

private Lazy<LocalMetadataDataOperations> _localTags;
public LocalMetadataDataOperations LocalMetadataDataOperations => LazyResolve(ref _localTags);

//
// Connection
//
Original file line number Diff line number Diff line change
@@ -85,7 +85,6 @@ private static ContainerBuilder RegisterIdentityDatabase(this ContainerBuilder c

// Abstractions
cb.RegisterType<MainIndexMeta>().InstancePerLifetimeScope();
cb.RegisterType<LocalMetadataDataOperations>().InstancePerLifetimeScope();
return cb;
}

Original file line number Diff line number Diff line change
@@ -3,17 +3,72 @@
using System.Threading.Tasks;
using Odin.Core.Storage.Database.Identity.Abstractions;
using Odin.Core.Storage.Database.Identity.Connection;
using Odin.Core.Time;

namespace Odin.Core.Storage.Database.Identity.Table;

public class TableDriveLocalTagIndex(
CacheHelper cache,
ScopedIdentityConnectionFactory scopedConnectionFactory,
IdentityKey identityKey)
: TableDriveLocalTagIndexCRUD(cache, scopedConnectionFactory), ITableMigrator
: TableDriveLocalTagIndexCRUD(cache, scopedConnectionFactory: scopedConnectionFactory), ITableMigrator
{
private readonly ScopedIdentityConnectionFactory _scopedConnectionFactory = scopedConnectionFactory;

/// <summary>
/// Updates only the local tags table used when Querying the files (querybatch, query-modified, and futures)
/// </summary>
public async Task UpdateLocalTagsAsync(Guid driveId, Guid fileId, List<Guid> tags)
{
await using var cn = await _scopedConnectionFactory.CreateScopedConnectionAsync();
await using var tx = await cn.BeginStackedTransactionAsync();

await this.DeleteAllRowsAsync(driveId, fileId);
await this.InsertRowsAsync(driveId, fileId, tags);

tx.Commit();
}

public async Task<int> UpdateLocalAppMetadataAsync(Guid driveId, Guid fileId, Guid newVersionTag, string localMetadataJson)
{
await using var cn = await _scopedConnectionFactory.CreateScopedConnectionAsync();
await using var updateCommand = cn.CreateCommand();

updateCommand.CommandText = $"UPDATE driveMainIndex " +
$"SET hdrLocalVersionTag=@hdrLocalVersionTag,hdrLocalAppData=@hdrLocalAppData,modified=@modified " +
$"WHERE identityId=@identityId AND driveid=@driveId AND fileId=@fileId;";

var sparam1 = updateCommand.CreateParameter();
var sparam2 = updateCommand.CreateParameter();
var sparam3 = updateCommand.CreateParameter();
var versionTagParam = updateCommand.CreateParameter();
var contentParam = updateCommand.CreateParameter();
var modifiedParam = updateCommand.CreateParameter();

sparam1.ParameterName = "@identityId";
sparam2.ParameterName = "@driveId";
sparam3.ParameterName = "@fileId";
versionTagParam.ParameterName = "@hdrLocalVersionTag";
contentParam.ParameterName = "@hdrLocalAppData";
modifiedParam.ParameterName = "@modified";

updateCommand.Parameters.Add(sparam1);
updateCommand.Parameters.Add(sparam2);
updateCommand.Parameters.Add(sparam3);
updateCommand.Parameters.Add(versionTagParam);
updateCommand.Parameters.Add(contentParam);
updateCommand.Parameters.Add(modifiedParam);

sparam1.Value = identityKey.ToByteArray();
sparam2.Value = driveId.ToByteArray();
sparam3.Value = fileId.ToByteArray();
versionTagParam.Value = newVersionTag.ToByteArray();
contentParam.Value = localMetadataJson;
modifiedParam.Value = UnixTimeUtcUnique.Now().uniqueTime;

return await updateCommand.ExecuteNonQueryAsync();
}

public new async Task<DriveLocalTagIndexRecord> GetAsync(Guid driveId, Guid fileId, Guid tagId)
{
return await base.GetAsync(identityKey, driveId, fileId, tagId);
Original file line number Diff line number Diff line change
@@ -282,19 +282,19 @@ public async Task SaveFileHeaderAsync(StorageDrive drive, ServerFileHeader heade

public async Task SaveLocalMetadataAsync(Guid driveId, Guid fileId, Guid newVersionTag, string metadataJson)
{
await db.LocalMetadataDataOperations.UpdateLocalAppMetadataAsync(driveId, fileId, newVersionTag, metadataJson);
await db.DriveLocalTagIndex.UpdateLocalAppMetadataAsync(driveId, fileId, newVersionTag, metadataJson);
}

public async Task SaveLocalMetadataTagsAsync(Guid driveId, Guid fileId, LocalAppMetadata metadata)
{
await using var tx = await db.BeginStackedTransactionAsync();

// Update the tables used to query
await db.LocalMetadataDataOperations.UpdateLocalTagsAsync(driveId, fileId, metadata.Tags);
await db.DriveLocalTagIndex.UpdateLocalTagsAsync(driveId, fileId, metadata.Tags);

// Update the official metadata field
var json = OdinSystemSerializer.Serialize(metadata);
await db.LocalMetadataDataOperations.UpdateLocalAppMetadataAsync(driveId, fileId, metadata.VersionTag, json);
await db.DriveLocalTagIndex.UpdateLocalAppMetadataAsync(driveId, fileId, metadata.VersionTag, json);

tx.Commit();
}
Original file line number Diff line number Diff line change
@@ -3,7 +3,6 @@
using System.Threading.Tasks;
using Autofac;
using NUnit.Framework;
using Odin.Core.Storage.Database.Identity.Abstractions;
using Odin.Core.Storage.Database.Identity.Table;
using Odin.Core.Storage.Factory;
using Odin.Core.Time;
@@ -23,7 +22,7 @@ public async Task UpdateLocalMetadataContentOnly(DatabaseType databaseType)
await RegisterServicesAsync(databaseType);
await using var scope = Services.BeginLifetimeScope();
var tblDriveMainIndex = scope.Resolve<TableDriveMainIndex>();
var localTagsDataOperations = scope.Resolve<LocalMetadataDataOperations>();
var localTagsDataOperations = scope.Resolve<TableDriveLocalTagIndex>();

var driveId = Guid.NewGuid();
var fileId = Guid.NewGuid();
@@ -103,7 +102,6 @@ public async Task UpdateLocalMetadataTagsOnly(DatabaseType databaseType)
await using var scope = Services.BeginLifetimeScope();
var tblDriveMainIndex = scope.Resolve<TableDriveMainIndex>();
var tableDriveLocalTagIndex = scope.Resolve<TableDriveLocalTagIndex>();
var localTagsDataOperations = scope.Resolve<LocalMetadataDataOperations>();

var driveId = Guid.NewGuid();
var fileId = Guid.NewGuid();
@@ -151,7 +149,7 @@ public async Task UpdateLocalMetadataTagsOnly(DatabaseType databaseType)
// Act
//
List<Guid> tags = [Guid.NewGuid(), Guid.NewGuid(), Guid.NewGuid()];
await localTagsDataOperations.UpdateLocalTagsAsync(driveId, fileId, tags);
await tableDriveLocalTagIndex.UpdateLocalTagsAsync(driveId, fileId, tags);

//
// Assert