Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DYN-7278: Fix dictionary preview by adding support for more types #15750

Merged
merged 9 commits into from
Jan 14, 2025
Merged

DYN-7278: Fix dictionary preview by adding support for more types #15750

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ec9ef14
fix dictionary preview by adding support for more types
aparajit-pratap Jan 10, 2025
ab6bad8
revert assemblyinfo
aparajit-pratap Jan 10, 2025
5fb2728
process json object for preview
aparajit-pratap Jan 10, 2025
4569a81
cleanup
aparajit-pratap Jan 10, 2025
432af98
add tests for byte and uint previews
aparajit-pratap Jan 14, 2025
688332e
add test for JSON preview
aparajit-pratap Jan 14, 2025
836f3f9
remove bad test
aparajit-pratap Jan 14, 2025
daa15f2
rename nodes
aparajit-pratap Jan 14, 2025
28b4f3d
review comments
aparajit-pratap Jan 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 33 additions & 2 deletions src/DynamoCoreWpf/Interfaces/IWatchHandler.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,15 +1,18 @@
using System;

using System;
using System.Collections;
using System.Collections.Generic;
using System.Globalization;
using System.Linq;
using Dynamo.Configuration;
using Newtonsoft.Json.Linq;

using Dynamo.Extensions;
using Dynamo.ViewModels;
using Dynamo.Wpf.Properties;
using ProtoCore.DSASM;
using ProtoCore.Mirror;
using ProtoCore.Utils;
using Newtonsoft.Json;

namespace Dynamo.Interfaces
{
Expand Down Expand Up @@ -83,7 +86,18 @@ private WatchViewModel ProcessThing(object value, ProtoCore.RuntimeCore runtimeC

return node;
}
if (value is JObject obj)
{
var dict = ConvertJObjectToDictionary(obj);
var node = new WatchViewModel(dict.Keys.Any() ? WatchViewModel.DICTIONARY : WatchViewModel.EMPTY_DICTIONARY, tag, RequestSelectGeometry, true);

foreach (var e in dict.Keys.Zip(dict.Values, (key, val) => new { key, val }))
{
node.Children.Add(ProcessThing(e.val, runtimeCore, tag + ":" + e.key, showRawData, callback));
}

return node;
}
if (!(value is string) && value is IEnumerable)
{
var list = (value as IEnumerable).Cast<dynamic>().ToList();
Expand Down Expand Up @@ -122,6 +136,23 @@ private WatchViewModel ProcessThing(object value, ProtoCore.RuntimeCore runtimeC
return new WatchViewModel(value, tag, RequestSelectGeometry);
}

private static Dictionary<string, object> ConvertJObjectToDictionary(JObject jObject)
{
Copy link
Member

@mjkkirschner mjkkirschner Jan 10, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a couple thoughts.

  1. If I am understanding correctly, this is essentially arbitrary deserialization of json using the ToObject() calls. It's not clear to me exactly what types this allows deserialization of.

I think the default deserialization calls use a secure serializer (TypeNameHandling = None), but I am not sure, it might be good to make it explicit by using a serializer here with settings that encode that specifically - for example:

Dynamo/src/DynamoCore/Models/DynamoModel.cs

Line 2158 in 3d8a262

TypeNameHandling = TypeNameHandling.None,

IMO this kind of thing becomes more important now that we have to support DaaS - though I guess maybe this code is never even present on the service? (wpf?)

  1. Where is this type coming from? The Forma nodes? Personally I feel we should ditch newtonsoft everywhere and instead use the built in system.text.json - it's harder to screw up security using it (though not impossible)

  2. I have not thought about it much - but wouldn't it be nice if packages could provide their own watch handlers?

  3. why are both dictionary and this newtonsoft parsing done in the object handler instead of defining more specific overloads - was there a performance reason for this?

Copy link
Contributor Author

@aparajit-pratap aparajit-pratap Jan 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. These are NewtonSoft.Json JObject types that are coming mainly from Forma elements represented as JSON. As of now, I'm not sure what other nodes exist that output such types that have preview issues but so far, I'm seeing this issue only with the Forma nodes. The JObject seems to have a structure similar to dictionaries or key-value pairs so it just seems natural to be able to convert them to a form such that they can be previewed such as dictionaries.
  2. Yes, to types coming from Forma. I'll see if I can switch the deserialization to use System.Text.Json instead but I think it would mean changing the DynamoForma package to use STJ as well. If that's the case, that would be out of scope for this fix.
  3. Not sure given that I think this is the first time we've encountered a case such as this where nodes in a package are having preview issues and therefore such cases seem to be rare but still open to discuss.
  4. I'm not sure I understand. What exactly are you referring to by object handler and overloads?
    I think I see what you mean. No particular reason for doing it this way. I can add an overload for ProcessThing to handle JObject and maybe another for Dictionary, if that's what you're alluding to.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes seem to be the first step in trying to support NewtonSoft types as native dynamo types.
If JObjects are passed around between nodes, do you foresee any other nodes/functionality that might not work with this type?

If we want to favor more system.text.json instead of Newtonsoft, I assume we'll encounter similar system.text.json types (like JsonElement and JSonDocument). Should we try to support these at the Proto level in the Marshaler code?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of like the packages could provide their own watch handlers? idea. Some packages might bring their own magic types, and would be nice if they could specify the way Dynamo presents the data.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pinzart90 these changes are only affecting the node previews for these types at the UI level and are not related to marshaling types so in that sense IMO these changes are safer. In order to support System.Text.Json, yes, I believe we'll need to add support for it here as well although I don't think anything specific would need to be done in the marshaler.

var settings = new JsonSerializerSettings
{
// Add any specific settings you need here
NullValueHandling = NullValueHandling.Ignore,
MissingMemberHandling = MissingMemberHandling.Ignore,
TypeNameHandling = TypeNameHandling.None
};

var json = jObject.ToString();
var dictionary = JsonConvert.DeserializeObject<Dictionary<string, object>>(json, settings);

return dictionary;
}


private WatchViewModel ProcessThing(double value, ProtoCore.RuntimeCore runtimeCore, string tag, bool showRawData, WatchHandlerCallback callback)
{
return new WatchViewModel(value, tag, RequestSelectGeometry);
Expand Down
12 changes: 12 additions & 0 deletions src/DynamoCoreWpf/ViewModels/Preview/WatchViewModel.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,12 @@ private static string GetStringFromObject(object obj)
return ((DateTime)obj).ToString(PreferenceSettings.DefaultDateFormat, CultureInfo.InvariantCulture);
case TypeCode.Object:
return ObjectToLabelString(obj);
case TypeCode.Byte:
return ((byte)obj).ToString(CultureInfo.InvariantCulture);
case TypeCode.UInt32:
return ((uint)obj).ToString(CultureInfo.InvariantCulture);
case TypeCode.UInt64:
return ((ulong)obj).ToString(CultureInfo.InvariantCulture);
default:
return (string)obj;
};
Expand Down Expand Up @@ -313,6 +319,12 @@ private string GetDisplayType(object obj)
return nameof(TypeCode.Object);
case TypeCode.String:
return nameof(TypeCode.String);
case TypeCode.Byte:
return nameof(TypeCode.Byte);
case TypeCode.UInt32:
return nameof(TypeCode.UInt32);
case TypeCode.UInt64:
return nameof(TypeCode.UInt64);
Copy link
Contributor Author

@aparajit-pratap aparajit-pratap Jan 11, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe the type returned from GetDisplayType is actually being used currently so it might just be better to remove this function for the time being.

case TypeCode.Empty:
return String.Empty;
default:
Expand Down
78 changes: 78 additions & 0 deletions test/DynamoCoreWpfTests/WatchNodeTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ protected override void GetLibrariesToPreload(List<string> libraries)
libraries.Add("VMDataBridge.dll");
libraries.Add("ProtoGeometry.dll");
libraries.Add("DesignScriptBuiltin.dll");
libraries.Add("DSCoreNodes.dll");
libraries.Add("FunctionObject.ds");
libraries.Add("FFITarget.dll");
base.GetLibrariesToPreload(libraries);
Expand Down Expand Up @@ -363,6 +364,83 @@ public void WatchMultiReturnNodeOrder()
Assert.AreEqual("1", children[3].NodeLabel);
}

[Test]
public void WatchDictionaryByteValuesDisplaysCorrectly()
{
string openPath = Path.Combine(TestDirectory, @"core\watch\WatchDictionaryByteValuesDisplaysCorrectly.dyn");
ViewModel.OpenCommand.Execute(openPath);
ViewModel.HomeSpace.Run();

var watchNode = ViewModel.Model.CurrentWorkspace.FirstNodeFromWorkspace<Watch>();
var watchVM = ViewModel.WatchHandler.GenerateWatchViewModelForData(
watchNode.CachedValue, watchNode.OutPorts.Select(p => p.Name),
ViewModel.Model.EngineController.LiveRunnerRuntimeCore,
watchNode.AstIdentifierForPreview.Name, true);

var list = watchVM.Children;
Assert.AreEqual(1, list.Count);
var children = list[0].Children;
Assert.AreEqual(12, children.Count);
Assert.AreEqual("Byte", children[0].ValueType);
Assert.AreEqual("72", children[0].NodeLabel);
}

[Test]
public void WatchDictionaryUintValuesDisplaysCorrectly()
{
string openPath = Path.Combine(TestDirectory, @"core\watch\WatchDictionaryUintValuesDisplaysCorrectly.dyn");
ViewModel.OpenCommand.Execute(openPath);
ViewModel.HomeSpace.Run();

var watchNode = ViewModel.Model.CurrentWorkspace.FirstNodeFromWorkspace<Watch>();
var watchVM = ViewModel.WatchHandler.GenerateWatchViewModelForData(
watchNode.CachedValue, watchNode.OutPorts.Select(p => p.Name),
ViewModel.Model.EngineController.LiveRunnerRuntimeCore,
watchNode.AstIdentifierForPreview.Name, true);

var list = watchVM.Children;
Assert.AreEqual(1, list.Count);
var children = list[0].Children;
Assert.AreEqual(5, children.Count);
Assert.AreEqual("UInt32", children[0].ValueType);
Assert.AreEqual("3", children[2].NodeLabel);

var uint64node = ViewModel.Model.CurrentWorkspace.GetDSFunctionNodeFromWorkspace("DummyZeroTouchClass.PreviewUint64Dictionary");
var vm = ViewModel.WatchHandler.GenerateWatchViewModelForData(
uint64node.CachedValue, watchNode.OutPorts.Select(p => p.Name),
ViewModel.Model.EngineController.LiveRunnerRuntimeCore,
watchNode.AstIdentifierForPreview.Name, true);

list = vm.Children;
Assert.AreEqual(1, list.Count);
children = list[0].Children;
Assert.AreEqual(5, children.Count);
Assert.AreEqual("UInt64", children[0].ValueType);
Assert.AreEqual("3", children[2].NodeLabel);
}

[Test]
public void WatchDictionaryJSONValuesDisplaysCorrectly()
{

string openPath = Path.Combine(TestDirectory, @"core\watch\WatchDictionaryJSONValuesDisplaysCorrectly.dyn");
ViewModel.OpenCommand.Execute(openPath);
ViewModel.HomeSpace.Run();

var watchNode = ViewModel.Model.CurrentWorkspace.FirstNodeFromWorkspace<Watch>();
var watchVM = ViewModel.WatchHandler.GenerateWatchViewModelForData(
watchNode.CachedValue, watchNode.OutPorts.Select(p => p.Name),
ViewModel.Model.EngineController.LiveRunnerRuntimeCore,
watchNode.AstIdentifierForPreview.Name, true);

var list = watchVM.Children;
Assert.AreEqual(1, list.Count);
var children = list[0].Children;
Assert.AreEqual(2, children.Count);
Assert.AreEqual("String", children[0].ValueType);
Assert.AreEqual("value1", children[0].NodeLabel);
}

[Test]
public void WatchNestedDictionaryPreviewFromMlutiReturnNode()
{
Expand Down
47 changes: 46 additions & 1 deletion test/Engine/FFITarget/DummyZeroTouchClass.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
using Dynamo.Graph.Nodes;
using Dynamo.Graph.Nodes;
using System.Text;
using System;
using System.Collections.Generic;
using Newtonsoft.Json.Linq;

namespace FFITarget
{
Expand All @@ -14,5 +18,46 @@ public int FunctionWithoutDescription(int a)
{
return 0;
}

public static Dictionary<string, object> PreviewByteDictionary()
{
// Example base64 encoded string
string encodedString = "SGVsbG8gV29ybGQh";

byte[] decodedBytes = Convert.FromBase64String(encodedString);
var result = new Dictionary<string, object>();
result.Add("decodedBytes", decodedBytes);

return result;
}

public static Dictionary<string, object> PreviewUint32Dictionary()
{
var uintArray = new uint[] { 1, 2, 3, 4, 5 };
var result = new Dictionary<string, object>();
result.Add("uint32List", uintArray);

return result;
}

public static Dictionary<string, object> PreviewUint64Dictionary()
{
var uintArray = new ulong[] { 1, 2, 3, 4, 5 };
var result = new Dictionary<string, object>();
result.Add("uint64List", uintArray);

return result;
}

public static Dictionary<string, object> PreviewJSONDictionary()
{
var json = new JObject();
json.Add("key1", "value1");
json.Add("key2", "value2");
var result = new Dictionary<string, object>();
result.Add("json", json);

return result;
}
}
}
150 changes: 150 additions & 0 deletions test/core/watch/WatchDictionaryByteValuesDisplaysCorrectly.dyn
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
{
"Uuid": "8a5b5d45-b86d-4b1f-9d04-bec0e4fefd1c",
"IsCustomNode": false,
"Description": "",
"Name": "WatchDictionaryByteValuesDisplaysCorrectly",
"ElementResolver": {
"ResolutionMap": {}
},
"Inputs": [],
"Outputs": [],
"Nodes": [
{
"ConcreteType": "CoreNodeModels.Watch, CoreNodeModels",
"WatchWidth": 200.0,
"WatchHeight": 200.0,
"Id": "e72bcbe3c45044399237b0f63ff261cd",
"NodeType": "ExtensionNode",
"Inputs": [
{
"Id": "3f76a802ad484a90b7a466783baad45a",
"Name": "",
"Description": "Node to show output from",
"UsingDefaultValue": false,
"Level": 2,
"UseLevels": false,
"KeepListStructure": false
}
],
"Outputs": [
{
"Id": "fa2127ef2e864999aae8a00dcc13489f",
"Name": "",
"Description": "Node output",
"UsingDefaultValue": false,
"Level": 2,
"UseLevels": false,
"KeepListStructure": false
}
],
"Replication": "Disabled",
"Description": "Visualizes a node's output"
},
{
"ConcreteType": "Dynamo.Graph.Nodes.ZeroTouch.DSFunction, DynamoCore",
"Id": "95889388fba84e11bc8905dcf80c10ea",
"NodeType": "FunctionNode",
"Inputs": [],
"Outputs": [
{
"Id": "19ff13f87d5d4ac28736af66e0a80e0b",
"Name": "var[]..[]",
"Description": "var[]..[]",
"UsingDefaultValue": false,
"Level": 2,
"UseLevels": false,
"KeepListStructure": false
}
],
"FunctionSignature": "FFITarget.DummyZeroTouchClass.PreviewByteDictionary",
"Replication": "Auto",
"Description": "DummyZeroTouchClass.PreviewByteDictionary ( ): var[]..[]"
}
],
"Connectors": [
{
"Start": "19ff13f87d5d4ac28736af66e0a80e0b",
"End": "3f76a802ad484a90b7a466783baad45a",
"Id": "35653deefde54d4297e35836f25c4282",
"IsHidden": "False"
}
],
"Dependencies": [],
"NodeLibraryDependencies": [
{
"Name": "FFITarget.dll",
"ReferenceType": "ZeroTouch",
"Nodes": [
"95889388fba84e11bc8905dcf80c10ea"
]
}
],
"EnableLegacyPolyCurveBehavior": true,
"Thumbnail": "",
"GraphDocumentationURL": null,
"ExtensionWorkspaceData": [
{
"ExtensionGuid": "28992e1d-abb9-417f-8b1b-05e053bee670",
"Name": "Properties",
"Version": "3.5",
"Data": {}
}
],
"Author": "",
"Linting": {
"activeLinter": "None",
"activeLinterId": "7b75fb44-43fd-4631-a878-29f4d5d8399a",
"warningCount": 0,
"errorCount": 0
},
"Bindings": [],
"View": {
"Dynamo": {
"ScaleFactor": 1.0,
"HasRunWithoutCrash": true,
"IsVisibleInDynamoLibrary": true,
"Version": "3.5.0.6885",
"RunType": "Automatic",
"RunPeriod": "1000"
},
"Camera": {
"Name": "_Background Preview",
"EyeX": -17.0,
"EyeY": 24.0,
"EyeZ": 50.0,
"LookX": 12.0,
"LookY": -13.0,
"LookZ": -58.0,
"UpX": 0.0,
"UpY": 1.0,
"UpZ": 0.0
},
"ConnectorPins": [],
"NodeViews": [
{
"Id": "e72bcbe3c45044399237b0f63ff261cd",
"Name": "Watch",
"IsSetAsInput": false,
"IsSetAsOutput": false,
"Excluded": false,
"ShowGeometry": true,
"X": 558.0000000000003,
"Y": 134.0
},
{
"Id": "95889388fba84e11bc8905dcf80c10ea",
"Name": "DummyZeroTouchClass.PreviewByteDictionary",
"IsSetAsInput": false,
"IsSetAsOutput": false,
"Excluded": false,
"ShowGeometry": true,
"X": 78.00000000000006,
"Y": 363.2
}
],
"Annotations": [],
"X": 0.0,
"Y": 0.0,
"Zoom": 1.0
}
}
Loading
Loading