Skip to content

Commit

Permalink
Demo Gen Organized Output (#764)
Browse files Browse the repository at this point in the history
* Starting on let = syntax

* Start on new setup for call stack

* Add TODO

* Started on node call stack

* Add another TODO

* Remove extra semicolon

* Adding parameters to function calls

* Add self parameter calling

* Adding struct calls

* Generation

* Formatting

* Give out_param a return

* Started on naming parameters consistently

* Add TODO

* Generation

* Formatting

* Avoid duplication of owned types, need to add back let = recursive names

* Generation

* Formatting

* Clippy

* Add variable naming back in

* Generation

* Formatting

* Update tests to check correct naming conventions

* Formatting

* Update naming check test

* Remove cyclic setup in parameter field name

* Generation

* Wrong Self param name

* Add test for variable name collisions

* Add detection of variable name collisions

* Update tests

* Formatting

* Generation

* Add test for getters

* Add getter support

* Generation

* Remove TODOs

* Formatting

* Updating docs

* Formatting
ambiguousname authored Jan 22, 2025
1 parent ad2ec60 commit a80a6a0
Showing 37 changed files with 561 additions and 398 deletions.
79 changes: 27 additions & 52 deletions docs/demo_gen.md
Original file line number Diff line number Diff line change
@@ -120,6 +120,8 @@ From there, we need to look at each Termini's parameters. Our ultimate goal for

So we have what's called a `MethodDependency` struct, to represent dependencies for any given method. Let's explore how each possible parameter type impacts our `MethodDependency`s fields.

Once we've fully evaluated a `MethodDependency`, we render the call of that method to Javascript and push it to a stack of similar Javascript calls, each dependent on the prior `MethodDependency`. This way, we convert the tree into a stack of calls.

The HIR splits parameters into a few different types:

##### 1. Primitive Types
@@ -148,22 +150,20 @@ The nice thing about Javascript is that we can evaluate a parameter as a functio

##### 5. Structs

Structs are a little different because they can be created through pure JS. So we treat each Struct as a `MethodDependency`, when in fact the "method" is just a function that assigns fields to the struct sequentially:
Structs are a little different because they can be created through pure JS. So we just use the helpful `FromFields` function that comes with every non-out struct[^out]:

```js
function (a, b, c, d) {
return new Struct(a, b, c, d);
}
Struct.FromFields(a, b, c, d);
```

[^out]: Out Structs cannot be generated from a simple function call, so demo_gen finds these as the result of other functions.

#### Step Two: Constructing RenderInfo

Now that we have a tree[^tree] of `MethodDependency`s, we can now create two items:
Now that we have a call stack of `MethodDependency`s, we can now create two items:
1. A Javascript function for our [renderer](#front-end-renderer) to call.
2. A JSON object for our [renderer](#front-end-renderer) to evaluate to know what Render Termini exist.

[^tree]: This tree only exists on the call stack. As we find each `MethodDependency`, it is quickly turned into Javascript and then passed up the call stack.

In this case, we just export an object called `RenderInfo`, with all the information any renderer will need to know about how we expect our Render Terminus to be called in Javascript.

The layout is something like:
@@ -252,53 +252,28 @@ pub fn new_static() -> Box<DataProvider> { /* ... */ }

We only have a static data provider in this case, but ICU4X has more complicated data providers in the real library. This is where we might use something like `#[diplomat::demo(external)]` to tell demo_gen that we would like to be able to provide `DataProvider` ourselves when calling `formatWrite` (see [attributes](#attributes) for more).

And so we continue to recurse through all opaque methods until we finally get the following JS output of all the recursive calls:
And so we continue to recurse through all opaque methods until we finally get the following stack of JS calls:

```js
export function formatWrite() {
var terminusArgs = arguments;
return (function (...args) { return args[0].formatWrite(...args.slice(1)) }).apply(
null,
[
FixedDecimalFormatter.tryNew.apply(
null,
[
Locale.new_.apply(
null,
[
terminusArgs[0]
]
),
DataProvider.newStatic.apply(
null,
[
]
),
(function (...args) {
let out = new FixedDecimalFormatterOptions();

out.groupingStrategy = args[0];

out.someOtherConfig = args[1];

return out;
}).apply(
null,
[
terminusArgs[1],
terminusArgs[2]
]
)
]
),
FixedDecimal.new_.apply(
null,
[
terminusArgs[3]
]
)
]
);
export function formatWrite(fixedDecimalFormatterLocaleName, fixedDecimalFormatterOptionsGroupingStrategy, fixedDecimalFormatterOptionsSomeOtherConfig, valueV) {

let fixedDecimalFormatterLocale = new Locale(fixedDecimalFormatterLocaleName);

let fixedDecimalFormatterProvider = DataProvider.newStatic();

let fixedDecimalFormatterOptions = FixedDecimalFormatterOptions.fromFields({
groupingStrategy: fixedDecimalFormatterOptionsGroupingStrategy,
someOtherConfig: fixedDecimalFormatterOptionsSomeOtherConfig
});

let fixedDecimalFormatter = FixedDecimalFormatter.tryNew(fixedDecimalFormatterLocale,fixedDecimalFormatterProvider,fixedDecimalFormatterOptions);

let value = new FixedDecimal(valueV);

let out = fixedDecimalFormatter.formatWrite(value);


return out;
}
```

2 changes: 1 addition & 1 deletion example/demo_gen/demo/FixedDecimal.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 8 additions & 12 deletions example/demo_gen/demo/FixedDecimal.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion example/demo_gen/demo/FixedDecimalFormatter.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

58 changes: 19 additions & 39 deletions example/demo_gen/demo/FixedDecimalFormatter.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion example/demo_gen/demo/index.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions example/demo_gen/test/test-demo.mjs
Original file line number Diff line number Diff line change
@@ -13,4 +13,9 @@ test("Test FixedDecimalFormatter", (t) => {

test("Custom Function", (t) => {
t.is(RenderInfo.termini["FixedDecimal.multiplyPow10"].func(3), "10000");
});

test("Variable Names", (t) => {
// Can't exactly check variable names without reading the file, but RenderInfo re-uses the same info, so we check that instead.
t.is(RenderInfo.termini["FixedDecimalFormatter.formatWrite"].parameters[0].name, "FixedDecimalFormatter:Locale:Name");
});
4 changes: 1 addition & 3 deletions example/src/locale.rs
Original file line number Diff line number Diff line change
@@ -10,9 +10,7 @@ pub mod ffi {
impl Locale {
/// Construct an [`Locale`] from a locale identifier represented as a string.
#[diplomat::attr(auto, constructor)]
pub fn new(
#[diplomat::demo(input(label = "Locale Name"))] name: &DiplomatStr,
) -> Box<Locale> {
pub fn new(name: &DiplomatStr) -> Box<Locale> {
Box::new(Locale(icu::locid::Locale::try_from_bytes(name).unwrap()))
}
}
4 changes: 4 additions & 0 deletions feature_tests/c/include/CyclicStructA.h

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 4 additions & 0 deletions feature_tests/cpp/include/CyclicStructA.d.hpp

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 21 additions & 0 deletions feature_tests/cpp/include/CyclicStructA.hpp

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

24 changes: 24 additions & 0 deletions feature_tests/dart/lib/src/CyclicStructA.g.dart

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 3 additions & 1 deletion feature_tests/demo_gen/demo/CyclicStructA.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 52 additions & 23 deletions feature_tests/demo_gen/demo/CyclicStructA.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion feature_tests/demo_gen/demo/CyclicStructC.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

49 changes: 18 additions & 31 deletions feature_tests/demo_gen/demo/CyclicStructC.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion feature_tests/demo_gen/demo/Float64Vec.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 8 additions & 12 deletions feature_tests/demo_gen/demo/Float64Vec.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 7 additions & 10 deletions feature_tests/demo_gen/demo/MyOpaqueEnum.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion feature_tests/demo_gen/demo/MyString.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

31 changes: 13 additions & 18 deletions feature_tests/demo_gen/demo/MyString.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 7 additions & 10 deletions feature_tests/demo_gen/demo/Opaque.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion feature_tests/demo_gen/demo/OptionString.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 8 additions & 12 deletions feature_tests/demo_gen/demo/OptionString.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion feature_tests/demo_gen/demo/Utf16Wrap.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

20 changes: 8 additions & 12 deletions feature_tests/demo_gen/demo/Utf16Wrap.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

48 changes: 42 additions & 6 deletions feature_tests/demo_gen/demo/index.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 15 additions & 2 deletions feature_tests/demo_gen/test/test.mjs
Original file line number Diff line number Diff line change
@@ -6,8 +6,21 @@ test("My String", (t) => {
})

test("Cyclic Parameters", (t) => {
t.is(RenderInfo.termini["CyclicStructA.cyclicOut"].parameters[0].name, "Self:A:Field");
t.is(RenderInfo.termini["CyclicStructC.cyclicOut"].parameters[0].name, "Self:A:A:Field");
t.is(RenderInfo.termini["CyclicStructA.cyclicOut"].parameters[0].name, "CyclicStructA:A:Field");
t.is(RenderInfo.termini["CyclicStructC.cyclicOut"].parameters[0].name, "CyclicStructC:A:A:Field");
t.is(CyclicStructADemo.cyclicOut(10), "10");
t.is(CyclicStructCDemo.cyclicOut(15), "15");
});

test("Variable Name Collisions", (t) => {
// Make sure that the names for the parameters are the same so that we're assured some collisions.
// As long as this test passes and the Javascript compiles, we should be good to go.
t.is(RenderInfo.termini["CyclicStructA.doubleCyclicOut"].parameters[0].name, "CyclicStructA:A:Field");
t.is(RenderInfo.termini["CyclicStructA.doubleCyclicOut"].parameters[1].name, "CyclicStructA:A:Field");

t.is(CyclicStructADemo.doubleCyclicOut(10, 20), "10 20");
});

test("Getter", (t) => {
t.is(CyclicStructADemo.getterOut(10), "10");
});
4 changes: 4 additions & 0 deletions feature_tests/js/api/CyclicStructA.d.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

34 changes: 34 additions & 0 deletions feature_tests/js/api/CyclicStructA.mjs

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions feature_tests/src/structs.rs
Original file line number Diff line number Diff line change
@@ -267,6 +267,20 @@ pub mod ffi {
pub fn cyclic_out(self, out: &mut DiplomatWrite) {
out.write_str(&self.a.field.to_string()).unwrap();
}

// For demo gen: tests having the same variables in the namespace
pub fn double_cyclic_out(self, cyclic_struct_a: Self, out: &mut DiplomatWrite) {
out.write_fmt(format_args!(
"{} {}",
&self.a.field, cyclic_struct_a.a.field
))
.unwrap();
}

#[diplomat::attr(auto, getter)]
pub fn getter_out(self, out: &mut DiplomatWrite) {
out.write_str(&self.a.field.to_string()).unwrap();
}
}

impl CyclicStructB {
4 changes: 2 additions & 2 deletions tool/src/demo_gen/mod.rs
Original file line number Diff line number Diff line change
@@ -196,15 +196,15 @@ pub(crate) fn run<'tcx>(

js_file_name: js_file_name.clone(),

node_call_stack: String::default(),
node_call_stack: Vec::default(),

// We set this in the init function of WebDemoGenerationContext.
typescript: false,

imports: BTreeSet::new(),
},

out_param_collision: HashMap::new(),
name_collision: HashMap::new(),

relative_import_path: import_path.clone(),
module_name: module_name.clone(),
301 changes: 172 additions & 129 deletions tool/src/demo_gen/terminus.rs

Large diffs are not rendered by default.

13 changes: 5 additions & 8 deletions tool/templates/demo_gen/method_dependency.js.jinja
Original file line number Diff line number Diff line change
@@ -1,8 +1,5 @@
{{ method_js }}.apply(
null,
[
{%- for param in params %}
{{ param.js|indent(8) }}{% if !loop.last %},{% endif -%}
{% endfor %}
]
)
let {{variable_name}} = {% if let Some(s) = self_param -%} {{s}}. {%- endif -%} {{ method_js }} {%- if !is_getter -%} (
{%- for param in params -%}
{{ param }}{% if !loop.last %},{% endif -%}
{%- endfor -%}
) {%- endif %}
12 changes: 5 additions & 7 deletions tool/templates/demo_gen/struct.js.jinja
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
(function (...args) {
return {{type_name}}.fromFields({
{%- for field in fields %}
{{field}}: args[{{loop.index0}}]{% if !loop.last %},{% endif %}
{%- endfor -%}
});
})
{
{%- for field in fields %}
{{field.field_name}}: {{field.param_name}}{% if !loop.last %},{% endif %}
{%- endfor %}
}
6 changes: 5 additions & 1 deletion tool/templates/demo_gen/terminus.js.jinja
Original file line number Diff line number Diff line change
@@ -3,6 +3,10 @@ export function {{function_name}}(
{{param.param_name}}{% if typescript %}: {{param.type_name}}{% endif %}{% if !loop.last %}, {% endif %}
{%- endfor -%}
){% if typescript %};{% else %} {
return {{node_call_stack|indent(4)}};
{% for method in node_call_stack %}
{{ method|indent(4) }};
{% endfor %}

return out;
}
{%- endif %}

0 comments on commit a80a6a0

Please sign in to comment.