Skip to content

Commit

Permalink
Introduce kinds backed by asset spec tags (#23779)
Browse files Browse the repository at this point in the history
## Summary

Introduces a `kinds` property to `AssetSpec`, a set of strings representing compute, storage etc kinds, based on dagster-io/internal#10821 (comment). These kinds are backed by specially constructed tags of the form `dagster/kind/python`, `dagster/kind/snowflake` which canonically have empty `""` values.

Right now, we expect these tags to be only set by internal Dagster code (e.g. integrations). A subsequent, stacked PR (#24154) introduces a top-level `kinds` parameter, which is immediately converted to this special tag format.

## Test Plan

Unit tests.
  • Loading branch information
benpankow authored Sep 4, 2024
1 parent 5d53389 commit 9327529
Show file tree
Hide file tree
Showing 15 changed files with 659 additions and 265 deletions.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions js_modules/dagster-ui/packages/ui-core/src/graphql/types.ts

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
)
from dagster._core.snap.node import GraphDefSnap, OpDefSnap
from dagster._core.storage.batch_asset_record_loader import BatchAssetRecordLoader
from dagster._core.storage.tags import KIND_PREFIX
from dagster._core.utils import is_valid_email
from dagster._core.workspace.permissions import Permissions
from dagster._utils.caching_instance_queryer import CachingInstanceQueryer
Expand Down Expand Up @@ -300,6 +301,7 @@ class GrapheneAssetNode(graphene.ObjectType):
partitionStats = graphene.Field(GraphenePartitionStats)
metadata_entries = non_null_list(GrapheneMetadataEntry)
tags = non_null_list(GrapheneDefinitionTag)
kinds = non_null_list(graphene.String)
op = graphene.Field(GrapheneSolidDefinition)
opName = graphene.String()
opNames = non_null_list(graphene.String)
Expand Down Expand Up @@ -1214,6 +1216,16 @@ def resolve_tags(self, _graphene_info: ResolveInfo) -> Sequence[GrapheneDefiniti
for key, value in (self._external_asset_node.tags or {}).items()
]

def resolve_kinds(self, _graphene_info: ResolveInfo) -> Sequence[str]:
if self._external_asset_node.compute_kind:
return [self._external_asset_node.compute_kind]

return [
key[len(KIND_PREFIX) :]
for key in (self._external_asset_node.tags or {}).keys()
if key.startswith(KIND_PREFIX)
]

def resolve_op(
self, _graphene_info: ResolveInfo
) -> Optional[Union[GrapheneSolidDefinition, GrapheneCompositeSolidDefinition]]:
Expand Down
Loading

1 comment on commit 9327529

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Deploy preview for dagit-core-storybook ready!

✅ Preview
https://dagit-core-storybook-hcdvjlpdh-elementl.vercel.app

Built with commit 9327529.
This pull request is being automatically deployed with vercel-action

Please sign in to comment.