Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Support UDFs management in Gravitino #6158

Open
xloya opened this issue Jan 9, 2025 · 1 comment
Open

[FEATURE] Support UDFs management in Gravitino #6158

xloya opened this issue Jan 9, 2025 · 1 comment
Labels
feature New feature or request

Comments

@xloya
Copy link
Contributor

xloya commented Jan 9, 2025

Describe the feature

In enterprise production practice, if the built-in functions of the computing engine cannot meet user needs, users usually customize UDFs(User-Defined-Functions) through relevant interfaces and reference them in the engine. Especially for scenarios that require a lot of reuse, it is very important to effectively manage these custom functions. In Xiaomi's internal practice, there is also a UDF management system that basically meets user needs (limited to Spark/Flink, and there is no good support for AI + Python scenarios).
As a unified Data / AI metadata center, I think Gravitino can also support UDFs management in Data / AI scenarios.
The following are some commercial product practices:

  1. Aliyun: https://help.aliyun.com/zh/flink/user-guide/manage-udfs
  2. UnityCatalog: https://docs.databricks.com/en/udf/unity-catalog.html

Motivation

No response

Describe the solution

No response

Additional context

No response

@xloya xloya added the feature New feature or request label Jan 9, 2025
@photogamerun
Copy link

what's the benifit, only manage the udf is just only manage it, is there other benifit we can gain from this catalog?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants