feat: [aiplatform] add protected_artifact_location_id to CustomJob (#…

…4809) * feat: add protected_artifact_location_id to CustomJob chore: add comment fixes to Feature and FeaturestoreService feat!: add FeatureViewDataFormat, FeatureViewDataKey to FeatureOnlineStoreService feat: add NumericRestriction to Index feat: add tpu_topology to MachineResources feat: add BatchDeletePipelineJobs, BatchDeletePipelineJobsRequest and BatchDeletePipelineJobsResponse to PipelineService feat: add StudyTimeConstraint to Study PiperOrigin-RevId: 581306075 Source-Link: googleapis/googleapis@7603960 Source-Link: googleapis/googleapis-gen@9d03dc6 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjlkMDNkYzY2MWJjMmI2NzIwMjliNmNjNjQwYTRkN2YzNTYxNDY2ZTcifQ== * feat: add protected_artifact_location_id to CustomJob feat: add value_type, version_column_name to Feature feat: add FeatureGroup, FeatureOnlineStore, FeatureOnlineStoreAdminService, FeatureOnlineStoreService, FeatureRegistryService, FeatureView, FeatureViewSync feat: add NumericRestriction to Index feat: add tpu_topology to MachineSpec chore: change DeleteOperationMetadata to DeleteModelVersionOperationMetadata in ModelService feat: add CountTokensRequest to PredictionService feat: add StudyTimeConstraint to Study PiperOrigin-RevId: 581306827 Source-Link: googleapis/googleapis@01af66a Source-Link: googleapis/googleapis-gen@842299d Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6Ijg0MjI5OWRlYzdhN2Y1ZDc2MjJiMmE5MThhYWExODg4ODEyZDgwZGMifQ== * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add ComputeTokens and CountTokens API PiperOrigin-RevId: 582364082 Source-Link: googleapis/googleapis@fdea53f Source-Link: googleapis/googleapis-gen@295f8ea Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjI5NWY4ZWFiMmE4Nzc0M2RjY2M2NWI4OGU3M2Q3MWY0ODAzMjMwMjMifQ== * feat: add deployment_timeout to UploadModel ModelContainerSpec feat: add shared_memory_size_mb to UploadModel ModelContainerSpec feat: add startup_probe, health_probe to UploadModel ModelContainerSpec PiperOrigin-RevId: 582381401 Source-Link: googleapis/googleapis@92d962d Source-Link: googleapis/googleapis-gen@c19c7a7 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6ImMxOWM3YTdmYTNlYWM4NTUwOGEyZmUwYWZlMTc3ZDc2ZjUxMGZjYWYifQ== * feat: add deployment_timeout to UploadModel ModelContainerSpec feat: add shared_memory_size_mb to UploadModel ModelContainerSpec feat: add startup_probe, health_probe to UploadModel ModelContainerSpec PiperOrigin-RevId: 582390619 Source-Link: googleapis/googleapis@49c44ef Source-Link: googleapis/googleapis-gen@7a1c0d1 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjdhMWMwZDE1YjgyZTA2MjY3OGE1MTZkMTg1NTFjNjdjMWIwMTc2YWEifQ== * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * feat: add ComputeTokens API PiperOrigin-RevId: 582473450 Source-Link: googleapis/googleapis@a9e4e72 Source-Link: googleapis/googleapis-gen@56ea7e1 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjU2ZWE3ZTFjMDY3ZWVkNTYyNTZiYTYzMTBmNDRkNmI5NGM2Nzc0OGEifQ== * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * build: update Node.js generator to compile protos PiperOrigin-RevId: 582493526 Source-Link: googleapis/googleapis@7c4e4b5 Source-Link: googleapis/googleapis-gen@368cfb6 Copy-Tag: eyJwIjoicGFja2FnZXMvZ29vZ2xlLWNsb3VkLWFpcGxhdGZvcm0vLk93bEJvdC55YW1sIiwiaCI6IjM2OGNmYjY1MTAxNmQ2YTkzY2E2ZTQ4OGNiYzM0ZTJkMWQ5ZDIxMmMifQ== * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md * Update index.ts * Update index.ts * Update index.ts * Update index.ts * 🦉 Updates from OwlBot post-processor See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md --------- Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com> Co-authored-by: sofisl <[email protected]>
googleapis · Nov 15, 2023 · 9da9501 · 9da9501
1 parent 4eae645
commit 9da9501
Show file tree

Hide file tree

Showing 204 changed files with 165,478 additions and 56,531 deletions.
diff --git a/packages/google-cloud-aiplatform/README.md b/packages/google-cloud-aiplatform/README.md
diff --git a/packages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/custom_job.proto b/packages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/custom_job.proto
@@ -181,6 +181,12 @@ message CustomJobSpec {
   //   * AIP_TENSORBOARD_LOG_DIR = `<base_output_directory>/<trial_id>/logs/`
   GcsDestination base_output_directory = 6;
 
+  // The ID of the location to store protected artifacts. e.g. us-central1.
+  // Populate only when the location is different than CustomJob location.
+  // List of supported locations:
+  // https://cloud.google.com/vertex-ai/docs/general/locations
+  string protected_artifact_location_id = 19;
+
   // Optional. The name of a Vertex AI
   // [Tensorboard][google.cloud.aiplatform.v1.Tensorboard] resource to which
   // this CustomJob will upload Tensorboard logs. Format:

diff --git a/packages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/feature.proto b/packages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/feature.proto
@@ -29,13 +29,15 @@ option java_package = "com.google.cloud.aiplatform.v1";
 option php_namespace = "Google\\Cloud\\AIPlatform\\V1";
 option ruby_package = "Google::Cloud::AIPlatform::V1";
 
-// Feature Metadata information that describes an attribute of an entity type.
-// For example, apple is an entity type, and color is a feature that describes
-// apple.
+// Feature Metadata information.
+// For example, color is a feature that describes an apple.
 message Feature {
   option (google.api.resource) = {
     type: "aiplatform.googleapis.com/Feature"
     pattern: "projects/{project}/locations/{location}/featurestores/{featurestore}/entityTypes/{entity_type}/features/{feature}"
+    pattern: "projects/{project}/locations/{location}/featureGroups/{feature_group}/features/{feature}"
+    plural: "features"
+    singular: "feature"
   };
 
   // A list of historical
@@ -69,6 +71,7 @@ message Feature {
         [(google.api.field_behavior) = OUTPUT_ONLY];
   }
 
+  // Only applicable for Vertex AI Legacy Feature Store.
   // An enum representing the value type of a feature.
   enum ValueType {
     // The value type is unspecified.
@@ -105,6 +108,7 @@ message Feature {
   // Immutable. Name of the Feature.
   // Format:
   // `projects/{project}/locations/{location}/featurestores/{featurestore}/entityTypes/{entity_type}/features/{feature}`
+  // `projects/{project}/locations/{location}/featureGroups/{feature_group}/features/{feature}`
   //
   // The last part feature is assigned by the client. The feature can be up to
   // 64 characters long and can consist only of ASCII Latin letters A-Z and a-z,
@@ -115,17 +119,17 @@ message Feature {
   // Description of the Feature.
   string description = 2;
 
-  // Required. Immutable. Type of Feature value.
-  ValueType value_type = 3 [
-    (google.api.field_behavior) = REQUIRED,
-    (google.api.field_behavior) = IMMUTABLE
-  ];
+  // Immutable. Only applicable for Vertex AI Feature Store (Legacy).
+  // Type of Feature value.
+  ValueType value_type = 3 [(google.api.field_behavior) = IMMUTABLE];
 
-  // Output only. Timestamp when this EntityType was created.
+  // Output only. Only applicable for Vertex AI Feature Store (Legacy).
+  // Timestamp when this EntityType was created.
   google.protobuf.Timestamp create_time = 4
       [(google.api.field_behavior) = OUTPUT_ONLY];
 
-  // Output only. Timestamp when this EntityType was most recently updated.
+  // Output only. Only applicable for Vertex AI Feature Store (Legacy).
+  // Timestamp when this EntityType was most recently updated.
   google.protobuf.Timestamp update_time = 5
       [(google.api.field_behavior) = OUTPUT_ONLY];
 
@@ -146,17 +150,24 @@ message Feature {
   // "overwrite" update happens.
   string etag = 7;
 
-  // Optional. If not set, use the monitoring_config defined for the EntityType
-  // this Feature belongs to. Only Features with type
+  // Optional. Only applicable for Vertex AI Feature Store (Legacy).
+  // If not set, use the monitoring_config defined for the EntityType this
+  // Feature belongs to.
+  // Only Features with type
   // ([Feature.ValueType][google.cloud.aiplatform.v1.Feature.ValueType]) BOOL,
   // STRING, DOUBLE or INT64 can enable monitoring.
   //
   // If set to true, all types of data monitoring are disabled despite the
   // config on EntityType.
   bool disable_monitoring = 12 [(google.api.field_behavior) = OPTIONAL];
 
-  // Output only. The list of historical stats and anomalies with specified
-  // objectives.
+  // Output only. Only applicable for Vertex AI Feature Store (Legacy).
+  // The list of historical stats and anomalies with specified objectives.
   repeated MonitoringStatsAnomaly monitoring_stats_anomalies = 11
       [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Only applicable for Vertex AI Feature Store.
+  // The name of the BigQuery Table/View columnn hosting data for this version.
+  // If no value is provided, will use feature_id.
+  string version_column_name = 106;
 }
diff --git a/packages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/feature_group.proto b/packages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/feature_group.proto
@@ -0,0 +1,95 @@
+// Copyright 2023 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package google.cloud.aiplatform.v1;
+
+import "google/api/field_behavior.proto";
+import "google/api/resource.proto";
+import "google/cloud/aiplatform/v1/io.proto";
+import "google/protobuf/timestamp.proto";
+
+option csharp_namespace = "Google.Cloud.AIPlatform.V1";
+option go_package = "cloud.google.com/go/aiplatform/apiv1/aiplatformpb;aiplatformpb";
+option java_multiple_files = true;
+option java_outer_classname = "FeatureGroupProto";
+option java_package = "com.google.cloud.aiplatform.v1";
+option php_namespace = "Google\\Cloud\\AIPlatform\\V1";
+option ruby_package = "Google::Cloud::AIPlatform::V1";
+
+// Vertex AI Feature Group.
+message FeatureGroup {
+  option (google.api.resource) = {
+    type: "aiplatform.googleapis.com/FeatureGroup"
+    pattern: "projects/{project}/locations/{location}/featureGroups/{feature_group}"
+    plural: "featureGroups"
+    singular: "featureGroup"
+  };
+
+  // Input source type for BigQuery Tables and Views.
+  message BigQuery {
+    // Required. Immutable. The BigQuery source URI that points to either a
+    // BigQuery Table or View.
+    BigQuerySource big_query_source = 1 [
+      (google.api.field_behavior) = IMMUTABLE,
+      (google.api.field_behavior) = REQUIRED
+    ];
+
+    // Optional. Columns to construct entity_id / row keys. Currently only
+    // supports 1 entity_id_column. If not provided defaults to `entity_id`.
+    repeated string entity_id_columns = 2
+        [(google.api.field_behavior) = OPTIONAL];
+  }
+
+  oneof source {
+    // Indicates that features for this group come from BigQuery Table/View.
+    // By default treats the source as a sparse time series source, which is
+    // required to have an entity_id and a feature_timestamp column in the
+    // source.
+    BigQuery big_query = 7;
+  }
+
+  // Output only. Name of the FeatureGroup. Format:
+  // `projects/{project}/locations/{location}/featureGroups/{featureGroup}`
+  string name = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. Timestamp when this FeatureGroup was created.
+  google.protobuf.Timestamp create_time = 2
+      [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. Timestamp when this FeatureGroup was last updated.
+  google.protobuf.Timestamp update_time = 3
+      [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Optional. Used to perform consistent read-modify-write updates. If not set,
+  // a blind "overwrite" update happens.
+  string etag = 4 [(google.api.field_behavior) = OPTIONAL];
+
+  // Optional. The labels with user-defined metadata to organize your
+  // FeatureGroup.
+  //
+  // Label keys and values can be no longer than 64 characters
+  // (Unicode codepoints), can only contain lowercase letters, numeric
+  // characters, underscores and dashes. International characters are allowed.
+  //
+  // See https://goo.gl/xmQnxf for more information on and examples of labels.
+  // No more than 64 user labels can be associated with one
+  // FeatureGroup(System labels are excluded)." System reserved label keys
+  // are prefixed with "aiplatform.googleapis.com/" and are immutable.
+  map<string, string> labels = 5 [(google.api.field_behavior) = OPTIONAL];
+
+  // Optional. Description of the FeatureGroup.
+  string description = 6 [(google.api.field_behavior) = OPTIONAL];
+}
diff --git a/...ages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/feature_online_store.proto b/...ages/google-cloud-aiplatform/protos/google/cloud/aiplatform/v1/feature_online_store.proto
@@ -0,0 +1,118 @@
+// Copyright 2023 Google LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+package google.cloud.aiplatform.v1;
+
+import "google/api/field_behavior.proto";
+import "google/api/resource.proto";
+import "google/protobuf/timestamp.proto";
+
+option csharp_namespace = "Google.Cloud.AIPlatform.V1";
+option go_package = "cloud.google.com/go/aiplatform/apiv1/aiplatformpb;aiplatformpb";
+option java_multiple_files = true;
+option java_outer_classname = "FeatureOnlineStoreProto";
+option java_package = "com.google.cloud.aiplatform.v1";
+option php_namespace = "Google\\Cloud\\AIPlatform\\V1";
+option ruby_package = "Google::Cloud::AIPlatform::V1";
+
+// Vertex AI Feature Online Store provides a centralized repository for serving
+// ML features and embedding indexes at low latency. The Feature Online Store is
+// a top-level container.
+message FeatureOnlineStore {
+  option (google.api.resource) = {
+    type: "aiplatform.googleapis.com/FeatureOnlineStore"
+    pattern: "projects/{project}/locations/{location}/featureOnlineStores/{feature_online_store}"
+  };
+
+  message Bigtable {
+    message AutoScaling {
+      // Required. The minimum number of nodes to scale down to. Must be greater
+      // than or equal to 1.
+      int32 min_node_count = 1 [(google.api.field_behavior) = REQUIRED];
+
+      // Required. The maximum number of nodes to scale up to. Must be greater
+      // than or equal to min_node_count, and less than or equal to 10 times of
+      // 'min_node_count'.
+      int32 max_node_count = 2 [(google.api.field_behavior) = REQUIRED];
+
+      // Optional. A percentage of the cluster's CPU capacity. Can be from 10%
+      // to 80%. When a cluster's CPU utilization exceeds the target that you
+      // have set, Bigtable immediately adds nodes to the cluster. When CPU
+      // utilization is substantially lower than the target, Bigtable removes
+      // nodes. If not set will default to 50%.
+      int32 cpu_utilization_target = 3 [(google.api.field_behavior) = OPTIONAL];
+    }
+
+    // Required. Autoscaling config applied to Bigtable Instance.
+    AutoScaling auto_scaling = 1 [(google.api.field_behavior) = REQUIRED];
+  }
+
+  // Possible states a featureOnlineStore can have.
+  enum State {
+    // Default value. This value is unused.
+    STATE_UNSPECIFIED = 0;
+
+    // State when the featureOnlineStore configuration is not being updated and
+    // the fields reflect the current configuration of the featureOnlineStore.
+    // The featureOnlineStore is usable in this state.
+    STABLE = 1;
+
+    // The state of the featureOnlineStore configuration when it is being
+    // updated. During an update, the fields reflect either the original
+    // configuration or the updated configuration of the featureOnlineStore. The
+    // featureOnlineStore is still usable in this state.
+    UPDATING = 2;
+  }
+
+  oneof storage_type {
+    // Contains settings for the Cloud Bigtable instance that will be created
+    // to serve featureValues for all FeatureViews under this
+    // FeatureOnlineStore.
+    Bigtable bigtable = 8;
+  }
+
+  // Output only. Name of the FeatureOnlineStore. Format:
+  // `projects/{project}/locations/{location}/featureOnlineStores/{featureOnlineStore}`
+  string name = 1 [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. Timestamp when this FeatureOnlineStore was created.
+  google.protobuf.Timestamp create_time = 3
+      [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Output only. Timestamp when this FeatureOnlineStore was last updated.
+  google.protobuf.Timestamp update_time = 4
+      [(google.api.field_behavior) = OUTPUT_ONLY];
+
+  // Optional. Used to perform consistent read-modify-write updates. If not set,
+  // a blind "overwrite" update happens.
+  string etag = 5 [(google.api.field_behavior) = OPTIONAL];
+
+  // Optional. The labels with user-defined metadata to organize your
+  // FeatureOnlineStore.
+  //
+  // Label keys and values can be no longer than 64 characters
+  // (Unicode codepoints), can only contain lowercase letters, numeric
+  // characters, underscores and dashes. International characters are allowed.
+  //
+  // See https://goo.gl/xmQnxf for more information on and examples of labels.
+  // No more than 64 user labels can be associated with one
+  // FeatureOnlineStore(System labels are excluded)." System reserved label keys
+  // are prefixed with "aiplatform.googleapis.com/" and are immutable.
+  map<string, string> labels = 6 [(google.api.field_behavior) = OPTIONAL];
+
+  // Output only. State of the featureOnlineStore.
+  State state = 7 [(google.api.field_behavior) = OUTPUT_ONLY];
+}