Release v1.14.0 (#2180)

Version changelog

1.14.0

• Added caching of SCIM Me API call to databricks_current_user data source and databricks_permissions resource. (#2170).
• Added example for automatically rotating a token (#2135).
• Added run_if condition in databricks_job resource (#2125).
• Added File Arrival trigger to databricks_job resource (#2142).
• Added source parameter for spark_python_task in databricks_job (#2157).
• Automatically add CAN_MANAGE permission on databricks_sql_endpoint for calling user (#2168).
• Deprecated enable_serverless_compute on databricks_sql_global_config resource (#2139).
• Document enable_serverless_compute API changes in databricks_sql_endpoint resource (#2137).
• Fix edge cases for databricks_permissions resource (#2158).
• Migrate databricks_cluster_policy data source to Go SDK (#2155).
• Update the description of ip_addresses parameter in databricks_ip_access_list docs (#2116).

Updated dependency versions:

• Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.25.0 to 2.26.1 (#2140).
• Bump github.com/zclconf/go-cty from 1.13.0 to 1.13.1 (#2124).

Version changelog

1.14.0

• Added caching of SCIM Me API call to databricks_current_user data source and databricks_permissions resource. (#2170).
• Added example for automatically rotating a token (#2135).
• Added run_if condition in databricks_job resource (#2125).
• Added File Arrival trigger to databricks_job resource (#2142).
• Added source parameter for spark_python_task in databricks_job (#2157).
• Automatically add CAN_MANAGE permission on databricks_sql_endpoint for calling user (#2168).
• Deprecated enable_serverless_compute on databricks_sql_global_config resource (#2139).
• Document enable_serverless_compute API changes in databricks_sql_endpoint resource (#2137).
• Fix edge cases for databricks_permissions resource (#2158).
• Migrate databricks_cluster_policy data source to Go SDK (#2155).
• Update the description of ip_addresses parameter in databricks_ip_access_list docs (#2116).

Updated dependency versions:

• Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.25.0 to 2.26.1 (#2140).
• Bump github.com/zclconf/go-cty from 1.13.0 to 1.13.1 (#2124).

Hi team,

An issue occurs when modifying the permissions on the root level of the workspace.

Configuration

resource "databricks_permissions" "root_workspace_permissions" {
  directory_id = "0" # Root folder of the databricks workspace
  access_control {
    group_name       = "AAD_SDP_Data_Engineers"
    permission_level = "CAN_READ"
  }

  access_control {
    group_name       = "AAD_SDP_Data_Operators"
    permission_level = "CAN_READ"
  }
}

Expected Behavior

When running terraform apply, the permissions should set.

Actual Behavior

terraform plan runs successfully. However, terraform apply gives the following error

│ Error: cannot update permissions: Cannot remove admins's CAN_MANAGE permission on 0
│ 
│   with databricks_permissions.root_workspace_permissions,
│   on workspace.tf line 35, in resource "databricks_permissions" "root_workspace_permissions":
│   35: resource "databricks_permissions" "root_workspace_permissions" {
│ 
╵

Steps to Reproduce

1. Run terraform init
2. Run terraform plan (which will run successfully
3. Run terraform apply

Terraform and provider versions

Terraform v1.3.5
on windows_amd64
+ provider registry.terraform.io/databricks/databricks v1.13.0
+ provider registry.terraform.io/hashicorp/azuread v2.33.0
+ provider registry.terraform.io/hashicorp/azurerm v3.43.0

We have SSO on our workspaces and intend to restrict password usage to only admins, as a way to log in should SSO stop working, in order to resolve the issue. When attempting to set this via Terraform, plans fail with Error: It is not possible to restrict any permissions from `admins`

Configuration

resource "databricks_permissions" "password_usage" {
  authorization = "passwords"

  access_control {
    group_name       = "admins"
    permission_level = "CAN_USE"
  }
}

Expected Behavior

The password usage policy should allow setting CAN_USE for the admins group. This is possible if done via the Databricks console

Actual Behavior

 Error: It is not possible to restrict any permissions from `admins`.
│ 
│   with module.workspace_sandbox.databricks_permissions.password_usage,
│   on modules/workspace/permissions.tf line 27, in resource "databricks_permissions" "password_usage":
│   27: resource "databricks_permissions" "password_usage" {
│ 

Steps to Reproduce

Debug Output

[WARN]  provider.terraform-provider-databricks_v1.11.1: Truncating attribute path of 1 diagnostics for TypeSet: timestamp=2023-03-08T12:25:50.547-0500
2023-03-08T12:25:50.547-0500 [ERROR] provider.terraform-provider-databricks_v1.11.1: Response contains error diagnostic: tf_rpc=ValidateResourceTypeConfig @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_attribute=AttributeName("access_control") diagnostic_detail= diagnostic_severity=ERROR tf_resource_type=databricks_permissions diagnostic_summary="It is not possible to restrict any permissions from `admins`." tf_proto_version=5.3 tf_provider_addr=registry.terraform.io/databricks/databricks tf_req_id=0696e93e-9ad4-40c3-8591-9969f2e9bf13 timestamp=2023-03-08T12:25:50.547-0500
2023-03-08T12:25:50.547-0500 [WARN]  provider.terraform-provider-databricks_v1.11.1: Truncating attribute path of 1 diagnostics for TypeSet: timestamp=2023-03-08T12:25:50.547-0500
2023-03-08T12:25:50.547-0500 [WARN]  provider.terraform-provider-databricks_v1.11.1: Truncating attribute path of 1 diagnostics for TypeSet: timestamp=2023-03-08T12:25:50.547-0500
2023-03-08T12:25:50.547-0500 [ERROR] provider.terraform-provider-databricks_v1.11.1: Response contains error diagnostic: diagnostic_attribute=AttributeName("access_control") diagnostic_detail= diagnostic_summary="It is not possible to restrict any permissions from `admins`." tf_req_id=4aaf4edc-bdbf-c6d9-a1ab-e0c29508a361 @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 @module=sdk.proto diagnostic_severity=ERROR tf_proto_version=5.3 tf_rpc=ValidateResourceTypeConfig tf_provider_addr=registry.terraform.io/databricks/databricks tf_resource_type=databricks_permissions timestamp=2023-03-08T12:25:50.547-0500
2023-03-08T12:25:50.547-0500 [ERROR] vertex "module.workspace_sandbox.databricks_permissions.password_usage" error: It is not possible to restrict any permissions from `admins`.
2023-03-08T12:25:50.547-0500 [ERROR] vertex "module.workspace_acm.databricks_permissions.password_usage" error: It is not possible to restrict any permissions from `admins`.
2023-03-08T12:25:50.547-0500 [ERROR] provider.terraform-provider-databricks_v1.11.1: Response contains error diagnostic: diagnostic_severity=ERROR tf_proto_version=5.3 tf_resource_type=databricks_permissions diagnostic_attribute=AttributeName("access_control") tf_provider_addr=registry.terraform.io/databricks/databricks @module=sdk.proto tf_req_id=efc9a692-03b0-0227-3113-eecb12f2f1bd tf_rpc=ValidateResourceTypeConfig @caller=/home/runner/work/terraform-provider-databricks/terraform-provider-databricks/vendor/github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/diag/diagnostics.go:55 diagnostic_detail= diagnostic_summary="It is not possible to restrict any permissions from `admins`." timestamp=2023-03-08T12:25:50.547-0500
2023-03-08T12:25:50.547-0500 [ERROR] vertex "module.workspace_core.databricks_permissions.password_usage" error: It is not possible to restrict any permissions from `admins`.
2023-03-08T12:25:50.592-0500 [DEBUG] provider: starting plugin: path=.terraform/providers/registry.terraform.io/databricks/databricks/1.11.1/darwin_arm64/terraform-provider-databricks_v1.11.1 args=[.terraform/providers/registry.terraform.io/databricks/databricks/1.11.1/darwin_arm64/terraform-provider-databricks_v1.11.1]

Terraform and provider versions

Terraform v1.3.8
on darwin_arm64
+ provider registry.terraform.io/alxrem/jsonnet v2.1.0
+ provider registry.terraform.io/databricks/databricks v1.9.2
+ provider registry.terraform.io/hashicorp/aws v4.50.0
+ provider registry.terraform.io/hashicorp/null v3.2.1

Fix edge cases for databricks_permissions resource (#2158)

Changes

Fix edge cases when setting permissions for root directory and password/token usage

Closes #2087 closes #2134

Tests

• [x] make test run locally
• [x] relevant change in docs/ folder
• [x] covered with integration tests in internal/acceptance
• [x] relevant acceptance tests are passing
• [x] using Go SDK

adding SECURITY.md (#352)

Changes

adding SECURITY.md

Tests

N/A

Changes

adding SECURITY.md

Tests

N/A

adding SECURITY.md (#2177)

Changes

adding SECURITY.md

Tests

N/A

misc doc fixes (#2166)

• misc doc fixes

• one more fix

Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.25.0 to 2.26.1 (#2140)

Bumps github.com/hashicorp/terraform-plugin-sdk/v2 from 2.25.0 to 2.26.1.

• Release notes
• Changelog
• Commits

updated-dependencies:

• dependency-name: github.com/hashicorp/terraform-plugin-sdk/v2 dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Added source parameter for spark_python_task in databricks_job (#2157)

Add caching of scim/me API to databricks_current_user data source and databricks_permissions resource. (#2170)

• Cache scim/me API

• add mutext to protect cached user

• naming fix

• defer mu.unlock and remove unused code

• cache ws client

• inline method and rename

Automatically add CAN_MANAGE permission on databricks_sql_endpoint for calling user (#2168)

• fix sql warehouse permission

• feedback

• update tests

Merge branch 'master' into fix/permissions_edge_case

Automatically add CAN_MANAGE permission on databricks_sql_endpoint for calling user (#2168)

• fix sql warehouse permission

• feedback

• update tests