nfx
Repos
20
Followers
63
Following
19

Databricks Terraform Provider

257
193

Events

[FEATURE] data source `databricks_job`

Hi there!

Similarly to https://github.com/databrickslabs/terraform-provider-databricks/issues/1270 , it is currently impossible to retrieve data about DataBricks Jobs other than merely the id

I propose a new data source which will provide the entire list of attributes for a databricks job.

Happy to work on this as well.

Created at 1 hour ago

Added databricks_job data resource (#1509)

Example: Getting the existing cluster id of specific databricks_job by name or by id:

data "databricks_job" "this" {
  job_name = "My job"
}
output "cluster_id" {
  value     = data.databricks_job.job_settings.existing_cluster_id
  sensitive = false
}

Co-authored-by: Serge Smertin 259697+nfx@users.noreply.github.com

Created at 1 hour ago
Added `databricks_job` data resource

Retrieves the settings of databricks_job by name or by id. Complements the feature of the databricks_jobs data source.

Example Usage

Getting the existing cluster id of specific databricks_job by name or by id:

data "databricks_job" "this" {
  job_name = "My job"
}
output "cluster_id" {
  value     = data.databricks_job.job_settings.existing_cluster_id
  sensitive = false
}

Attribute Reference

This data source exports the following attributes:

Related Resources

The following resources are used in the same context:

Created at 1 hour ago
nfx delete branch feature/add-entitlement-resource
Created at 1 hour ago
nfx delete branch issue-1621
Created at 1 hour ago
nfx delete branch docs/misc_updates
Created at 1 hour ago
nfx delete branch dlt-autoscale-mode
Created at 1 hour ago
nfx delete branch issue-1628
Created at 1 hour ago

Fix dashboard acceptance test (#1618)

DBSQL resources are organized in a workspace structure as of recently. As such it is no longer possible to use the same name for different assets. This acceptance test used to use the same name for both the dashboard and the query and started failing. The solution here is to use a different suffix for the different resources.

Fix unity catalog docs to use correct metastore (#1620)

aws_iam_policy.unity_metastore is referencing aws_s3_bucket.unity_metastore.arn, which is non-existent. Fix to use aws_s3_bucket.metastore.arn instead. This also matches with the online tutorial.

https://docs.databricks.com/data-governance/unity-catalog/automate.html

Bump google.golang.org/api from 0.95.0 to 0.96.0 (#1612)

Bumps google.golang.org/api from 0.95.0 to 0.96.0.


updated-dependencies:

  • dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.22.0 to 2.23.0 (#1611)

Bumps github.com/hashicorp/terraform-plugin-sdk/v2 from 2.22.0 to 2.23.0.


updated-dependencies:

  • dependency-name: github.com/hashicorp/terraform-plugin-sdk/v2 dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

fix secret field name in databricks_mlflow_webhook resource (#1625)

fixes #1621

Doc updates for databricks_table and databricks_mws_vpc_endpoint (#1616)

  • update privatelink guide with autoaccept
  • remove databricks_table resource from doc
  • close #1609
  • fix following suggestions

Fixed gcp_availability field in databricks_insance_pool resource (#1610)

Co-authored-by: Alex Ott alexey.ott@databricks.com

Add autoscale mode configuration to databricks_pipeline resource (#1600)

Fix Azure authentication for dev and staging workspaces (#1607)

  • Fix Azure authentication for dev and staging workspaces
  • use env variable
  • Fix client attributes tests
  • Rename AzureDatabricksResourceId to AzureDatabricksLoginAppId
  • Simplify GetAzureDatabricksLoginAppId

Fixed minor errors in Unity Catalog docs (#1626)

  • Fixed errors in docs
  • Fixed Azure SPOT and elastic disk
  • Added fallback on SPOT_AZURE configuration

Release v1.3.1 (#1629)

Version changelog

1.3.1

Documentation improvements:

Updated dependency versions:

  • Bump github.com/hashicorp/terraform-plugin-sdk/v2 from 2.21.0 to 2.23.0 (#1597, #1611).
  • Bump google.golang.org/api from 0.94.0 to 0.96.0 (#1598, #1612).

Bump github.com/hashicorp/hcl/v2 from 2.14.0 to 2.14.1 (#1634)

Bumps github.com/hashicorp/hcl/v2 from 2.14.0 to 2.14.1.


updated-dependencies:

  • dependency-name: github.com/hashicorp/hcl/v2 dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump google.golang.org/api from 0.96.0 to 0.97.0 (#1633)

Bumps google.golang.org/api from 0.96.0 to 0.97.0.


updated-dependencies:

  • dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Correct handling of references in databricks_permissions resources (#1631)

this fixes #1628

expand documentation for databricks_sql_visualization resource (#1630)

  • expand documentation for options field in sql-visualization resource
  • add wording for how to restore during issue scenario

Prevent stripping of CAN_MANAGE permission from caller of databricks_sql_query resource (#1644)

Make debugging work again (#1560)

Merge branch 'master' into issue-1272

Created at 1 hour ago
nfx delete branch fix-debug
Created at 1 hour ago

Make debugging work again (#1560)

Created at 1 hour ago
Make debugging work again

TODO: test

Created at 1 hour ago
Make debugging work again
Created at 1 hour ago

Use resize API to scale running clusters (#1541)

Release v1.2.1 (#1564)

Add version for gotestsum and goimports, and add note that dev containers is not officially supported (#1561)

Add -no-color option to commands in issue template (#1566)

Fix databricks_mws_private_access_settings defaults (#1567)

Support for ANY value for private_access_level in databricks_mws_private_access_settings resource was removed by engineering. ACCOUNT is the new default now

Extend databricks_job with additional supported fields and document them (#1537)

Bump google.golang.org/api from 0.93.0 to 0.94.0 (#1569)

Bumps google.golang.org/api from 0.93.0 to 0.94.0.


updated-dependencies:

  • dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Remove "Public Preview" marker from Unity Catalog resources docs (#1570)

Custom suppress diff for storage in databricks_pipeline (#1574)

When the storage option isn't provided, DLT generates a temp one in the form of dbfs:/pipelines/<uuid>. But in the sequential apply the drift was detected, and pipeline was recreated. Marking it with computed or suppress_diff doesn't help because it prevents the pipeline from the detection of change of storage from value to null.

Add force_destroy flag to databricks_schema & databricks_catalog (#1578)

Fixed databricks_permissions for calling user to CAN_MANAGE on databricks_cluster (#1579)

Added databricks_group_role resource (#1575)

Fix databricks_user creation with force on account (#1577)

Added databricks_recipient resource for Delta Sharing (#1571)

Bump github.com/hashicorp/hcl/v2 from 2.13.0 to 2.14.0 (#1582)

Bumps github.com/hashicorp/hcl/v2 from 2.13.0 to 2.14.0.


updated-dependencies:

  • dependency-name: github.com/hashicorp/hcl/v2 dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Minor changes to instance pool docs (#1581)

Fix TestAccAwsOboFlow (#1584)

Document sql_task configuration block in databricks_job resource (#1589)

Docs fix: supported languages in Unity Shared clusters (#1587)

Added initial Unity Catalog integration tests (#1592)

Added some coverage for:

  • databricks_grants
  • databricks_schemas
  • databricks_schema
  • databricks_catalogs
  • databricks_catalog
  • databricks_external_location
  • databricks_storage_credential
  • databricks_metastore_assignment
  • databricks_metastore_data_access
  • databricks_metastore
Created at 1 hour ago

Prevent stripping of CAN_MANAGE permission from caller of databricks_sql_query resource (#1644)

Created at 12 hours ago
prevent stripping of `CAN_MANAGE` permission from caller of `sql_query` resource

relates to https://github.com/databricks/terraform-provider-databricks/issues/1615

this PR uses the fix from https://github.com/databricks/terraform-provider-databricks/issues/1504 to append CAN_MANAGE perms to the caller explicitly when modifying permissions on databricks sql queries

Created at 12 hours ago
[DOC] Update resource diagram with databricks_mws_permission_assignment

Thank you for the feature request! Currently, the team operates in a limited capacity, carefully prioritizing, and we cannot provide a timeline to implement this feature. Please make a Pull Request if you'd like to see this feature sooner, and we'll guide you through the journey.

Created at 19 hours ago

expand documentation for databricks_sql_visualization resource (#1630)

  • expand documentation for options field in sql-visualization resource
  • add wording for how to restore during issue scenario
Created at 3 days ago
expand documentation for `databricks_sql_visualization` resource
  • add example configuration for file() reference to hopefully avoid users passing incorrect options value
  • add known issues section and link to https://github.com/databricks/terraform-provider-databricks/issues/1615
Created at 3 days ago

Correct handling of references in databricks_permissions resources (#1631)

this fixes #1628

Created at 3 days ago
[ISSUE] Issue with `databricks_permissions` exporter for sql endpoints

Configuration

resource "databricks_permissions" "sql_endpoint_starter_endpoint" {
    sql_endpoint_id = "fb8......xyz"
    access_control {
        permission_level = "CAN_USE"
        group_name = data.databricks_group.users.display_name
    }
    access_control {
        user_name = databricks_user.k0p00te.user_name
        permission_level  = "CAN_MANAGE"
    }
}

Expected Behavior

IDs for resources should be provided with dotted notation as in: a reference to a value (to be known in the future)

Actual Behavior

The Terraform Exporter is writing some hcl configuration files with IDs that are hardcoded for resources. Resources with these IDs do no exist when other resources look for them. The culprit here is sql_endpoint_id = "fb8......xyz" which leads to the error message

Error: cannot create permissions: endpoints fb8.......xyz does not exist

Steps to Reproduce

  • Create a new SQL Endpoint
  • Add user permissions to manage the endpoint
  • Run the terraform exporter with -services=sql,access

Terraform and provider versions

Databricks terraform Provider 1.3.0 Terraform - versions above 0.15

Important Factoids

Technically the output shown above is not wrong. It reflects the state of the workspace but the user can't use it to run apply because IDs are autogenerated. As a result, all resources should have reference notation.

Created at 3 days ago
Correct handling of references in `databricks_permissions` resources

this fixes #1628

Created at 3 days ago

Bump google.golang.org/api from 0.96.0 to 0.97.0 (#1633)

Bumps google.golang.org/api from 0.96.0 to 0.97.0.


updated-dependencies:

  • dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Merge branch 'master' into issue-1628

Created at 3 days ago

Bump google.golang.org/api from 0.96.0 to 0.97.0 (#1633)

Bumps google.golang.org/api from 0.96.0 to 0.97.0.


updated-dependencies:

  • dependency-name: google.golang.org/api dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Created at 3 days ago
Bump google.golang.org/api from 0.96.0 to 0.97.0

Bumps google.golang.org/api from 0.96.0 to 0.97.0.

Dependabot compatibility score

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


You can trigger Dependabot actions by commenting on this PR:

  • @dependabot rebase will rebase this PR
  • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
  • @dependabot merge will merge this PR after your CI passes on it
  • @dependabot squash and merge will squash and merge this PR after your CI passes on it
  • @dependabot cancel merge will cancel a previously requested merge and block automerging
  • @dependabot reopen will reopen this PR if it is closed
  • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
  • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
  • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Created at 3 days ago