Sample apps and code written for Google Cloud in the Go programming language.
Apache License 2.0
3665
179
1590

This sample does not have end-to-end test coverage. This coverage should be added.

A direct health check is minimal, but since this sample demonstrates handling message from Pub/Sub it may be worth seeing if the following flow could be done without too much flakiness:

  • Deploy the Cloud Run service
  • Create a Pub/Sub topic
  • Create a Pub/Sub subscription
  • Publish a message to the Pub/Sub topic
  • Check the log output

This issue is blocked on finding a solution for testing log details in #696.

This sample does not have end-to-end test coverage. This coverage should be added.

The existing unit test checks we write the expected log. The essential benefit of an end-to-end test is confirmation that Cloud Logging is successfully parsing the data structure. There is a node.js implementation of this test in nodejs-docs-samples:/run/logging-manual.

This issue is blocked on finding a solution for testing log details in #696.

I have trained and deployed a model for image classification (multi-label). When I try to use
the following code, I get an error => rpc error: code = InvalidArgument desc = Model ID is invalid. It should start with 3 letters."
But in the dashboard If i go to the console.cloud.google.com > vertex ai > model registry and then select version details Tab, The Model_ID does not have any 3 letter prefix in it. Its a plain 18 digit number. Dont know what to do here. Help required.

client, err := automl.NewPredictionClient(ctx)

....

req := &automlpb.PredictRequest{
		Name: fmt.Sprintf("projects/%s/locations/%s/models/%s", "MY_PROJECT_ID", "LOCATION", "MODEL_ID"), //<= where can I get this Model's Prefix
		Payload: &automlpb.ExamplePayload{
			Payload: &automlpb.ExamplePayload_Image{
				Image: &automlpb.Image{
					Data: &automlpb.Image_ImageBytes{
						ImageBytes: bytes,
					},
				},
			},
		},
		// Params is additional domain-specific parameters.
		Params: map[string]string{
			// score_threshold is used to filter the result.
			"score_threshold": "0.1",
		},
	}

	resp, err := client.Predict(ctx, req)
	if err != nil {
		 return fmt.Errorf("Predict: %v", err)
	}

Note: #2570 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: bc18844
buildURL: Build Status, Sponge
status: failed

Test output
    get_similar_products_uri_test.go:39: getSimilarProductsURI: ProductSearch: rpc error: code = NotFound desc = No matching products found. Please verify that the ProductSet exists, has images and has been indexed.

Note: #2288 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: 92964f9
buildURL: Build Status, Sponge
status: failed

Test output
    create_and_delete_glossary_test.go:52: deleteGlossary: Wait: rpc error: code = DeadlineExceeded desc = context deadline exceeded

Note: #2569 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: bc18844
buildURL: Build Status, Sponge
status: failed

Test output
    get_similar_products_test.go:39: getSimilarProducts: ProductSearch: rpc error: code = NotFound desc = No matching products found. Please verify that the ProductSet exists, has images and has been indexed.

# exit_code collects all of the exit codes of the tests, and is used to set the
# exit code at the end of the script.
exit_code=0
set +e # Don't exit on errors to make sure we run all tests.

...

There are no updates to exit_code, so it seems this will always exit 0.

Environment details

  • OS type and version: Cloud Run Jobs
  • Go version: 1.15 and 1.17
  • google-cloud-logging version: 1.4.2

Steps to reproduce

  1. Use this code for a working example in Cloud Run
    Main.go
package main

import (
	"context"
	"log"
	"cloud.google.com/go/logging"
)

var logger *log.Logger

func main() {
	ctx := context.Background()

	// Sets your Google Cloud Platform project ID.
	projectID := "gdglyon-cloudrun"

	// Creates a client.
	client, err := logging.NewClient(ctx, projectID)
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}
	defer client.Close()

	// Sets the name of the log to write to.
	logName := "my-log"

	logger = client.Logger(logName).StandardLogger(logging.Warning)

	// Logs "hello world", log entry is visible at
	// Cloud Logs.
	logWarning()
}

func logWarning() {
	logger.Println("hello world")
}

go.mod

module golang.com/gblaquiere

go 1.15

require cloud.google.com/go/logging v1.4.2

Dockerfile

FROM golang:1.17 AS builder

# Set necessary environmet variables needed for our image
ENV GO111MODULE=on \
    CGO_ENABLED=0 \
    GOOS=linux \
    GOARCH=amd64

# Move to working directory /build
WORKDIR /build

# Copy and download dependency using go mod
COPY go.mod .
COPY go.sum .
RUN go mod download

# Copy the code into the container
COPY . .

# Build the application
RUN go build -o main .

FROM debian:buster-slim
RUN set -x && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y \
    ca-certificates && \
    rm -rf /var/lib/apt/lists/*

COPY --from=builder /build/main /

ENTRYPOINT ["/main"]

Deploy on cloud run. It failed (no webserver deployed) but the warning trace is present in the Cloud Run logs section with that label part

resource: {
labels: {
configuration_name: "log-format-go"
location: "us-central1"
project_id: "gdglyon-cloudrun"
revision_name: "log-format-go-00003-tuj"
service_name: "log-format-go"
}
type: "cloud_run_revision"
}
severity: "WARNING"

Now, do the same thing with Cloud Run Jobs
Same container, same code

This time the logs are present but not visible in the Cloud Run Jobs logs section. Why? Because the label is not correct:

resource: {
labels: {
project_id: "gdglyon-cloudrun"
}
type: "project"
}
severity: "WARNING"

The Cloud Logging Library doesn't correctly detect the Cloud Run Jobs runtime environment and don't put the correct values in the logs that cause a filtering issue in the Cloud Run Jobs UI.

(Cloud Run team is aware of that issue)

Note: same issue in Python library: googleapis/python-logging#663

Add samples demonstrating getting and setting Autoclass feature for a bucket. Currently, Autoclass supports new buckets only - a bucket can be enabled for Autoclass only at creation, once disabled it cannot be enabled again

  • region tags: storage_get_autoclass and storage_set_autoclass

canonical samples (internal): go/gcs-autoclass-canonical-sample
details and coordination (internal): go/gcs-dpe-autoclass
python-storage example: googleapis/python-storage#791

Please wait for approval from myself before merging/releasing so we can coordinate around release dates.

I have close to 40 secrets in my secret store and I am using golang sdk's ListSecrets to fetch all secrets along with their value.

This takes somewhere 20-25 secs to return all secrets. Here is my code -
https://github.com/pravinpushkar/components-contrib/blob/ad8f21d3b246c2bd11addf34c39687a9bc816104/secretstores/gcp/secretmanager/secretmanager.go#L132

which is also similar to this - https://github.com/GoogleCloudPlatform/golang-samples/blob/692f934095411b2cc5d4a8ab0c8cc804034c31f6/secretmanager/list_secrets.go

Is their recommended or better way to handle this operation ?

I am using secretemanager#1.7.0

Note: #2403 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: 030b37e
buildURL: Build Status, Sponge
status: failed

Test output
received:
entries:{log_name:"projects/golang-samples-tests-3/logs/cloudaudit.googleapis.com%2Factivity" resource:{type:"logging_log" labels:{key:"name" value:"log-example"} labels:{key:"project_id" value:"golang-samples-tests-3"}} proto_payload:{[type.googleapis.com/google.cloud.audit.AuditLog]:{service_name:"logging.googleapis.com" method_name:"google.logging.v2.LoggingServiceV2.DeleteLog" resource_name:"projects/golang-samples-tests-3/logs/log-example" status:{} authentication_info:{principal_email:"kokoro-golang-samples-tests-3@golang-samples-tests-3.iam.gserviceaccount.com" service_account_key_name:"//iam.googleapis.com/projects/golang-samples-tests-3/serviceAccounts/kokoro-golang-samples-tests-3@golang-samples-tests-3.iam.gserviceaccount.com/keys/05f4f474c5e39177cefa244f629ad07d6636a8cb" principal_subject:"serviceAccount:kokoro-golang-samples-tests-3@golang-samples-tests-3.iam.gserviceaccount.com"} authorization_info:{resource:"projects/golang-samples-tests-3/logs/log-example" permission:"logging.logs.delete" granted:true resource_attributes:{service:"logging.googleapis.com" name:"projects/golang-samples-tests-3/logs/log-example"}} request_metadata:{caller_ip:"34.70.17.196" caller_supplied_user_agent:"grpc-go/1.44.1-dev,gzip(gfe)" request_attributes:{time:{seconds:1649784784 nanos:686170707} auth:{}} destination_attributes:{}} request:{fields:{key:"@type" value:{string_value:"type.googleapis.com/google.logging.v2.DeleteLogRequest"}} fields:{key:"logName" value:{string_value:"projects/golang-samples-tests-3/logs/log-example"}}}}} timestamp:{seconds:1649784784 nanos:672573289} receive_timestamp:{seconds:1649784786 nanos:265711659} severity:NOTICE insert_id:"2if0r7ddrxi"}
    taillogs_test.go:90: tailLogs sample failed to complete after 2 minutes

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.


commit: 5b0b37e
buildURL: Build Status, Sponge
status: failed

Test output
    get_supported_languages_test.go:34: getSupportedLanguages got:
        ----
        Supported languages:
        Language code: af
        Language code: am
        Language code: ar
        Language code: az
        Language code: be
        Language code: bg
        Language code: bn
        Language code: bs
        Language code: ca
        Language code: ceb
        Language code: ckb
        Language code: co
        Language code: cs
        Language code: cy
        Language code: da
        Language code: de
        Language code: el
        Language code: en
        Language code: eo
        Language code: es
        Language code: et
        Language code: eu
        Language code: fa
        Language code: fi
        Language code: fr
        Language code: fy
        Language code: ga
        Language code: gd
        Language code: gl
        Language code: gu
        Language code: ha
        Language code: haw
        Language code: he
        Language code: hi
        Language code: hmn
        Language code: hr
        Language code: ht
        Language code: hu
        Language code: hy
        Language code: id
        Language code: ig
        Language code: is
        Language code: it
        Language code: iw
        Language code: ja
        Language code: jw
        Language code: ka
        Language code: kk
        Language code: km
        Language code: kn
        Language code: ko
        Language code: ku
        Language code: ky
        Language code: la
        Language code: lb
        Language code: lo
        Language code: lt
        Language code: lv
        Language code: mai
        Language code: mg
        Language code: mi
        Language code: mk
        Language code: ml
        Language code: mn
        Language code: mr
        Language code: ms
        Language code: mt
        Language code: my
        Language code: ne
        Language code: nl
        Language code: no
        Language code: ny
        Language code: or
        Language code: pa
        Language code: pl
        Language code: ps
        Language code: pt
        Language code: ro
        Language code: ru
        Language code: rw
        Language code: sd
        Language code: si
        Language code: sk
        Language code: sl
        Language code: sm
        Language code: sn
        Language code: so
        Language code: sq
        Language code: sr
        Language code: st
        Language code: su
        Language code: sv
        Language code: sw
        Language code: ta
        Language code: te
        Language code: tg
        Language code: th
        Language code: tk
        Language code: tl
        Language code: tr
        Language code: tt
        Language code: ug
        Language code: uk
        Language code: ur
        Language code: uz
        Language code: vi
        Language code: xh
        Language code: yi
        Language code: yo
        Language code: zh
        Language code: zh-TW
        Language code: zu
        ----
        Want to contain:
        ----
        zh-CN
        ----

Copied from googleapis/google-cloud-go#3682, filed by https://github.com/mrgleeco.

Request:

A golang analog to the Java implementation as described at https://cloud.google.com/bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1

What we need:

Example showing how to use https://pkg.go.dev/github.com/apache/arrow/go/arrow and relevant subpackages for reading individual row batches, and at least some schema handling code.

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.


commit: 6f68c4b
buildURL: Build Status, Sponge
status: failed

Test output
    disks_test.go:255: setDiskAutodelete got err: unable to wait for the operation: googleapi: Error 503: Authentication backend unavailable.

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.


commit: 5000f56
buildURL: Build Status, Sponge
status: failed

Test output
    custom_machine_type_test.go:95: unable to get instance: googleapi: Error 404: The resource 'projects/golang-samples-tests-4/zones/europe-central2-b/instances/test-vm-06-17-2022-2713917555295625236' was not found
    custom_machine_type_test.go:104: incorrect instance MachineType got "", want "https://www.googleapis.com/compute/v1/projects/golang-samples-tests-4/zones/europe-central2-b/machineTypes/n2-custom-8-10240"
    custom_machine_type_test.go:109: deleteInstance got err: googleapi: Error 404: The resource 'projects/golang-samples-tests-4/zones/europe-central2-b/instances/test-vm-06-17-2022-2713917555295625236' was not found
    custom_machine_type_test.go:179: modifyInstanceWithExtendedMemory got err: unable to get instance: googleapi: Error 404: The resource 'projects/golang-samples-tests-4/zones/europe-central2-b/instances/test-vm-06-17-2022-2713917555295625236' was not found
    custom_machine_type_test.go:182: modifyInstanceWithExtendedMemory got "Instance created\n", want "Instance updated"
    custom_machine_type_test.go:187: unable to get instance: googleapi: Error 404: The resource 'projects/golang-samples-tests-4/zones/europe-central2-b/instances/test-vm-06-17-2022-2713917555295625236' was not found
    custom_machine_type_test.go:191: incorrect instance MachineType got "", want suffix "819200-ext"
    custom_machine_type_test.go:196: deleteInstance got err: googleapi: Error 404: The resource 'projects/golang-samples-tests-4/zones/europe-central2-b/instances/test-vm-06-17-2022-2713917555295625236' was not found

To support deployment to Cloud Run, let's get a Dockerfile for the bookshelf app.

Reference from other getting started apps:

We want to update the dual-region samples to uniformly include location, locationType, and customPlacementConfig metadata in the output.

This should be a small change to: https://github.com/GoogleCloudPlatform/golang-samples/blob/HEAD/storage/buckets/create_bucket_dual_region.go

Here's an example from Node.js storage: googleapis/nodejs-storage#2016

Our minimum supported version for go is now ensures all clients should respect error wrapping format directives. Let's audit all our existing BigQuery samples and ensure they do things correctly with respect introspecting error details.

ex: instances of fmt.Errorf("blah: %v", err) should likely be rewritten as fmt.Errorf("blah: %w", err)

This test failed!

To configure my behavior, see the Flaky Bot documentation.

If I'm commenting on this issue too often, add the flakybot: quiet label and
I will stop commenting.


commit: 593a3e5
buildURL: Build Status, Sponge
status: failed

Test output
    subscription_test.go:281: pullMsgs got Received 0 messages
    want Received 1 messages</pre></details>

Note: #2088 was also for this test, but it was closed more than 10 days ago. So, I didn't mark it flaky.


commit: ba5aa11
buildURL: Build Status, Sponge
status: failed

Test output
    pubsublite_admin_test.go:323: listSubscriptionsInTopic got err: subPathIter.Next got err: rpc error: code = NotFound desc = Resource projects/770622176810/locations/us-west1-c/topics/admin-test-8bf4e0ed-9d95-4afe-92c9-0b015a487336 of type TOPIC does not exist.
        error details: name = ErrorInfo reason = RESOURCE_NOT_EXIST domain = pubsublite.googleapis.com metadata = map[resource_identifier:projects/770622176810/locations/us-west1-c/topics/admin-test-8bf4e0ed-9d95-4afe-92c9-0b015a487336 resource_type:TOPIC]