just for the reference: https://issues.shopware.com/issues/NEXT-25323

PHP Version

8.1

Shopware Version

6.4.18

Expected behaviour

The autoloader classmap for plugins which are not managed by composer should only be generated once and then reused. For Plugins which are not active we can skip the generation anyways.

Actual behaviour

the classmap is generated on every request which is painfully slow on Network Mounts

How to reproduce

install a plugin which is not managed by composer onto a NFS share.

i hacked together a "dirty" fix (cant get the cache_dir properly into the KernelPluginLoader):

// vendor/shopware/core/Framework/Plugin/KernelPluginLoader/KernelPluginLoader.php

// ...
	private function registerPluginNamespaces(string $projectDir): void
	{
		foreach ($this->pluginInfos as $plugin) {
			$pluginName = $plugin['name'] ?? $plugin['baseClass'];

			// plugins managed by composer are already in the classMap
			if ($plugin['managedByComposer']) {
				continue;
			}

			// if not active we skip the classmap generation
			if (!$plugin['active']) {
				continue;
			}

                        // thats the part where its dirty, better to use kernel.cache_dir here
			$classmapFile = sprintf("%s/%s.classmap", sys_get_temp_dir(), str_replace('\\', '_', $pluginName));

			if (file_exists($classmapFile)) {
				$classmap = file_get_contents($classmapFile);
				if ($classmap !== false) {
					$classmap = unserialize($classmap);
					if (is_array($classmap)) {
						$this->classLoader->addClassMap($classmap);
					}
					continue;
				}
			}

			if (!isset($plugin['autoload'])) {
				$reason = sprintf(
					'Unable to register plugin "%s" in autoload. Required property `autoload` missing.',
					$plugin['baseClass']
				);

				throw new KernelPluginLoaderException($pluginName, $reason);
			}

			$psr4 = $plugin['autoload']['psr-4'] ?? [];
			$psr0 = $plugin['autoload']['psr-0'] ?? [];

			if (empty($psr4) && empty($psr0)) {
				$reason = sprintf(
					'Unable to register plugin "%s" in autoload. Required property `psr-4` or `psr-0` missing in property autoload.',
					$plugin['baseClass']
				);

				throw new KernelPluginLoaderException($pluginName, $reason);
			}

			$files = [];

			foreach ($psr4 as $namespace => $paths) {
				if (\is_string($paths)) {
					$paths = [$paths];
				}
				$mappedPaths = $this->mapPsrPaths($pluginName, $paths, $projectDir, $plugin['path']);
				$this->classLoader->addPsr4($namespace, $mappedPaths);
				if ($this->classLoader->isClassMapAuthoritative()) {
					foreach ($mappedPaths as $mappedPath) {
						array_push($files, ...ClassMapGenerator::createMap($mappedPath));
					}
				}
			}

			foreach ($psr0 as $namespace => $paths) {
				if (\is_string($paths)) {
					$paths = [$paths];
				}
				$mappedPaths = $this->mapPsrPaths($pluginName, $paths, $projectDir, $plugin['path']);

				$this->classLoader->add($namespace, $mappedPaths);
				if ($this->classLoader->isClassMapAuthoritative()) {
					foreach ($mappedPaths as $mappedPath) {
						array_push($files, ...ClassMapGenerator::createMap($mappedPath));
					}
				}
			}

			$this->classLoader->addClassMap($files);

			file_put_contents($classmapFile , serialize($files));
		}
	}

// ...

fix test

plan file compatiblity for windows

cov

pipeline

deps

allow use of local Terraform executable in tests

If the env var TFEXEC_E2ETEST_TERRAFORM_PATH is set to the path of a valid local Terraform executable, when e2e tests are run, only tests appropriate to that executable's version will be run.

This involves both the runTest() and runTestVersions() codepaths, since both are used in e2e tests.

add custom binary test instructions

document test env vars

explain why clean temp dir is used

Update CHANGELOG.md

v0.17.0 [skip ci]

Bump github.com/hashicorp/hc-install from 0.3.2 to 0.4.0 (#319)

Bumps github.com/hashicorp/hc-install from 0.3.2 to 0.4.0.

• Release notes
• Changelog
• Commits

updated-dependencies:

• dependency-name: github.com/hashicorp/hc-install dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Fix for state_push: path for statefile is reqired for run state push cmd

add basic e2e tests for state push/pull

Update CHANGELOG.md

v0.17.1 [skip ci]

Bump github.com/hashicorp/go-version from 1.5.0 to 1.6.0 (#323)

Bumps github.com/hashicorp/go-version from 1.5.0 to 1.6.0.

• Release notes
• Changelog
• Commits

updated-dependencies:

• dependency-name: github.com/hashicorp/go-version dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Update CHANGELOG.md

tfexec/internal/e2etest: Skip TestShow_noInitEtcdBackend on >=1.3.0 and prereleases (#325)

Reference: https://github.com/hashicorp/terraform/pull/31326

Fixes the new test failures against Terraform main branch. This also adds testing for the cloud and remote backends, which should be more stable as they are integral to the Terraform experience.

Previously:

$ TFEXEC_E2ETEST_VERSIONS=refs/heads/main go test -count=1 -run='TestShow_noInitEtcdBackend' -v ./tfexec/internal/e2etest
=== RUN   TestShow_noInitEtcdBackend
=== RUN   TestShow_noInitEtcdBackend/etcd_backend-refs/heads/main
    util_test.go:112: caching exec "gitref:refs/heads/main" in dir "/var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/tfinstall2067874872/gitref-refs-heads-main"
2022/06/30 15:56:57 git_revision.go:88: running terraform pre-clone check (timeout: 1m0s)
2022/06/30 15:56:57 git_revision.go:94: terraform pre-clone check finished
2022/06/30 15:56:57 git_revision.go:123: cloning terraform repository from https://github.com/hashicorp/terraform.git to /var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-build-terraform3971808670 (timeout: 5m0s)
2022/06/30 15:57:37 git_revision.go:136: cloning terraform finished
2022/06/30 15:57:37 git_revision.go:142: terraform repository HEAD is at acb79a75451e39126f1cdc596ff802a91d64d712
2022/06/30 15:57:37 git_revision.go:169: building terraform (timeout: 10m0s)
2022/06/30 15:57:37 go_build.go:55: executing go ["build" "-o" "/var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-terraform-acb79a75451e39126f1cdc596ff802a91d64d7123048666161/terraform"] in "/var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-build-terraform3971808670"
2022/06/30 15:57:51 git_revision.go:171: building of terraform finished
    util_test.go:173: [INFO] running Terraform command: /var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-terraform-acb79a75451e39126f1cdc596ff802a91d64d7123048666161/terraform show -json -no-color
    show_test.go:232: expected error ErrNoInit, got *fmt.wrapError: exit status 1

        Error: Invalid backend type

          on main.tf line 2, in terraform:
           2:   backend "etcd" {

        The "etcd" backend is not supported in Terraform v1.3 or later.
--- FAIL: TestShow_noInitEtcdBackend (55.76s)
    --- FAIL: TestShow_noInitEtcdBackend/etcd_backend-refs/heads/main (55.76s)
FAIL
FAIL    github.com/hashicorp/terraform-exec/tfexec/internal/e2etest     56.060s
FAIL

After change:

$ TFEXEC_E2ETEST_VERSIONS=refs/heads/main go test -count=1 -run='TestShow_noInitEtcdBackend' -v ./tfexec/internal/e2etest
=== RUN   TestShow_noInitEtcdBackend
=== RUN   TestShow_noInitEtcdBackend/etcd_backend-refs/heads/main
    util_test.go:112: caching exec "gitref:refs/heads/main" in dir "/var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/tfinstall2843106250/gitref-refs-heads-main"
2022/06/30 15:59:17 git_revision.go:88: running terraform pre-clone check (timeout: 1m0s)
2022/06/30 15:59:17 git_revision.go:94: terraform pre-clone check finished
2022/06/30 15:59:17 git_revision.go:123: cloning terraform repository from https://github.com/hashicorp/terraform.git to /var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-build-terraform22313428 (timeout: 5m0s)
2022/06/30 15:59:59 git_revision.go:136: cloning terraform finished
2022/06/30 15:59:59 git_revision.go:142: terraform repository HEAD is at acb79a75451e39126f1cdc596ff802a91d64d712
2022/06/30 15:59:59 git_revision.go:169: building terraform (timeout: 10m0s)
2022/06/30 15:59:59 go_build.go:55: executing go ["build" "-o" "/var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-terraform-acb79a75451e39126f1cdc596ff802a91d64d7122130413204/terraform"] in "/var/folders/w8/05f3x02n27x72g0mc2jy6_180000gp/T/hc-install-build-terraform22313428"
2022/06/30 16:00:12 git_revision.go:171: building of terraform finished
    show_test.go:223: etcd backend was removed in Terraform 1.3, so test is not valid
--- PASS: TestShow_noInitEtcdBackend (56.66s)
    --- SKIP: TestShow_noInitEtcdBackend/etcd_backend-refs/heads/main (56.66s)
PASS
ok      github.com/hashicorp/terraform-exec/tfexec/internal/e2etest     57.704s

tfexec: Add (Terraform).SetLogCore() and (Terraform).SetLogProvider() methods (#324)

Reference: https://github.com/hashicorp/terraform-exec/issues/101

Update CHANGELOG.md

v0.17.2 [skip ci]

use last stable xcode 12 for mac builds

Certain terraform-exec tests run versions of Terraform earlier than 0.12 in order to test compatibility behaviour.

Any version of Terraform complied with Go < 1.11 causes errors when run on macOS Monterey 12 or later, which corresponds to Xcode version 13 or later. Please see the following pages for more information: https://github.com/golang/go/wiki/MacOS12BSDThreadRegisterIssue https://xcodereleases.com/

We must therefore use Xcode 12 for these tests. CircleCI supports only the latest stable version, currently v12.5.1.

When Xcode 16 is released, CircleCI will drop support for Xcode 12, so we will need to resolve this problem another way.

try newer versions

fix force-copy

Signed-off-by: Chanwit Kaewkasi chanwit@gmail.com

Bump Go module compatibility to Go 1.17 (#284)

This will match the README documentation, especially now that Go 1.18 is released.

Updated via:

go mod edit -go=1.17
go mod tidy

replace use of testify

remove testify dependency

deps: Remove direct dependency on github.com/sergi/go-diff (#287)

It was only being used for "pretty" text differences in 3 tests. It is still an indirect dependency due to hc-install.

$ go mod why -m github.com/sergi/go-diff
# github.com/sergi/go-diff
github.com/hashicorp/terraform-exec/tfexec/internal/testutil
github.com/hashicorp/hc-install/build
github.com/go-git/go-git/v5
github.com/sergi/go-diff/diffmatchpatch

deps: Remove github.com/andybalholm/crlf (#289)

Replaced via trivial strings handling as the strings are not large.

v0.16.1 changelog

v0.16.1 [skip ci]

Add support for state push/pull

statePull: tfjson does not support state subcommand, output string instead

go fmt example code

Signed-off-by: Gennady Lipenkov xgen@yandex-team.ru

bump v1.1 test version

fix missed testTempDir(t)

Bump github.com/google/go-cmp from 0.5.7 to 0.5.8 (#297)

Bumps github.com/google/go-cmp from 0.5.7 to 0.5.8.

• Release notes
• Commits

updated-dependencies:

• dependency-name: github.com/google/go-cmp dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Bump github.com/hashicorp/hc-install from 0.3.1 to 0.3.2 (#302)

Bumps github.com/hashicorp/hc-install from 0.3.1 to 0.3.2.

• Release notes
• Changelog
• Commits

updated-dependencies:

• dependency-name: github.com/hashicorp/hc-install dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

tfexec: Add (Terraform).SetLog() method (#291)

• tfexec: Add (Terraform).SetLog() method

Reference: https://github.com/hashicorp/terraform-exec/issues/290

• tfexec: Add testing for SetLog

• tfexec: Add SetLogPath testing

• tfexec: Ensure SetLog version compatibility is 0.15 minimum, testing updates for darwin/arm64

make tests pass (or skip) on darwin/arm64 (#304)

• make tests pass on darwin/arm64

• address PR feedback (add extra v1 tests)

Bump github.com/hashicorp/go-version from 1.4.0 to 1.5.0 (#306)

Bumps github.com/hashicorp/go-version from 1.4.0 to 1.5.0.

• Release notes
• Changelog
• Commits

updated-dependencies:

• dependency-name: github.com/hashicorp/go-version dependency-type: direct:production update-type: version-update:semver-minor ...

Signed-off-by: dependabot[bot] support@github.com

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Test for runTerraformCmd leaked go-routine (#299)

• Update TestContext_sleepTimeoutExpired to check for canceling within timeframe

Add a timeout to the test to ensure that the terraform apply cancels within a reasonable time of the 5s timeout.

Currently, this test is not canceling the terraform apply as expected. In the logs you can see that the test takes 1 min rather than ~5s:

    --- PASS: TestContext_sleepTimeoutExpired/sleep-0.12.31 (62.13s)

=== RUN   TestContext_sleepTimeoutExpired/sleep-0.12.31
    util_test.go:113: [INFO] running Terraform command: /var/folders/6y/gy9gggt14379c_k39vwb50lc0000gn/T/terraform_1378921380/terraform apply -no-color -auto-approve -input=false -lock=true -parallelism=10 -refresh=true
    util_test.go:103: CLI Output:
	    // truncated ...
        time_sleep.sleep: Creating...
        time_sleep.sleep: Still creating... [10s elapsed]
        time_sleep.sleep: Still creating... [20s elapsed]
        time_sleep.sleep: Still creating... [30s elapsed]
        time_sleep.sleep: Still creating... [41s elapsed]
        time_sleep.sleep: Still creating... [51s elapsed]
        time_sleep.sleep: Creation complete after 1m0s [id=2022-05-06T17:40:20Z]

        Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

• Remove runTerraformCmd check for cmd.ProcessState

Processes were not being killed because cmd.ProcessState was nil. With this change, processes will be able to make the request to Kill().

Added a temporary log to printout cmd.ProcessState to demonstrate. Will be removed in next commit.

Note: this will cause hanging TestContext_sleepTimeoutExpired due to a known Golang issue with killing a command when Stdout or Stderr are set to anything besides nil or *os.File. This is because the Kill does not notify the stdout/stderr subprocesses to stop. cmd.Wait (called by cmd.Run) waits indefinitely for those subprocesses to stop.

• Read logs from Stderr/out Pipe to avoid hanging

TestContext_sleepTimeoutExpired can occasionally hang when killing a command that has Stdout or Stderr set to anything besides nil or *os.File. https://github.com/golang/go/issues/23019

Use workaround to read from StdoutPipe and StderrPipe rather than setting Stdout / Stderr

• Test for runTerraformCmd leaked go-routine

Currently, when runTerraformCmd is called, it starts a go-routine to kill the Terraform CLI on context.Done(). However, when the Terraform CLI completes and runTerraformCmd() finishes, the go-routine continues running unnecessarily.

If the caller cancels the context down the line, this will stop the go-routine and it will log the error: "error from kill: os: process already finished" because the Terraform CLI has already finished.

Added a test for this in cmd_default.go and cmd_linux.go. Have not tried it in linux yet though.

When running with the race detector:

==================
WARNING: DATA RACE
Read at 0x00c0002516c8 by goroutine 7:
  bytes.(*Buffer).String()
      /usr/local/go/src/bytes/buffer.go:65 +0x36a
  github.com/hashicorp/terraform-exec/tfexec.Test_runTerraformCmd_default()
      /Users/lornasong/go/src/github.com/hashicorp/terraform-exec/tfexec/cmd_default_test.go:35 +0x360
  testing.tRunner()
  // truncated ...

Previous write at 0x00c0002516c8 by goroutine 8:
  bytes.(*Buffer).grow()
      /usr/local/go/src/bytes/buffer.go:147 +0x3b1
  bytes.(*Buffer).Write()
      /usr/local/go/src/bytes/buffer.go:172 +0xcd
  log.(*Logger).Output()
      /usr/local/go/src/log/log.go:184 +0x466
  log.(*Logger).Printf()
      /usr/local/go/src/log/log.go:191 +0x6e
  github.com/hashicorp/terraform-exec/tfexec.(*Terraform).runTerraformCmd.func1()
      /Users/lornasong/go/src/github.com/hashicorp/terraform-exec/tfexec/cmd_default.go:24 +0x2a5
  // truncated ...
==================

* Use CommandContext to kill instead of manually doing it

* Fix EOF error check to use error.Is()

This also fixes a race condition caused by using the pointer to the io.EOF

* Update tests to use separate string.Builder-s for stdout and stderr

string.Builder is a non-comparable type which is not safe for concurrent use
when shared by Cmd.Stdout and Cmd.Stderr. Causes a race condition when accessing
the the builder when Cmd is running.

* Fixes to runTerraformCmd for race conditions

 - Use waitgroups for more readability
 - Improve handling errors from writeOutput
 - Finish reading from pipes before calling cmd.Wait - fixes a race condition
 that leads to an error :`read |0: file already closed`
 - Because now waiting for pipes to finish reading, need to update waitGroup to
 close buf.Read on context cancel. Otherwise buf.Read blocks until next line
 before stopping. Causes TestContext_sleepTimeoutExpired takes a little too long
 to cancel (~20s)

Co-authored-by: Kyle Carberry <kyle@carberry.com>
                                    

Update CHANGELOG.md

Update CHANGELOG.md

deps

update pipelines

untaint

gcp backend

i get the following warnings in my HA logs:

/config/custom_components/dreame_vacuum/dreame/types.py:971: SyntaxWarning: "is not" with a literal. Did you mean "!="?
  elif self.type is not 0 and SEGMENT_TYPE_CODE_TO_NAME.get(self.type):
/config/custom_components/dreame_vacuum/dreame/device.py:1184: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if command is "" or parameters is None:
/config/custom_components/dreame_vacuum/dreame/device.py:1430: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if not suction_level or suction_level is "":
/config/custom_components/dreame_vacuum/dreame/device.py:1433: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if not water_volume or water_volume is "":
/config/custom_components/dreame_vacuum/dreame/device.py:2117: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if cleaning_sequence is "" or not cleaning_sequence:
/config/custom_components/dreame_vacuum/dreame/device.py:2490: SyntaxWarning: "is" with a literal. Did you mean "=="?
  and self.battery_level is 100
/config/custom_components/dreame_vacuum/dreame/map.py:1722: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if custom_name is "":
/config/custom_components/dreame_vacuum/dreame/map.py:1895: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if segment_id is 63:
/config/custom_components/dreame_vacuum/dreame/map.py:1897: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if segment_id is 62:
/config/custom_components/dreame_vacuum/dreame/map.py:1899: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if segment_id is 61:
/config/custom_components/dreame_vacuum/dreame/map.py:2248: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if segment_id is 63:
/config/custom_components/dreame_vacuum/dreame/map.py:2251: SyntaxWarning: "is" with a literal. Did you mean "=="?
  elif segment_id is 62:
/config/custom_components/dreame_vacuum/dreame/map.py:2254: SyntaxWarning: "is" with a literal. Did you mean "=="?
  elif segment_id is 61:
/config/custom_components/dreame_vacuum/switch.py:49: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if value is 0
/config/custom_components/dreame_vacuum/number.py:39: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if value is 0

i think it might be fine, but im not sure. see https://adamj.eu/tech/2020/01/21/why-does-python-3-8-syntaxwarning-for-is-literal/

BugFix

fix initializing phase by sending logs to buffer during initialization by introducing a second variable initializing so subsequent log calls wont call the initialize phase again.

Update

• removed deprecated sequenceToken
• added proper typehints (might break php7, but its deprecated anyways)

fixes #111 fixes #86

its definitly a BC break, so a new major version would be needed

well actually it doesnt solve the problem in reality. the solutions was to disable createGroup and create the group & stream proactivly.

Remember PHP is stateless, so every Requests kicks on the Describe Groups Stuff which raises the RateLimitException on heavy load