Upgrade forbiddenapis to 3.5

Upgrade forbiddenapis to 3.5

Hi, I noticed on a fork of Elasticsearch that the thirdPartyAudit Gradle plugin fails now because it parses the output of the checker with regular expressions. It looks like sobody has to rewrite it and at least rmeoves the "exclusions" feature: Due to https://github.com/policeman-tools/forbidden-apis/pull/210 the full list of class names that were not found while parsing the 3rd party JAR is no longer printed to stdout in the CLI (and no longer logged as warning). This makes the consistency checks fail. I don't fully understand why it is done like that, but IMHO it should maybe just pass the "ignore-missing-classes" parameter and not parse its output.

Upgrade forbiddenapis to 3.5

Signed-off-by: Uwe Schindler uwe@thetaphi.de

Upgrade forbiddenapis to 3.5 (#1494)

See https://github.com/policeman-tools/forbidden-apis/wiki/Changes#version-35-released-2023-03-27

This is caused by this change: https://github.com/policeman-tools/forbidden-apis/pull/210

There's no workaround as all class names missing are no longer listed, so maybe the code is too strict, but I have no idea how to fix it - because I don't know why it complains at all.

I think you cant do it like that anymore. The thirdPartyAuditChecker relies on regular expressions to find some missing classes. This does not work anymore, sorry.

Maybe scan those thirs party audits with the old CLI version. I do not even understand what the code is doing... @rmuir ?

This is caused by thirdPartyAudit: "Unnecessary exclusions, following classes are not missing"

This no longer works that way as the forbiddenapis checker has toned down warning messages, so the check may need to be disabled. I have no idea where it can be found...

At moment Github is completely down, no merging of PRs works not adding comments. See Twitter...

I only tried:

$ gradlew forbiddenApis

Passed.

I did not try precommit but it fails for whatever reason. I assume it is because of the new version changes some logging messages and the CLI parser falls on this. I have no time to look into this, as this third party audit task is some hack by @rmuir.

No idea why the checks failed. Please merge if you like, it is up to you. I won't add any more fixes to the PR.

This PR is of informational "theres something t upgrade after new release" purpose only. Do whatever you think is right. Thanks.

No idea why the checks failed. Please merge if you like, it is up to you. I won't add any more fixes to the PR.

This PR is of informational "theres something t upgrade after new release" purpose only. Do whatever you think is right. Thanks.

Description

The new version was released a minute ago: https://github.com/policeman-tools/forbidden-apis/wiki/Changes#version-35-released-2023-03-27

Summary of relevant changes for Opensearch:

• Faster startup time as Gradle plugin initialization was compiled statically and is part of JAR file
• Support for Java 20 signatures and bytecode of Java 21

Updated the release notes to include the fix for CVE-2022-41917. (#5434)

Signed-off-by: David Venable dlv@amazon.com

Signed-off-by: David Venable dlv@amazon.com

[Weighted Shard Routing] Fail open requests on search shard failures (#5072)

• Fail open requests on search shard failures (

Signed-off-by: Anshu Agarwal anshukag@amazon.com

Address fail open comments (#5778)

[Weighted Shard Routing] Refactor and fix singleton in FailAwareWeightedRouting

Signed-off-by: Anshu Agarwal anshukag@amazon.com

Awareness health api rest and transport layer changes (#5694)

• Adding awareness health api rest and transport layer changes

Signed-off-by: Nishchay Malhotra nishcha@amazon.com

[Remote Translog] Trimming based on remote segment upload and cleaning older tlog files (#5662)

• RemoteFSTranslog Trimming and GC Logic

Signed-off-by: Gaurav Bafna gbbafna@amazon.com

Reduce number of operations in RemoteFSTranslogTests.testConcurrentWriteViewsAndSnapshot (#5789)

Signed-off-by: Sachin Kale kalsac@amazon.com

Gracefully handle concurrent zone decommission action (#5542)

• Control concurrency and handle retries during decommissioning of awareness attributes

Signed-off-by: Rishab Nahata rnnahata@amazon.com

Send replicated boolean on supported versions (#5809)

Signed-off-by: Suraj Singh surajrider@gmail.com

Signed-off-by: Suraj Singh surajrider@gmail.com

Revert 'Added jackson dependency to server" and change extension reading (#5768)

• Remove two permissions from server security policy and change extension reading

Signed-off-by: Ryan Bogan rbogan@amazon.com

• Addressed PR Comments and added CHANGELOG

Signed-off-by: Ryan Bogan rbogan@amazon.com

• Revert 'Added jackson dependency to server'

Signed-off-by: Ryan Bogan rbogan@amazon.com

• Update SHAs

Signed-off-by: Ryan Bogan rbogan@amazon.com

• Ignore test that uses removed permission

Signed-off-by: Ryan Bogan rbogan@amazon.com

• Fixed spotless

Signed-off-by: Ryan Bogan rbogan@amazon.com

Signed-off-by: Ryan Bogan rbogan@amazon.com

Bump azure-core-http-netty from 1.12.7 to 1.12.8 in /plugins/repository-azure (#5761)

• Bump azure-core-http-netty in /plugins/repository-azure

Bumps azure-core-http-netty from 1.12.7 to 1.12.8.

• Release notes
• Commits

updated-dependencies:

• dependency-name: com.azure:azure-core-http-netty dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

• Updating SHAs

Signed-off-by: dependabot[bot] support@github.com

• Update changelog

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

Add v2.5.0 release notes (#5816)

Signed-off-by: Kunal Kotwani kkotwani@amazon.com

Signed-off-by: Kunal Kotwani kkotwani@amazon.com

Update release notes (#5820)

Signed-off-by: Kunal Kotwani kkotwani@amazon.com

Signed-off-by: Kunal Kotwani kkotwani@amazon.com

Update release notes for v2.5 (#5839)

Signed-off-by: Kunal Kotwani kkotwani@amazon.com

Signed-off-by: Kunal Kotwani kkotwani@amazon.com

Upgrading Jettison due to CVE-2022-45685 (#5777)

• Upgrading Jettison due to CVE

Signed-off-by: Sarat Vemulapalli vemulapallisarat@gmail.com

• Updated Changelog

Signed-off-by: Sarat Vemulapalli vemulapallisarat@gmail.com

Signed-off-by: Sarat Vemulapalli vemulapallisarat@gmail.com

Enhance searchable snapshots to enable a read-only view of older snapshots (#5812)

• Enhance searchable snapshots to enable a read-only view of older snapshots (#5429)

• Enhance searchable snapshots to enable a read-only view of older snapshots

This change removes the guardrails around N-1 backward compatibility and uses Lucene's "expert" APIs to read snapshots (Lucene segments) older than N-1 on a best-effort basis. The functionality is gated by an additional feature flag, separate from the searchable snapshots flag. Note that the Lucene integration is rather inefficient because the necessary "expert" Lucene APIs are still package-private.

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Added some unit tests

This change also includes a test index ZIP file for the unit tests. The change also introduces a bug fix in the readAnySegmentsInfo method to close the reader before returning the SegmentInfos object - this avoids dangling/open file handles.

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Incorporating PR feedback

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Incorporate PR comments from andrross

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Remove use of IndexSetting for minimum version for snapshots backwards compatibility

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Moved ES 6.3.0 test data to a subdirectory

This change also includes an update to the file name to clarify that it is an ES index, and changing the associated markdown file name to just README.md. All tests that reference this ZIP file have corresponding changes to the path they reference.

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Update unit tests to use try-with-resources

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Added FeatureFlagSetter helper class

Also refactored unit test classes to use the helper class.

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Incorporating PR feedback from @mch2

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Fix IndexSettingsTests

Updated the asserts in IndexSettingsTests to account for the new defaulting behavior.

Signed-off-by: Kartik Ganesh gkart@amazon.com

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Fixed compile issues after cherry-pick

Note that the unit tests are still failing at this commit since the Lucene 9 libraries no longer hold constants for Lucene 7 and below, so the fromId logic resolves the Lucene version to 8.

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Fix multiple aspects of version resolution

This change fixes resolution of the Lucene version for legacy versions since the Lucene 9 libraries no longer hold constants for Lucene 7 and below. The change also updates DECLARED_VERSIONS to derive from the Versions class rather than LegacyESVersions (thereby ignoring legacy versions). This in turn required a change to the minimumIndexCompatibleVersion logic for LegacyESVersion. Finally, the testMinimumIndexCompatibilityVersion unit test was updated to use accurate version identifiers.

All unit tests pass and the code compiles, but actual functionality is still broken because some backwards compatibility logic was removed in the current branch that is retained in 2.x

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Reintroducing backwards compatibility logic in certain classes

This reverts changes made in #4728 and #4702. These were only made in main and not backported to 2.x This change also adds unit tests for IndexMetadataGenerations

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Declared LegacyESVersion constants for better readability

This commit also includes a correction to documentation, and removes the unnecessary "afterWriteSnapBlob" runnable from BlobStoreRepository

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Fixing PR reference in CHANGELOG

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Revert CHANGELOG update

This feature was released in 2.5.0 so it no longer needs to be listed in the changelog.

Signed-off-by: Kartik Ganesh gkart@amazon.com

• Remove outdated comment

Signed-off-by: Kartik Ganesh gkart@amazon.com

Signed-off-by: Kartik Ganesh gkart@amazon.com

adding check to avoid parallel replication events taking place (#5831)

Signed-off-by: Poojita Raj poojiraj@amazon.com

Signed-off-by: Poojita Raj poojiraj@amazon.com

Bump reactor-netty from 1.1.1 to 1.1.2 in /plugins/repository-azure (#5876)

• Bump reactor-netty from 1.1.1 to 1.1.2 in /plugins/repository-azure

Bumps reactor-netty from 1.1.1 to 1.1.2.

• Release notes
• Commits

updated-dependencies:

• dependency-name: io.projectreactor.netty:reactor-netty dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

• Updating SHAs

Signed-off-by: dependabot[bot] support@github.com

• Update changelog

Signed-off-by: dependabot[bot] support@github.com

Signed-off-by: dependabot[bot] support@github.com Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <dependabot[bot]@users.noreply.github.com>

Add cluster manager task throttling stats in nodes stats API (#5790)

• Add cluster manager task throttling stats in nodes stats API

Signed-off-by: Dhwanil Patel dhwanip@amazon.com

[Remote Translog] Use InputStream that supports mark and reset while uploading translog files (#5868)

• Use stream that supports mark and reset for translog upload

Signed-off-by: Sachin Kale kalsac@amazon.com

Removed unnecessary use of Long.toString from BlobStoreRepository. (#5833)

Signed-off-by: Mani singh.mani1231@gmail.com

Description

The new version was released a minute ago: https://github.com/policeman-tools/forbidden-apis/wiki/Changes#version-35-released-2023-03-27

Summary of relevant changes for Elasticserach:

• Faster startup time as Gradle plugin initialization was compiled statically and is part of JAR file
• Support for Java 20 signatures and bytecode of Java 21

I have no time to submit PR (and don't want to due to legal reasons => no open source license). So this is for informative purposes only.

Upgrade forbiddenapis to version 3.5 (#12215)

Upgrade forbiddenapis to version 3.5. This tones down some verbose warnings printed while checking Java 19 and Java 20 sourcesets for the MR-JAR

This tones down some verbose warnings printed while checking Java 19 and Java 20 sourcesets for the MR-JAR.

$ gradlew forbiddenApis
Starting a Gradle Daemon (subsequent builds will be faster)

> Task :errorProneSkipped
WARNING: errorprone disabled (skipped on builds not running inside CI environments, pass -Pvalidation.errorprone=true to enable)

> Task :lucene:core:forbiddenApisMain19
While scanning classes to check, the following referenced classes were not found on classpath (this may miss some violations):
  java.lang.foreign.MemorySegment, java.lang.foreign.MemorySession, java.lang.foreign.ValueLayout,... (and 5 more).

> Task :lucene:core:forbiddenApisMain20
While scanning classes to check, the following referenced classes were not found on classpath (this may miss some violations):
  java.lang.foreign.Arena, java.lang.foreign.MemorySegment, java.lang.foreign.SegmentScope,... (and 6 more).

BUILD SUCCESSFUL in 35s
233 actionable tasks: 76 executed, 157 up-to-date

update PR number

See https://github.com/policeman-tools/forbidden-apis/issues/207 for details.

update issue number

This tones down some verbose warnings printed while checking Java 19 and Java 20 sourcesets for the MR-JAR.

Switch to next version (maybe go to 4.0 later)