gmsantos
Repos
81
Followers
146
Following
96

Alternative inspiring quotes for Laravel

18
6

Examples for ZCE Certification Preparation

19
8

URL Shortener API build on Lumen with Tests

1
0

Preparation resources for exams DP-200 and DP-201

33
16

Events

Created at 1 day ago

Bump symfony/mime from 4.3.5 to 4.4.1

Bumps symfony/mime from 4.3.5 to 4.4.1.

Signed-off-by: dependabot[bot] support@github.com

Bump symfony/http-foundation from 4.3.5 to 4.4.1

Bumps symfony/http-foundation from 4.3.5 to 4.4.1.

Signed-off-by: dependabot[bot] support@github.com

change app service name

upgrade checkout action to v2

update composer depedencies

change home message

ignore sqlite

use default path for sqlite database

Update deploy action

relax CI rules

Bump symfony/http-foundation from 4.4.5 to 4.4.7

Bumps symfony/http-foundation from 4.4.5 to 4.4.7.

Signed-off-by: dependabot[bot] support@github.com

Bump symfony/http-kernel from 4.4.5 to 4.4.13

Bumps symfony/http-kernel from 4.4.5 to 4.4.13.

Signed-off-by: dependabot[bot] support@github.com

update composer lock

Remove deploy workflow

Lock CI to PHP 7.2

Bump PHP version

Move containers to php 7.4

Embrace multi-stage build and update MySQL container to 8.0

Update Queue connection sample env var

Remove extra line

Bump composer/composer from 1.10.22 to 1.10.23

Bumps composer/composer from 1.10.22 to 1.10.23.


updated-dependencies:

  • dependency-name: composer/composer dependency-type: indirect ...

Signed-off-by: dependabot[bot] support@github.com

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Update index.md

Created at 5 days ago

Create index.md

Created at 5 days ago

Update mkdocs.yml

Created at 5 days ago

Create index.md

Created at 5 days ago

Create mkdocs.yml

Created at 5 days ago

Replace emoji to symbol

Created at 1 week ago

add SUSE-based setup

Created at 1 week ago
issue comment
Bump FAB to 4.1.4

Awesome! Thank you for all the hard work @potiuk !

Created at 1 week ago
issue comment
Bump FAB to 4.1.4

@ashb just a reminder to add this PR to the 2.4.1 milestone and to not lose it in the changelogs :)

Created at 1 week ago
delete branch
gmsantos delete branch fab-4-1-4
Created at 1 week ago

Clear autoregistered DAGs if there are any import errors (#26398)

We need to clear any autoregistered DAGs that may have been already registered if we encounter any import errors while parsing a given DAG file.

This maintains the behavior before we autoregistered DAGs.

Fix params reference for default_constraints_branch (#26406)

Fixes breeze commands that depends on the default constraints branch, such as the generated diff for breeze release-management generate-constraints

Fix UI redirect (#26409)

Co-authored-by: Konstantin Weddige konstantin.weddige@lutrasecurity.com

Add min attrs version (#26408)

I'm not sure when we started using attrs directly, but we do and we need

=22.1.0 as we use the min_length validator.

Adding podAnnotations to StatsD deployment template (#25732)

Attempt to create k8s venv without constraints if constraint build fails (#26407)

There is an issue with trying to install the k8s env where the current requirements conflict with constraints. The k8s env creation is failing in this case.

This PR will not fail hard when the constraint-build fails, instead it will attempt to build the venv without constraints. This will work in vast majority of cases.

Bump FAB to 4.1.4 (#26393)

No changes that should be replicated in airflow/www/fab_security was found.

See https://github.com/dpgaspar/Flask-AppBuilder/compare/v4.1.3...v4.1.4

I expect that the constraints dependencies got updated to:

flask-wtf==1.0.1 wtforms==3.0.1

Credentials are needed in order to push branch (#26413)

Manual triggering of author sync workflow (#26414)

  • Allow committers to run author sync manually

  • Fix test failures

update contributing documentations (#26411)

  • CONTRIBUTING.rst: replace markdown style link as rst style
  • CONTRIBUTORS_QUICK_START.rst
    • unify python version in examples to 3.8 (originally, the first example uses 3.7 which could break the setup flow)
    • fix testing command typo
Created at 1 week ago

Respect max_active_runs for dataset-triggered dags (#26348)

Co-authored-by: Ash Berlin-Taylor ash_github@firemirror.com Co-authored-by: Jed Cunningham 66968678+jedcunningham@users.noreply.github.com

Add simple blame ignore file (#26382)

Use 'git config blame.ignoreRevsFile .git-blame-ignore-revs' to apply!

Actually run the "quick image build" (#26351)

The check does not use matrix build but we left the check for matrix so the check was always skipped.

Use the full path to the workflow being edited (#26386)

Fix pre-commit for checking revision heads map (#26373)

We need to check with the base_version instead of the parsed version obj

Better validation of Dataset URI during dag parse (#26389)

Previously we had the validation on the Dataset model, but we since moved the "dag" facing class to a separate one. This adds validation to the public class, and extends the validation to not allow space-only strings

Allow .git-blame-ignore-revs file to not have license header (#26396)

Require dag_id arg for dags list-runs (#26357)

While it would ideal to transition to a positional arg like was attempted in #25978, this unfortunately does result in a breaking change so we cannot do it now.

By instead marking the existing arg as required, we maintain backcompat while also providing a helpful error message to the user if they forget it.

This reverts commit ed6ea72f181a1d381cc1ff6c801f10cc0bc0d830.

Created at 2 weeks ago
issue comment
Fix params reference for default_constraints_branch
<a href="/apache/airflow/constraints-source-providers-3.10.txt">Moved Permanently</a>.

curl is getting this message because it's trying to download the following file pattern:

https://raw.githubusercontent.com/apache/airflow//constraints-source-providers-3.10.txt

At the first moment, I thought that curl was not following redirects and it was missing a curl -L, but the problem was the container receiving a blank value for DEFAULT_CONSTRAINTS_BRANCH.

Created at 2 weeks ago
create branch
gmsantos create branch breeze-diff-current-constraint
Created at 2 weeks ago
pull request opened
Fix params reference for default_constraints_branch

Fixes breeze commands that depends on the default constraints branch, such as the generated diff for breeze release-management generate-constraints

Created at 2 weeks ago
issue comment
Bump FAB to 4.1.4

Unfortunately, this PR didn't pass the CI step check for a canary run:

https://github.com/apache/airflow/actions/runs/3054848681/jobs/4927231877#step:8:616

pr-labels = []
target-repo = apache/airflow
head-repo = gmsantos/airflow
pr-number = 26393
event-name = pull_request
runs-on = ubuntu-20.04
in-workflow-build = false
build-job-description = Skip Build (look in pull_request_target)
canary-run = false
run-coverage = false

I think it's because I opened this PR from my fork, and it makes sense not to try to update the constraints from a PR outside the airflow main repo :)

Created at 2 weeks ago
issue comment
Bump FAB to 4.1.4

Nice workflow @potiuk ! 🤞

Created at 2 weeks ago

Respect max_active_runs for dataset-triggered dags (#26348)

Co-authored-by: Ash Berlin-Taylor ash_github@firemirror.com Co-authored-by: Jed Cunningham 66968678+jedcunningham@users.noreply.github.com

Add simple blame ignore file (#26382)

Use 'git config blame.ignoreRevsFile .git-blame-ignore-revs' to apply!

Actually run the "quick image build" (#26351)

The check does not use matrix build but we left the check for matrix so the check was always skipped.

Use the full path to the workflow being edited (#26386)

Merge branch 'main' into fab-4-1-4

Created at 2 weeks ago
pull request opened
Bump FAB to 4.1.4

No changes that should be replicated in airflow/www/fab_security was found.

See https://github.com/dpgaspar/Flask-AppBuilder/compare/v4.1.3...v4.1.4

I expect that the constraints dependencies got updated to:

flask-wtf==1.0.1 wtforms==3.0.1


^ Add meaningful description above

Read the Pull Request Guidelines for more information. In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed. In case of a new dependency, check compliance with the ASF 3rd Party License Policy. In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.

Created at 2 weeks ago
create branch
gmsantos create branch fab-4-1-4
Created at 2 weeks ago

Fix asset compilation via setup.py (#25201)

Asset compilation via setup.py has been broken in #25169.

This PR fixes it.

Add ts types generation to static checks (#25167)

Fix missing space for breeze build-image hint command (#25204)

Add Tessian to list of companies using Airflow (#25206)

make MsSQL tests runnable on Python 3.8 (#25214)

mysql lib does support newer version of python https://github.com/pymssql/pymssql/pull/659

Migrate Google example bigquery_to_mssql to new design AIP-47 (#25174)

related: #22447, #22430

call updateNodeLabels after expandGroup (#25217)

make MsSQL tests runnable on Python 3.8 (2nd) (#25216)

I missed the hive provider from the commit in https://github.com/apache/airflow/pull/25214

Also compile assets in non-main (#25220)

Assets compilation when CI and PROD images are prepared should also be run in non-main branch.

SQSPublishOperator should allow sending messages to a FIFO Queue (#25171)

Show dataset readiness for the next run (#25141)

Include missing mention of external_executor_id in sql_engine_collation_for_ids docs (#25197)

Dataset details view (#25208)

  • dataset details view

  • Fix sorting and naming

  • update table tests

  • update comments

Convert RDS Export Sample DAG to System Test (AIP-47) (#25205)

  • Convert RDS Export Sample DAG to System Test

  • PR Fixes

Stop SLA callbacks gazumping other callbacks and DOS'ing the DagProcessorManager queue (#25147)

Upgrade to Pip 22.2 (#25218)

The 22.2 version of pip has just been released. Looks like it does not break neither constraints nor "eager upgrade" builds when we build images locally so it's safe to update to it in main.

Improve ElasticsearchTaskHandler (#21942)

  • Improve ElasticsearchTaskHandler:
  • use builtin logging.makeLogRecord instead of strange _ESJsonLogFmt
  • do not re-sort already sorted logs
  • apply ISO 8601 datetime format
  • fixed several found bugs

Fix order of returned rows in a flaky test_outlets_dataset test (#25231)

In Postgres especially (but in generaly in all databases, if there is no order specified in select query, the rows might come in random order. It depends on many factors.

The test query here retrieved the dags without any order but expected the list to be in specific order.

This PR adds ordering - it also removes side-effects of the test by using fixture that clears the datasets before and after the tests that rely/modify datasets - because othrwise failure of one of the tests can create side effects that fail the other tests (this is what happened in this case)

Sagemaker System Tests - Part 3 of 3 - example_sagemaker_endpoint.py (AIP-47) (#25134)

  • Sagemaker System Tests - Part 3 of 3 - example_sagemaker_endpoint.py

  • PR Fixes

  • Fix failing static checks - unused import

Common SQLCheckOperators Various Functionality Update (#25164)

  • Add batching to SQL Check Operators

Commit adds a WHERE clause to the sql statement that allows for arbitrary batching in a given table.

  • Fix bug with multiple table checks

When multiple table checks are given to the SQLTableCheckOperator and at least one is not a fully aggregate statement, a GROUP BY clause was previously needed. This commit updates the operator to use the get_pandas_df() method instead of _get_first() to return a pandas dataframe object that contains the check names and check results from the new style of query. The new style of query uses UNION ALL to run each test as its own SELECT statement, bypassing the need to do a GROUP BY.

  • Update test failure logic

Changed name of method from _get_failed_tests to _get_failed_checks to better match naming, and updated logic of the method to include an optional column param. The query in the column check operator is removed from the failed test exception message, as it was only ever showing the last query, instead of the relevant one(s). This is replaced by the column, which will be more useful in debugging.

  • Add table alias to SQLTableCheckOperator query

Without a table alias, the query does not run on Postgres and other databases. The alias is arbitrary and used only for proper query execution.

  • Fix formatting error in operator

  • Add batching to SQL Check Operators

Commit adds a WHERE clause to the sql statement that allows for arbitrary batching in a given table.

  • Fix bug with multiple table checks

When multiple table checks are given to the SQLTableCheckOperator and at least one is not a fully aggregate statement, a GROUP BY clause was previously needed. This commit updates the operator to use the get_pandas_df() method instead of _get_first() to return a pandas dataframe object that contains the check names and check results from the new style of query. The new style of query uses UNION ALL to run each test as its own SELECT statement, bypassing the need to do a GROUP BY.

  • Update test failure logic

Changed name of method from _get_failed_tests to _get_failed_checks to better match naming, and updated logic of the method to include an optional column param. The query in the column check operator is removed from the failed test exception message, as it was only ever showing the last query, instead of the relevant one(s). This is replaced by the column, which will be more useful in debugging.

  • Add table alias to SQLTableCheckOperator query

Without a table alias, the query does not run on Postgres and other databases. The alias is arbitrary and used only for proper query execution.

  • Fix formatting error in operator

  • Move alias to proper query build statement

The table alias should be in the self.sql query build statement as that is where the table it needs to alias is defined.

  • Add batching to SQL Check Operators

Commit adds a WHERE clause to the sql statement that allows for arbitrary batching in a given table.

  • Fix bug with multiple table checks

When multiple table checks are given to the SQLTableCheckOperator and at least one is not a fully aggregate statement, a GROUP BY clause was previously needed. This commit updates the operator to use the get_pandas_df() method instead of _get_first() to return a pandas dataframe object that contains the check names and check results from the new style of query. The new style of query uses UNION ALL to run each test as its own SELECT statement, bypassing the need to do a GROUP BY.

  • Update test failure logic

Changed name of method from _get_failed_tests to _get_failed_checks to better match naming, and updated logic of the method to include an optional column param. The query in the column check operator is removed from the failed test exception message, as it was only ever showing the last query, instead of the relevant one(s). This is replaced by the column, which will be more useful in debugging.

  • Add table alias to SQLTableCheckOperator query

Without a table alias, the query does not run on Postgres and other databases. The alias is arbitrary and used only for proper query execution.

  • Fix formatting error in operator

  • Bug fixes and updates to test and operator

Fixed bug in test where the dataframe column names did not match the operator's expected dataframe column names. Added more info to the SQLColumnCheckOperator's batch arg. Fixed the location of table aliasing in SQLTableCheckOperator.

  • Remove merge conflict lines

  • Rename parameter batch to partition_clause

Gives a clearer name to the parameter and adds templating to the SQLTableCheckOperator.

  • Fix typo in docstring

  • Reformat operator file

Created at 2 weeks ago
issue comment
Kubernetes does not start when another context is set

I could not reproduce the issue in 4.12.0, it seems fixed for me.

@brki @zbpan-sh @samalone I'm closing this issue (for now), let me know if you still have this issue after 4.12.0, happy to reopen it if persists for you.

Created at 2 weeks ago