lidalei
Repos
32
Followers
39
Following
49

Various data mining algorithms implemented with sklearn and tensorflow.

13
7

Algorithms and implementations to participate in Kaggle YouTube-8M Video Understanding Competition

0
0

Project of Massively Parallel Machine Learning at Technical University of Madrid.

0
0

Java interface for fastText

5
2

Events

Created at 1 day ago
Created at 1 day ago
Created at 2 days ago
Created at 2 days ago
Created at 3 days ago
Created at 3 days ago
Created at 6 days ago
started
Created at 6 days ago
lidalei create branch custom-pubsub-proto-to-bq
Created at 1 week ago
started
Created at 1 week ago
started
Created at 2 weeks ago
Created at 2 weeks ago
Created at 2 weeks ago
Created at 2 weeks ago
started
Created at 2 weeks ago

Update README.md

Created at 2 weeks ago
delete branch
lidalei delete branch fix_bq_job_nont_found_on_kill
Created at 2 weeks ago
Created at 2 weeks ago

fix test for bigquery cancel_query

Created at 2 weeks ago
started
Created at 3 weeks ago
pull request opened
set project_id and location when cancelijng bigquery job

This PR fixes an issue when canceling BigQuery jobs. The issue arises when the service account is in a Google Cloud project A but the job runs in another project B. The default project and location won't work anymore.

Created at 3 weeks ago

Fix Vertex AI Custom Job training issue (#25367)

Remove extraneous word in installation guide (#25371)

Removes a single stray word from the installation instructions

updated documentation for databricks operator (#24599)

Remove unnecessary asset compilaton for prod images (#25374)

When prod image is built, we install airflow from packages and asset compilation happens as part of the package preparation.

There is no need whatsoever to repeat it here for PROD images (it is still needed for CI images though).

add downstream events to task instances (#25375)

Restore pushing CI image as latest to GHCR.io (#25380)

We stopped pushing latest CI/PROD images to ghcr.io when we started to run multiplatform builds and switched to --cache-from option of buildx. Hoever there are still some workflows that might require the latest image (for example Codespaces) and generally speaking someone could just pull the image if they are curious.

This PR adds back pushing the image (only "linux/amd" during image cache preparation.

fix - resolve bash by absolute path (#25331)

Co-authored-by: Matt Rixman MatrixManAtYrService@users.noreply.github.com

Convert ECS Fargate Sample DAG to System Test (#25316)

fix: change disable_verify_ssl behaviour (#25023)

The problem is that verify_ssl is overwritten by the configuration from the kube_config or load_incluster_config file.

Translate system tests migration (AIP-47) (#25340)

Use newer kubernetes authentication method in internal vault client (#25351)

This gets rid of the hvac deprecation warning about auth_kubernetes being removed in version 1.0 that is currently printed when using this authentication method.

Memorystore assets & system tests migration (AIP-47) (#25361)

Check expand_kwargs() input type before unmapping (#25355)

Change stdout and stderr access mode to append in commands (#25253)

Co-authored-by: Iuhos Zoltán iuhosz@ukatemi.com Co-authored-by: Tzu-ping Chung uranusjr@gmail.com

Add missing import in best-practices code example (#25391)

  • Add missing import in best-practices code example

This PR adds a missing import to the "unit test for a custom operator" code example. While this code example won't run on its own any way since MyCustomOperator isn't defined (and probably shouldn't be for simplicity), I noticed when applying this code to my own custom operator that an import for DAG was missing. This PR adds that back in, so the only missing import is for the user-added custom operator.

YandexCloud provider: Support new Yandex SDK features for DataProc (#25158)

Fix datasets list page (#25382)

  • add link to dag dependencies from datasets

  • fix dataset selection

Adjust limits when constructing cross-provider dependencies (#25364)

When we are constructing cross-provider dependencies, we should adjust the limits to account for some apache-airflow packages that are not yet released.

The adjustment is to add ".*" after the version number when dependency is lower-bound with >=. It's not explicitly mentioned in PEP 440 - it is only mentioned there that it works for equality, but since >= is also part of equal it also works there.

Example is a common-sql package that might be limited to >=1.1.0 in google provider as part of the change, but it might not yet be released as it is being released together with the package it is needed by.

We need to adjust the version whenever in our providers we refer to other providers with >= install clause because --pre flag in pip only allows to install direct pre-release (and development) dependencies but it does not modify requirements of those package to also include pre-releases as transitive dependency.

Also, we need to remove the limits in devel dependencies because the packages are not yet released at the moment we use those dependencies, so the limits in devel should be removed, allowing the developers to install Airflow without those devel packages.

Also a bug was found that would prevent to generate the dependencies in case provider.yaml file only changed (bad specification of .pre-commit include)

Move all "old" SQL operators to common.sql providers (#25350)

Previously, in #24836 we moved Hooks and added some new operators to the common.sql package. Now we are salso moving the operators and sensors to common.sql.

Add missing option when pushing latest images to cache :( (#25399)

The #25380 re-introduced pushing image as latest on main success and (as unfortunately happens) since this is only testable after merge, a small bug crippled in making main build fail.

This flag should fix the problem.

Created at 3 weeks ago

set project_id when get bigquery job

Created at 3 weeks ago
create branch
lidalei create branch fix_bq_job_nont_found_on_kill
Created at 3 weeks ago
started
Created at 3 weeks ago
Created at 3 weeks ago
Created at 1 month ago