Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

use larger runners for build job

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

use larger runners for build job

I opened this as a "real" PR because I assumed the workflow would otherwise not register.

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

use larger runners for build job

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

use larger runners for build job

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

Workflow for building platform packages

Kicked off manually.

Inputs:

• list of packages (can be a shell word list with wildcards)
• stack to build for (choice)
• overwrite existing packages (bool)
• publish package repo after builds (bool, default true)
• GitHub Actions runner concurrency (number, default 3)

still WIP, but need this PR open for the workflow to register

Yup, can definitely be split into two, maybe three PRs. Since the PHP buildpack relies on this branch, I'll get to cleaning it up and opening those PRs sometime.

I can confirm a hit of about 60 ms, measured on different architectures and CPU sizes, with e.g. the default Ubuntu CA certs, compared to 1.1.1.

Heroku-20 beta was just released: https://devcenter.heroku.com/changelog-items/1937

I tried installing this buildpack on a Heroku-20 app, and the build succeeded, though I didn't test any further.

It would be great to add some per-stack testing to this buildpack, to ensure it works on Heroku-20 at runtime and to prevent future stack-specific regressions like #9).

Many thanks :-)

test all stacks

Closes #20

GUS-W-8276311

Merge pull request #34 from heroku/ci-stacks-matrix

Test all stacks

Closes #20

GUS-W-8276311

Hi

I noticed that currently the buildpack doesn't invalidate the cache when the app's $STACK changes, which will cause issues depending on changes to the format of APT's cache between Ubuntu versions. (And even if APT is safely migrates/ignores old files, the cache would be bloated with the previous distro's .deb files, so it would be good to clean it out regardless.)

A possible pattern for solving this would be: https://github.com/heroku/heroku-buildpack-apt/blob/0357ec1e8863a803ada6c2d57f0850f2c5fa0366/bin/compile#L32-L41

Many thanks :-)

use poppler-utils from stack

GUS-W-12618598

Merge pull request #32 from heroku/stack-poppler

Use poppler-utils from stack

Current main and update-ffmpeg will now fail, as the recent stack image update brought in all the libraries that poppler-utils needs as part of the libvips install.

As a result, the find for .so files piped to xargs dirname in bin/compile will fail, because no .so files are actually added:

remote: -----> ActiveStorage Preview app detected
remote: -----> Installing binary dependencies for ActiveStorage Preview
remote:        Detected ffmpeg version change, flushing cache
remote:        Downloading packagesW: http://apt.postgresql.org/pub/repos/apt/dists/jammy-pgdg/InRelease: Key is stored in legacy trusted.gpg keyring (/etc/apt/trusted.gpg), see the DEPRECATION section in apt-key(8) for details.
remote: ..
remote:        Installing packages..
remote: dirname: missing operand
remote: Try 'dirname --help' for more information.
remote:  !     Push rejected, failed to compile ActiveStorage Preview app.
remote: 
remote:  !     Push failed

This simplifies bin/compile greatly by removing all the now-unnecessary Apt trickery:

% heroku buildpacks:set -i 1 https://github.com/heroku/heroku-buildpack-activestorage-preview.git#stack-poppler
Buildpack set. Next release on blooming-castle-09486 will use:
  1. https://github.com/heroku/heroku-buildpack-activestorage-preview.git#stack-poppler
  2. heroku/ruby
Run git push heroku main to create a new release using these buildpacks.

% git push heroku master
…
remote: -----> ActiveStorage Preview app detected
remote: -----> Installing binary dependencies for ActiveStorage Preview
remote:        Downloading ffmpeg version 5.1.2
remote:        Installing ffmpeg
remote:        Done
remote:        Exporting environment variables
remote:        Done
…

% heroku run which ffmpeg pdftocairo
Running which ffmpeg pdftocairo on ⬢ blooming-castle-09486... up, run.3697 (Basic)
/app/.heroku/activestorage-preview/bin/ffmpeg
/usr/bin/pdftocairo

GUS-W-12618598

Also closes #21

GUS-W-8276306

Using this buildpack, I seem to have ffmpeg version 4.2.3.

It looks like there are plenty of subsequent ffmpeg releases, including 4.4 and 5.0.

I am having some odd behavior that I don't know is caused by a bug fixed in a subsequent version of ffmpeg, but normally, if I could control it, updating to the latest ffmpeg would be one thing I might try first.

Are there any plans/schedule for ever updating the ffmpeg version here, or any chance of getting it updated?

Hi, my app uses this buildpack to get ffmpeg/ffprobe installed, for use with Rails.

But then the app also calls ffmpeg and ffprobe directly for it's own uses, not just the way Rails calls it.

On heroku-22 stack, some calls to ffmpeg or ffprobe as installed by this app will result in a segfault.

In particular, ffmpeg and ffprobe both have a feature where they can be called with a URL argument for remote input, not just a local filesystem argument. This is especially useful for ffprobe, as it does not need to download the whole file to probe it, it'll just download the prefix it needs. Eg:

ffprobe https://scih-data-dev.s3.amazonaws.com/test_video/SampleVideo_360x240_1mb.mp4

ffmpeg -i https://scih-data-dev.s3.amazonaws.com/test_video/SampleVideo_360x240_1mb.mp4 converted.ogg

On heroku-22 stack, with this buildpack, both of the above commands result in a segfault.

On heroku-20 with this backpack, both of the above commands work fine.

Please let me know if there is a better place to report this; also please let me know if this is considered a bug, or if since Rails itself may not currently call ffmpeg/ffprobe with a URL argument, perhaps it is not supported?

Also, if anyone knows any other way to get ffmpeg/ffprobe installed on heroku-22 other than this buildpack, please let me know!

Update buildpack to pull from S3

Previously we were hosting a .deb with the updates from https://github.com/heroku/heroku-buildpack-activestorage-preview/pull/28 we are now using a tar.gz.

We can get rid of the cache as we have to download from S3 either way. This change is gated on the full rollout of https://devcenter.heroku.com/changelog-items/2547 as #28 relies on updated system packages to be present.

Shellcheck

Intentional variable splitting

Update test string

.DS_Store to gitignore

Test FFMPEG works

Change SFFMPEG_VERSION to FFMPEG_VERSION

Add changelog entry and clarify versioning

Refactor bin/compile

• Remove cache of deb packages

Previously we stored a cache of the static ffmpeg binary, however both the cache and the binary are stored on S3 so this does not improve compile times. By moving to not use the cache and deleting old contents we can reduce cache size while preserving compile times

• Move location of dpkg -x to be logically consistent with installing poppler

Previously we were installing ffmpeg and poppler via the apt-get mechanism. With the move to have FFMPEG be a simple curl to disk, it makes more sense to have the dpkg -x logic be with the poppler download code.

• Add additional logging

Log download and install for poppler and ffmpegcache. Adds the version of ffmpeg to the log output.

Add test for poppler execution

Merge pull request #31 from heroku/update-ffmpeg

Upgrade to FFMPEG 5.1.2 and fix heroku-22 execution