dtrudg
Repos
27
Followers
34

Events

closed issue
After the administrator adds the fakeroot mapping, I can't use the fakeroot parameter.

Version of Singularity:

What version of Singularity are you using? Run:

$ singularity --version 
singularity version 3.7.1-5.1.ohpc.2.1

Expected behavior

What did you expect to see when you do...?

After the administrator adds the fakeroot mapping, I can use the fakeroot parameter.

Actual behavior

image-20221130105957180

What actually happend? Why was it incorrect?

How can others reproduce this issue/problem?

Following the singularity-userdocs, the administrator(root) executs the following command to add the fakeroot mapping.

ref: https://docs.sylabs.io/guides/3.5/admin-guide/user_namespace.html#config-fakeroot

root

singularity config fakeroot --add ai2010814474
singularity config fakeroot --enable ai2010814474

After the operation, the user information(uid 1089) appears in the subuid file.

root

[root@mgt ~]cat /etc/subuid
admin:100000:65536
1001:4294836224:65536
1089:4294770688:65536
[root@mgt ~]cat /etc/subgid
admin:100000:65536
1001:4294836224:65536
1089:4294770688:65536

average user (ai2010814474)

(base) [ai2010814474@login02 ckq_pytorch]$ id
uid=1089(ai2010814474) gid=1090(ai2010814474) groups=1090(ai2010814474)

However, I fail to build an image with --fakeroot feature:

average user (ai2010814474)

(base) [ai2010814474@login02 ckq_pytorch]$ singularity build --fakeroot test.sif dayuanzhong.def 
FATAL:   could not use fakeroot: no mapping entry found in /etc/subuid for ai2010814474

When I open a subuid file, it is empty.

average user (ai2010814474)

(base) [ai2010814474@login02 ckq_pytorch]$ cat /etc/subuid
(base) [ai2010814474@login02 ckq_pytorch]$ cat /etc/subgid

ref: https://docs.sylabs.io/guides/3.5/user-guide/fakeroot.html#build

What OS/distro are you running

$ cat /etc/os-release
NAME="Rocky Linux"
VERSION="8.5 (Green Obsidian)"
ID="rocky"
ID_LIKE="rhel centos fedora"
VERSION_ID="8.5"
PLATFORM_ID="platform:el8"
PRETTY_NAME="Rocky Linux 8.5 (Green Obsidian)"
ANSI_COLOR="0;32"
CPE_NAME="cpe:/o:rocky:rocky:8.5:GA"
HOME_URL="https://rockylinux.org/"
BUG_REPORT_URL="https://bugs.rockylinux.org/"
ROCKY_SUPPORT_PRODUCT="Rocky Linux"
ROCKY_SUPPORT_PRODUCT_VERSION="8"

Conclusion

I don't know why I still can't build an image as fakeroot after the admin added the fakeroot mapping in subuid. We mainly refer to the two links mentioned above. Thank you very much!

Created at 1 hour ago
delete branch
dtrudg delete branch issue598
Created at 1 hour ago

feat: oci: support namespace flags

Support namespace request CLI options.

  • --ipc - no effect, always used in --oci mode.
  • --net - only supported with --network none.
  • --pid - no effect, always used in --oci mode.
  • -u / --userns - only effective for root, non-root always uses user ns.
  • --uts

Add info logging where the option is redundant.

Closes #1026

chore: refactor for easier testing

test: oci: Test_addNamespaces

e2e: minimal --oci namespace request tests

fix: Don't set Process.Terminal in oci launcher if no term

If we are running such that stdin is not a terminal, then the OCI runtime config should have Process.Terminal=false to avoid errors.

https://github.com/opencontainers/runc/blob/main/docs/terminals.md#issues

Merge pull request #1157 from dtrudg/issue598

feat: oci: support namespace flags

Created at 1 hour ago
pull request closed
feat: oci: support namespace flags

Description of the Pull Request (PR):

Support namespace request CLI options.

  • --ipc - no effect, always used in --oci mode.
  • --net - only supported with --network none.
  • --pid - no effect, always used in --oci mode.
  • -u / --userns - only effective for root, non-root always uses user ns.
  • --uts

Add info logging where the option is redundant.

This fixes or addresses the following GitHub issues:

  • Fixes #1026

Before submitting a PR, make sure you have done the following:

Created at 1 hour ago

fix: Don't set Process.Terminal in oci launcher if no term

If we are running such that stdin is not a terminal, then the OCI runtime config should have Process.Terminal=false to avoid errors.

https://github.com/opencontainers/runc/blob/main/docs/terminals.md#issues

Created at 20 hours ago

e2e: minimal --oci namespace request tests

Created at 21 hours ago

test: oci: Test_addNamespaces

e2e: minimal --oci namespace request tests

Created at 21 hours ago

e2e: minimal --oci namespace request tests

Created at 22 hours ago
pull request opened
Issue598

Description of the Pull Request (PR):

Support namespace request CLI options.

  • --ipc - no effect, always used in --oci mode.
  • --net - only supported with --network none.
  • --pid - no effect, always used in --oci mode.
  • -u / --userns - only effective for root, non-root always uses user ns.
  • --uts

Add info logging where the option is redundant.

This fixes or addresses the following GitHub issues:

  • Fixes #1026

Before submitting a PR, make sure you have done the following:

Created at 22 hours ago

feat: oci: support namespace flags

Support namespace request CLI options.

  • --ipc - no effect, always used in --oci mode.
  • --net - only supported with --network none.
  • --pid - no effect, always used in --oci mode.
  • -u / --userns - only effective for root, non-root always uses user ns.
  • --uts

Add info logging where the option is redundant.

Closes #1026

chore: refactor for easier testing

test: oci: Test_addNamespaces

e2e: minimal --oci namespace request tests

Created at 22 hours ago

chore: refactor for easier testing

test: oci: Test_addNamespaces

e2e: minimal --oci namespace request tests

Created at 22 hours ago

chore: refactor for easier testing

Created at 22 hours ago

chore: refactor for easier testing

Created at 22 hours ago
create branch
dtrudg create branch issue598
Created at 23 hours ago

fix: correct CAPSET_MAX for changes in #1072

In #1072, additional capabilities were added in Go code. The C starter code uses a loop up to CAPSET_MAX when reconciling capabilities vs supported on the machine. Because CAPSET_MAX was still 37, not 40, it failed to trim the 3 upper capabilities that are unsupported on older kernels.

Correct CAPSET_MAX to fix the issue on older kernels.

Fixes #1079

feat: Allow kernel supported unpriv overlay

When we are running a container in a user namespace, check whether the kernel supports rootless overlay. If it does, use this instead of underlay.

At present, the visible effects of this are that:

  • When running under a user namespace, on a supported system, the overlay session layout is used. The rootfs in the container will appear as an overlay fs mount.

  • When running under a user namespace, on a supported system, the --writable-tmpfs option can be used, with the same semantics as in set-uid mode.

Note that this changeset does not allow unprivileged persistent overlays (singularity --overlay flag). It will be possible to enable unprivileged overlay directory mounts at a later point, and potentially other overlay mounts via FUSE drivers.

The principle benefit at this stage is to avoid the complex, and numerous, bind mounts used by the underlay layout.

ToDo: Refactor the underlay / overlay handling to a simpler, more linear flow.

Fixes #818

Merge pull request #1080 from dtrudg/issue1079

fix: correct CAPSET_MAX for changes in #1072

e2e: fix tests for unpriv overlay

  • skip issue5307 test where unpriv overlay is available.
  • fix underlay config test where unpriv overlay is available.

Merge pull request #1077 from dtrudg/issue818

feat: Allow kernel supported unpriv overlay

build(deps): bump github.com/spf13/cobra from 1.6.0 to 1.6.1

Bumps github.com/spf13/cobra from 1.6.0 to 1.6.1.


updated-dependencies:

  • dependency-name: github.com/spf13/cobra dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

build(deps): bump github.com/containerd/containerd from 1.6.8 to 1.6.9

Bumps github.com/containerd/containerd from 1.6.8 to 1.6.9.


updated-dependencies:

  • dependency-name: github.com/containerd/containerd dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

feat: allow directory overlay in non-setuid mode

Kernel unprivileged overlay now means we can permit users to request a persistent overlay from a directory, when running in a non-setuid flow.

In setuid mode it still remains unsafe, as the mount occurs privileged.

Fixes #1078

Merge pull request #1085 from sylabs/dependabot/go_modules/main/github.com/containerd/containerd-1.6.9

build(deps): bump github.com/containerd/containerd from 1.6.8 to 1.6.9

Merge pull request #1084 from sylabs/dependabot/go_modules/main/github.com/spf13/cobra-1.6.1

build(deps): bump github.com/spf13/cobra from 1.6.0 to 1.6.1

refactor: Move runc/conmon functionality out of app/singularity

In preparation for further OCI runtime work, move the code that calls out to runc/conmon from internal/app/singularity into internal/pkg/runtime/launcher/oci

The oci.Launcher will make use of the basic OCI operations to run containers from its Exec function, so this is a good location while that work proceeds. The functions will be likely be modified considerably, and potentially moved again in future, as the design/implementation of the OCI runtime interaction is developed.

The internal/app/singularity OCI* functions are left as a minimal shim layer, between the CLI layer and the launcher, at this time.

Merge pull request #1081 from dtrudg/issue1078

feat: allow directory overlay in non-setuid mode

Merge pull request #1086 from dtrudg/refactor-oci

refactor: Move runc/conmon functionality out of app/singularity

fix: don't freeze for http(s) pull without content-length

When an http(s) pull doesn't provide a content-length header, we need to ensure the progress bar completes. If we don't do this, the CLI hangs indefinitely.

Fixes #1087

build(deps): bump github.com/docker/docker

Bumps github.com/docker/docker from 20.10.20+incompatible to 20.10.21+incompatible.


updated-dependencies:

  • dependency-name: github.com/docker/docker dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

Merge pull request #1088 from dtrudg/issue1087

fix: don't freeze for http(s) pull without content-length

Merge pull request #1091 from sylabs/dependabot/go_modules/main/github.com/docker/docker-20.10.21incompatible

build(deps): bump github.com/docker/docker from 20.10.20+incompatible to 20.10.21+incompatible

build(deps): bump github.com/sylabs/sif/v2 from 2.8.2 to 2.8.3

Bumps github.com/sylabs/sif/v2 from 2.8.2 to 2.8.3.


updated-dependencies:

  • dependency-name: github.com/sylabs/sif/v2 dependency-type: direct:production update-type: version-update:semver-patch ...

Signed-off-by: dependabot[bot] support@github.com

e2e: Use local http source for cache tests

When testing the cache commands in the e2e tests, pull from a local http server, rather than the library. This dramatically speeds up the time taken to run these tests, which are necessarily sequential.

Fixes #1089

Merge pull request #1097 from sylabs/dependabot/go_modules/main/github.com/sylabs/sif/v2-2.8.3

build(deps): bump github.com/sylabs/sif/v2 from 2.8.2 to 2.8.3

Created at 1 day ago
issue comment
After the administrator adds the fakeroot mapping, I can't use the fakeroot parameter.

Hello, first noting that the version here singularity version 3.7.1-5.1.ohpc.2.1 is rather old - it was released in Jan 2021, prior to the SingularityCE / Apptainer fork. We only support the current open source version of SingularityCE, which is 3.10.4 at present. Additionally, the ohpc suffix suggests this is an OpenHPC package, which may have customizations. However, I don't believe these are a cause of the issue here.

Your problem is probably indicated by the observation that:

When I open a subuid file, it is empty.

(base) [ai2010814474@login02 ckq_pytorch]$ cat /etc/subuid
(base) [ai2010814474@login02 ckq_pytorch]$ cat /etc/subgid

It looks like the root commands to create the subuid mappings were performed on a host called mgt , but you are running on a host called login02?

The content of /etc/subuid and /etc/subgid need to be synchronised to all nodes in a cluster. You must be able to see the entries on the host you are using --fakeroot on, for it to work.

Created at 2 days ago
issue comment
In dir.create(gsub(basename(args$out), "", args$out), recursive = TRUE) : cannot create dir

Hello, this doesn't look like a bug with Singularity, rather a usage question for a particular piece of software.

I would suggest asking for help on the Slack channels, or in the google group, links for which can be found at:

https://sylabs.io/singularity/

From the error messages, it's most likely you need to --bind mount a directory into the container, so that demuxify can write to a specific location. Singularity containers are read-only by default. However, I'm afraid that the developers here are not familiar with that software, and we don't have the resources to offer in-depth workflow specific troubleshooting.

There may be someone in the Slack channel or Google group who is already familiar with demuxify, and able to offer more assistance.

Created at 2 days ago
delete branch
dtrudg delete branch issue-1140
Created at 2 days ago
closed issue
oci: tmpfs mounts using --oci don't honor singularity.conf session size

Version of Singularity

main

Describe the bug

tmpfs mounts in --oci mode are hard-coded to 64MB. Need to honor the singularity.conf session size... which we should also consider changing the default for, as it is very low there.

Created at 2 days ago

fix: use conf file value for oci tmpfs size

Use the configuration file sessiondir max size value for --oci mode tmpfs mounts.

Increase the default from 16M -> 64M. The 16M default is very low, and has periodically caused issues running programs that create even small amounts of temporary data on --contained filesystems.

Fixes #1140

Merge pull request #1151 from dtrudg/issue-1140

fix: use conf file value for oci tmpfs size

Created at 2 days ago
pull request closed
fix: use conf file value for oci tmpfs size

Description of the Pull Request (PR):

Use the configuration file sessiondir max size value for --oci mode tmpfs mounts.

Increase the default from 16M -> 64M. The 16M default is very low, and has periodically caused issues running programs that create even small amounts of temporary data on --contained filesystems.

This fixes or addresses the following GitHub issues:

  • Fixes #1140

Before submitting a PR, make sure you have done the following:

Created at 2 days ago

fix: use conf file value for oci tmpfs size

Use the configuration file sessiondir max size value for --oci mode tmpfs mounts.

Increase the default from 16M -> 64M. The 16M default is very low, and has periodically caused issues running programs that create even small amounts of temporary data on --contained filesystems.

Fixes #1140

Created at 2 days ago

fix: use conf file value for oci tmpfs size

Use the configuration file sessiondir max size value for --oci mode tmpfs mounts.

Increase the default from 16M -> 64M. The 16M default is very low, and has periodically caused issues running programs that create even small amounts of temporary data on --contained filesystems.

Fixes #1140

Created at 2 days ago
pull request opened
fix: use conf file value for oci tmpfs size

Description of the Pull Request (PR):

Use the configuration file sessiondir max size value for --oci mode tmpfs mounts.

Increase the default from 16M -> 64M. The 16M default is very low, and has periodically caused issues running programs that create even small amounts of temporary data on --contained filesystems.

This fixes or addresses the following GitHub issues:

  • Fixes #1140

Before submitting a PR, make sure you have done the following:

Created at 2 days ago
create branch
dtrudg create branch issue-1140
Created at 2 days ago