manual tar then ch-tar2dir difficulties
I wanted to do some work in a Charliecloud container and then tar up the results to use later. I got the following:
[trandles@cn812 ~]$ ch-tar2dir /scratch/beedev/arm64v8.gcc.tar.gz /var/tmp
creating new image /var/tmp/arm64v8.gcc
/var/tmp/arm64v8.gcc unpacked ok
[trandles@cn812 ~]$ ch-run -w /var/tmp/arm64v8.gcc/ -- /bin/bash
trandles@cn812:/$ ls
WEIRD_AL_YANKOVIC boot etc lib mnt proc run srv tmp var
bin dev home media opt root sbin sys usr
trandles@cn812:/$ git clone https://github.com/hpc/charliecloud.git
Cloning into 'charliecloud'...
remote: Counting objects: 3078, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 3078 (delta 0), reused 1 (delta 0), pack-reused 3071
Receiving objects: 100% (3078/3078), 3.02 MiB | 2.06 MiB/s, done.
Resolving deltas: 100% (2011/2011), done.
trandles@cn812:/$ ls
WEIRD_AL_YANKOVIC boot dev home media opt root sbin sys usr
bin charliecloud etc lib mnt proc run srv tmp var
trandles@cn812:/$ exit
exit
[trandles@cn812 ~]$ cd /var/tmp
[trandles@cn812 tmp]$ tar -zcf /scratch/beedev/arm64v8.gcc.charliecloud.tar.gz arm64v8.gcc/
[trandles@cn812 tmp]$ cd
[trandles@cn812 ~]$ rm -rf /var/tmp/arm64v8.gcc/
[trandles@cn812 ~]$ ch-tar2dir /scratch/beedev/arm64v8.gcc.charliecloud.tar.gz /var/tmp
creating new image /var/tmp/arm64v8.gcc.charliecloud
/var/tmp/arm64v8.gcc.charliecloud unpacked ok
[trandles@cn812 ~]$ ch-run -w /var/tmp/arm64v8.gcc.charliecloud/ -- /bin/bash
ch-run: can't bind /etc/passwd to /ram/var/tmp/arm64v8.gcc.charliecloud/etc/passwd: No such file or directory (ch-run.c:203 2)
[trandles@cn812 ~]$ ls /var/tmp/arm64v8.gcc.charliecloud/
arm64v8.gcc dev mnt WEIRD_AL_YANKOVIC
[trandles@cn812 ~]$
This behavior makes sense in hindsight, but was definitely unexpected. I can think of a few ways to address this:
- Document the behavior and provide an example of the correct way to tar up a modified image. The following works fine.
[trandles@cn812 tmp]$ cd /var/tmp/arm64v8.gcc/
[trandles@cn812 arm64v8.gcc]$ ls
bin charliecloud etc lib mnt proc run srv tmp var
boot dev home media opt root sbin sys usr WEIRD_AL_YANKOVIC
[trandles@cn812 arm64v8.gcc]$ tar -zcf /scratch/beedev/arm64v8.gcc.charliecloud-2.tar.gz ./
[trandles@cn812 arm64v8.gcc]$ cd
[trandles@cn812 ~]$ rm -rf /var/tmp/arm64v8.gcc/
[trandles@cn812 ~]$ ch-tar2dir /scratch/beedev/arm64v8.gcc.charliecloud-2.tar.gz /var/tmp
creating new image /var/tmp/arm64v8.gcc.charliecloud-2
/var/tmp/arm64v8.gcc.charliecloud-2 unpacked ok
[trandles@cn812 ~]$ ch-run -w /var/tmp/arm64v8.gcc.charliecloud-2/ -- /bin/bash
trandles@cn812:/$ ls
WEIRD_AL_YANKOVIC boot dev home media opt root sbin sys usr
bin charliecloud etc lib mnt proc run srv tmp var
trandles@cn812:/$ exit
exit
[trandles@cn812 ~]$
- Make changes to ch-tar2dir so that it does some inspection and sanity checking of the tarball it's unpacking to make sure it's not nesting directories, leading to the ch-run error. For instance, if ch-tar2dir unpacks a tarball and there's no etc directory in the same directory that it places WEIRD_AL_YANKOVIC, there's a problem. Alternatively ch-tar2dir could search the unpacked tarball for WEIRD_AL_YANKOVIC and throw an error if it's in an unexpected location.
- Write a utility like "ch-dir2tar" that can be used to tar up an image in a sane way that should untar as expected.
Tim