Quantcast

2.1.19 testing failures on the debian build daemons

classic Classic list List threaded Threaded
29 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
Hey all--

the debian build daemons are all failing to build 2.1.19-1 (which
includes many post-release bugfix patches, on top of 2.1.19 itself):

https://buildd.debian.org/status/logs.php?pkg=gnupg2&ver=2.1.19-1&suite=experimental

The failures are in the test suite.

(the one visible success -- m68k -- is a build daemon that deliberately
doesn't run the test suites)

An example of the failures is:

make[3]: Leaving directory '/«PKGBUILDDIR»/build/tests/gpgscm'
Making check in openpgp
make[3]: Entering directory '/«PKGBUILDDIR»/build/tests/openpgp'
LC_ALL=C EXEEXT= PATH=../gpgscm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games srcdir=/«PKGBUILDDIR»/build/../tests/openpgp objdir=/«PKGBUILDDIR»/build GPGSCM_PATH=/«PKGBUILDDIR»/build/../tests/gpgscm:/«PKGBUILDDIR»/build/../tests/openpgp /«PKGBUILDDIR»/build/tests/gpgscm/gpgscm \
  run-tests.scm  version.scm enarmor.scm mds.scm decrypt.scm decrypt-multifile.scm decrypt-dsa.scm decrypt-session-key.scm sigs.scm sigs-dsa.scm encrypt.scm encrypt-multifile.scm encrypt-dsa.scm compression.scm seat.scm clearsig.scm encryptp.scm detach.scm detachm.scm armsigs.scm armencrypt.scm armencryptp.scm signencrypt.scm signencrypt-dsa.scm armsignencrypt.scm armdetach.scm armdetachm.scm genkey1024.scm conventional.scm conventional-mdc.scm multisig.scm verify.scm verify-multifile.scm gpgv-forged-keyring.scm armor.scm import.scm import-revocation-certificate.scm ecc.scm 4gb-packet.scm tofu.scm gpgtar.scm use-exact-key.scm default-key.scm export.scm ssh-import.scm ssh-export.scm quick-key-manipulation.scm key-selection.scm delete-keys.scm gpgconf.scm issue2015.scm issue2346.scm issue2417.scm issue2419.scm issue2929.scm issue2941.scm
("/«PKGBUILDDIR»/build/tools/gpg-connect-agent" --verbose "--agent-program=/«PKGBUILDDIR»/build/agent/gpg-agent|--debug-quick-random" /bye) failed: ("gpg-connect-agent: no running gpg-agent - starting '/«PKGBUILDDIR»/build/agent/gpg-agent|--debug-quick-random'\ngpg-connect-agent: waiting for the agent to come up ... (5s)\ngpg-connect-agent: waiting for the agent to come up ... (4s)\ngpg-connect-agent: waiting for the agent to come up ... (3s)\ngpg-connect-agent: waiting for the agent to come up ... (2s)\ngpg-connect-agent: waiting for the agent to come up ... (1s)\ngpg-connect-agent: can't connect to the agent: File name too long\ngpg-connect-agent: error sending standard options: No agent running\n")
FAIL: setup.scm
("/«PKGBUILDDIR»/build/tools/gpg-connect-agent" --verbose "--agent-program=/«PKGBUILDDIR»/build/agent/gpg-agent|--debug-quick-random" /bye) failed: ("gpg-connect-agent: no running gpg-agent - starting '/«PKGBUILDDIR»/build/agent/gpg-agent|--debug-quick-random'\ngpg-connect-agent: waiting for the agent to come up ... (5s)\ngpg-connect-agent: waiting for the agent to come up ... (4s)\ngpg-connect-agent: waiting for the agent to come up ... (3s)\ngpg-connect-agent: waiting for the agent to come up ... (2s)\ngpg-connect-agent: waiting for the agent to come up ... (1s)\ngpg-connect-agent: can't connect to the agent: IPC connect call failed\ngpg-connect-agent: error sending standard options: No agent running\n")
FAIL: version.scm
("/«PKGBUILDDIR»/build/tools/gpgtar" --extract --directory=. "/«PKGBUILDDIR»/build/tests/openpgp/gpgscm-20170317T084956-run-tests-UA2ct9/gpgscm-20170317T084956-run-tests-SYHm3M/environment-cache") failed: ("gpgtar: error opening '/«PKGBUILDDIR»/build/tests/openpgp/gpgscm-20170317T084956-run-tests-UA2ct9/gpgscm-20170317T084956-run-tests-SYHm3M/environment-cache': No such file or directory\n")
0: tests.scm:133: (throw (string-append (stringify what) " failed") (:stderr result))
1: defs.scm:435: (call-check `(,(tool 'gpgtar) --extract --directory=. ,(cadr *args*)))

unpacking those first failures, it looks like "gpg-connect-agent" call
is failing, because the agent fails to start the gpg-agent daemon.

in this case, the path for the tests in question is:

  /build/gnupg2-K_EEBD/gnupg2-2.1.19/build/tests/openpgp

which should be short enough if it were used as a homedir, though i
think setup.scm uses an ephemeral homedir -- whose path i haven't
tracked down yet.

the version.scm failure shows:

 gpgtar: error opening
 '/build/gnupg2-K_EEBD/gnupg2-2.1.19/build/tests/openpgp/gpgscm-20170317T084956-run-tests-UA2ct9/gpgscm-20170317T084956-run-tests-SYHm3M/environment-cache':
 No such file or directory

if that's the test path, then it certainly is longer than sun-path.  I
don't understand why there would be two nested directories named
gpgscm-$TIMESTAMP-run-tests-$SUFFIX though.

Doing the build myself in a normal environment *doesn't* have these
failures, but building in a pbuilder/cowbuilder environment *does* show
them.

Any suggestions?

Sorry to keep raising these build process hiccups.  clearly the build
process and test suite are more brittle than we'd like them to be.  are
there checks we should be doing up front in the test process to detect
whether the local system is one we can actually run the tests on?

    --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Justus Winter
Hi :)

Daniel Kahn Gillmor <[hidden email]> writes:

> Hey all--
>
> the debian build daemons are all failing to build 2.1.19-1 (which
> includes many post-release bugfix patches, on top of 2.1.19 itself):
>
> https://buildd.debian.org/status/logs.php?pkg=gnupg2&ver=2.1.19-1&suite=experimental
>
> The failures are in the test suite.
>
> (the one visible success -- m68k -- is a build daemon that deliberately
> doesn't run the test suites)
>
> An example of the failures is:
>
> make[3]: Leaving directory '/«PKGBUILDDIR»/build/tests/gpgscm'
> Making check in openpgp
> make[3]: Entering directory '/«PKGBUILDDIR»/build/tests/openpgp'
> LC_ALL=C EXEEXT= PATH=../gpgscm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games srcdir=/«PKGBUILDDIR»/build/../tests/openpgp objdir=/«PKGBUILDDIR»/build GPGSCM_PATH=/«PKGBUILDDIR»/build/../tests/gpgscm:/«PKGBUILDDIR»/build/../tests/openpgp /«PKGBUILDDIR»/build/tests/gpgscm/gpgscm \
>   run-tests.scm  version.scm enarmor.scm mds.scm decrypt.scm decrypt-multifile.scm decrypt-dsa.scm decrypt-session-key.scm sigs.scm sigs-dsa.scm encrypt.scm encrypt-multifile.scm encrypt-dsa.scm compression.scm seat.scm clearsig.scm encryptp.scm detach.scm detachm.scm armsigs.scm armencrypt.scm armencryptp.scm signencrypt.scm signencrypt-dsa.scm armsignencrypt.scm armdetach.scm armdetachm.scm genkey1024.scm conventional.scm conventional-mdc.scm multisig.scm verify.scm verify-multifile.scm gpgv-forged-keyring.scm armor.scm import.scm import-revocation-certificate.scm ecc.scm 4gb-packet.scm tofu.scm gpgtar.scm use-exact-key.scm default-key.scm export.scm ssh-import.scm ssh-export.scm quick-key-manipulation.scm key-selection.scm delete-keys.scm gpgconf.scm issue2015.scm issue2346.scm issue2417.scm issue2419.scm issue2929.scm issue2941.scm
> ("/«PKGBUILDDIR»/build/tools/gpg-connect-agent" --verbose "--agent-program=/«PKGBUILDDIR»/build/agent/gpg-agent|--debug-quick-random" /bye) failed: ("gpg-connect-agent: no running gpg-agent - starting '/«PKGBUILDDIR»/build/agent/gpg-agent|--debug-quick-random'\ngpg-connect-agent: waiting for the agent to come up ... (5s)\ngpg-connect-agent: waiting for the agent to come up ... (4s)\ngpg-connect-agent: waiting for the agent to come up ... (3s)\ngpg-connect-agent: waiting for the agent to come up ... (2s)\ngpg-connect-agent: waiting for the agent to come up ... (1s)\ngpg-connect-agent: can't connect to the agent: File name too long\ngpg-connect-agent: error sending standard options: No agent running\n")
> FAIL: setup.scm
I bet gpgconf --create-socketdir fails.  Can you arrange make check to
be invoked with verbose=3?

(I'll change this to emit an error of course.)

> the version.scm failure shows:
>
>  gpgtar: error opening
>  '/build/gnupg2-K_EEBD/gnupg2-2.1.19/build/tests/openpgp/gpgscm-20170317T084956-run-tests-UA2ct9/gpgscm-20170317T084956-run-tests-SYHm3M/environment-cache':
>  No such file or directory

That is the tarball that setup.scm failed to create.

> if that's the test path, then it certainly is longer than sun-path.  I
> don't understand why there would be two nested directories named
> gpgscm-$TIMESTAMP-run-tests-$SUFFIX though.

It is not the test path, but we now create temporary directories below
<gnupg-obj>/tests/openpgp, so they are longer now.


Justus

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
Hi Justus--

On Fri 2017-03-17 15:09:59 -0400, Justus Winter wrote:
> Daniel Kahn Gillmor <[hidden email]> writes:
>
>> the debian build daemons are all failing to build 2.1.19-1 (which
>> includes many post-release bugfix patches, on top of 2.1.19 itself):
>
> I bet gpgconf --create-socketdir fails.  Can you arrange make check to
> be invoked with verbose=3?

sure, will do.

I'm quite sure that --create-socketdir fails, fwiw.  I just tested this
on a user account which had a deliberately-locked-out /run/user/$(id -u)
directory.  That test failed in the same way.  if you care, trying to
run these commands directly looks like this:

0 abc123@testsystem:~$ gpgconf --create-socketdir
gpgconf: socketdir is '/home/abc123/.gnupg'
gpgconf: general error
gpgconf: bad permissions
gpgconf: using homedir as fallback
gpgconf: error creating socket directory
gpgconf: fatal error (exit status 1)
1 abc123@testsystem:~$ GNUPGHOME=$(mktemp -d) gpgconf --create-socketdir
gpgconf: socketdir is '/tmp/tmp.CHRspkaiJX'
gpgconf: general error
gpgconf: bad permissions
gpgconf: using homedir as fallback
gpgconf: error creating socket directory
gpgconf: fatal error (exit status 1)
1 abc123@testsystem:~$


I believe that the debian build daemons are not guaranteed to have
/run/user/$(id -u) available during the build process.  The build may be
done inside of a chroot, or in some other particularly constrained
container that does not guarantee the presence of /run at all.

So i'm wondering what you think i should do about this.  I see a few
options (i'd be happy to entertain others if you can suggest them):

 a) i can try to update the debian builder infrastructure to always
    supply /run/user/$(id -u) as part of any package build process

 b) the GnuPG source can not require it as a part of the test suite

 c) i can just have the GnuPG package skip the test suite entirely.

 d) i can configure the GnuPG package to deliberately skip the parts of
    the test suite that require an active and present /run/user/$(id -u)


(a) is likely to take quite a long time (if it succeeds at all; i don't
know what kinds of pushback i'm likely to get).  If this is the desired
outcome, it may be a while before a new version of GnuPG is buildable on
debian's infrastructure.

(c) is easiest for me to do, but it sounds like a terrible plan -- the
test suite is a great thing for the GnuPG project, and i like the idea
of running it in as many environments as possible.

So i think that (b) would be my preference, though if there was a
straightforward way to do (d) i'd be willing to accept it as an interim
step.

        --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Werner Koch
On Sun, 19 Mar 2017 23:48, [hidden email] said:

> 0 abc123@testsystem:~$ gpgconf --create-socketdir
> gpgconf: socketdir is '/home/abc123/.gnupg'
> gpgconf: general error

This is probably a failed stat(2).  That can be expected if the /run
directy etc does not exits ...

An strace would be very helpful to see exactly what is going wrong.

>  d) i can configure the GnuPG package to deliberately skip the parts of

   e) We fix how gpgconf is used.  Given that a GNUGHOME is set the
      regular code to locate the socketdir will use GNUPGHOME which is
      what we need.  Only gpgconf --create-socketdir is picky about
      permissions etc. and tells the user this.  I think this is correct
      behaviour because the user requested to create a socketdir.


Salam-Shalom,

   Werner


--
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

attachment0 (233 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
On Mon 2017-03-20 05:34:06 -0400, Werner Koch wrote:

> On Sun, 19 Mar 2017 23:48, [hidden email] said:
>
>> 0 abc123@testsystem:~$ gpgconf --create-socketdir
>> gpgconf: socketdir is '/home/abc123/.gnupg'
>> gpgconf: general error
>
> This is probably a failed stat(2).  That can be expected if the /run
> directy etc does not exits ...
>
> An strace would be very helpful to see exactly what is going wrong.
Attached is an strace of:

  GNUPGHOME=$(mktemp -d) gpgconf --create-socketdir

>>  d) i can configure the GnuPG package to deliberately skip the parts of
>
>    e) We fix how gpgconf is used.  Given that a GNUGHOME is set the
>       regular code to locate the socketdir will use GNUPGHOME which is
>       what we need.  Only gpgconf --create-socketdir is picky about
>       permissions etc. and tells the user this.  I think this is correct
>       behaviour because the user requested to create a socketdir.

Thanks for this suggestion!  I'm hoping that the test suite would work
in either situation:

 a) where the build path is short enough, or
 b) where /run/user/$(id -u) is present and writable

and only fail when both conditions are not met.

I've tested it in a situation where the build path is insanely long, but
where /run/user/$(id -u) is available, and the test suite completes
fine.

So it's just the other way around that causes the tests to fail.  can
you suggest a patch?

thanks for the followup,

    --dkg


14661 10:21:44.753427 execve("/usr/bin/gpgconf", ["gpgconf", "--create-socketdir"], [/* 22 vars */]) = 0 <0.001464>
14661 10:21:44.755574 brk(NULL)         = 0x55c7a18be000 <0.000084>
14661 10:21:44.755868 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000061>
14661 10:21:44.756034 mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb760e07000 <0.000021>
14661 10:21:44.756120 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory) <0.000017>
14661 10:21:44.756284 open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 <0.000026>
14661 10:21:44.756357 fstat(3, {st_mode=S_IFREG|0644, st_size=209040, ...}) = 0 <0.000014>
14661 10:21:44.756420 mmap(NULL, 209040, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb760dd3000 <0.000019>
14661 10:21:44.756481 close(3)          = 0 <0.000012>
14661 10:21:44.756543 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000256>
14661 10:21:44.756870 open("/lib/x86_64-linux-gnu/libgcrypt.so.20", O_RDONLY|O_CLOEXEC) = 3 <0.000023>
14661 10:21:44.756938 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300\254\0\0\0\0\0\0"..., 832) = 832 <0.000017>
14661 10:21:44.757003 fstat(3, {st_mode=S_IFREG|0644, st_size=1108088, ...}) = 0 <0.000013>
14661 10:21:44.757064 mmap(NULL, 3204352, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb7608d8000 <0.000020>
14661 10:21:44.757125 mprotect(0x7fb7609df000, 2093056, PROT_NONE) = 0 <0.000030>
14661 10:21:44.757195 mmap(0x7fb760bde000, 36864, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x106000) = 0x7fb760bde000 <0.000027>
14661 10:21:44.757283 close(3)          = 0 <0.000013>
14661 10:21:44.757346 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000016>
14661 10:21:44.757411 open("/lib/x86_64-linux-gnu/libgpg-error.so.0", O_RDONLY|O_CLOEXEC) = 3 <0.000020>
14661 10:21:44.757475 read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300*\0\0\0\0\0\0"..., 832) = 832 <0.000015>
14661 10:21:44.757534 fstat(3, {st_mode=S_IFREG|0644, st_size=84032, ...}) = 0 <0.000013>
14661 10:21:44.757593 mmap(NULL, 2179304, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb7606c3000 <0.000019>
14661 10:21:44.757653 mprotect(0x7fb7606d7000, 2093056, PROT_NONE) = 0 <0.000022>
14661 10:21:44.757714 mmap(0x7fb7608d6000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x13000) = 0x7fb7608d6000 <0.000021>
14661 10:21:44.757790 close(3)          = 0 <0.000013>
14661 10:21:44.757853 access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory) <0.000043>
14661 10:21:44.757963 open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3 <0.000020>
14661 10:21:44.758032 read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\3\2\0\0\0\0\0"..., 832) = 832 <0.000015>
14661 10:21:44.758094 fstat(3, {st_mode=S_IFREG|0755, st_size=1685264, ...}) = 0 <0.000013>
14661 10:21:44.758159 mmap(NULL, 3791264, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7fb760325000 <0.000019>
14661 10:21:44.758218 mprotect(0x7fb7604ba000, 2093056, PROT_NONE) = 0 <0.000024>
14661 10:21:44.758281 mmap(0x7fb7606b9000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x194000) = 0x7fb7606b9000 <0.000024>
14661 10:21:44.758354 mmap(0x7fb7606bf000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7fb7606bf000 <0.000019>
14661 10:21:44.758423 close(3)          = 0 <0.000013>
14661 10:21:44.758513 mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7fb760dd1000 <0.000016>
14661 10:21:44.758579 arch_prctl(ARCH_SET_FS, 0x7fb760dd1700) = 0 <0.000013>
14661 10:21:44.758741 mprotect(0x7fb7606b9000, 16384, PROT_READ) = 0 <0.000022>
14661 10:21:44.758857 mprotect(0x7fb7608d6000, 4096, PROT_READ) = 0 <0.000018>
14661 10:21:44.759009 mprotect(0x7fb760bde000, 8192, PROT_READ) = 0 <0.000019>
14661 10:21:44.759204 mprotect(0x55c7a111a000, 8192, PROT_READ) = 0 <0.000019>
14661 10:21:44.759267 mprotect(0x7fb760e0a000, 4096, PROT_READ) = 0 <0.000022>
14661 10:21:44.759328 munmap(0x7fb760dd3000, 209040) = 0 <0.000036>
14661 10:21:44.759555 brk(NULL)         = 0x55c7a18be000 <0.000014>
14661 10:21:44.759609 brk(0x55c7a18df000) = 0x55c7a18df000 <0.000015>
14661 10:21:44.759701 fstat(0, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 32), ...}) = 0 <0.000015>
14661 10:21:44.759771 fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 32), ...}) = 0 <0.000012>
14661 10:21:44.759829 fstat(2, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 32), ...}) = 0 <0.000012>
14661 10:21:44.759923 open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3 <0.000023>
14661 10:21:44.760021 fstat(3, {st_mode=S_IFREG|0644, st_size=3251664, ...}) = 0 <0.000013>
14661 10:21:44.760099 mmap(NULL, 3251664, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb76000b000 <0.000019>
14661 10:21:44.760169 close(3)          = 0 <0.000013>
14661 10:21:44.760317 access("/etc/gcrypt/fips_enabled", F_OK) = -1 ENOENT (No such file or directory) <0.000049>
14661 10:21:44.761190 open("/proc/sys/crypto/fips_enabled", O_RDONLY) = 3 <0.000053>
14661 10:21:44.761356 fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0 <0.000013>
14661 10:21:44.761431 read(3, "0\n", 1024) = 2 <0.000018>
14661 10:21:44.761495 close(3)          = 0 <0.000015>
14661 10:21:44.761550 open("/etc/gcrypt/hwf.deny", O_RDONLY) = -1 ENOENT (No such file or directory) <0.000016>
14661 10:21:44.762251 getuid()          = 1003 <0.000014>
14661 10:21:44.762330 stat("/run/user/1003", {st_mode=S_IFDIR|000, st_size=100, ...}) = 0 <0.000020>
14661 10:21:44.762401 getuid()          = 1003 <0.000012>
14661 10:21:44.762449 getuid()          = 1003 <0.000011>
14661 10:21:44.762496 stat("/run/user/1003", {st_mode=S_IFDIR|000, st_size=100, ...}) = 0 <0.000014>
14661 10:21:44.762555 getuid()          = 1003 <0.000011>
14661 10:21:44.762601 stat("/run/user/1003/gnupg", 0x7ffc5fe3d570) = -1 EACCES (Permission denied) <0.000014>
14661 10:21:44.762687 write(2, "gpgconf: socketdir is '/tmp/tmp."..., 42) = 42 <0.000046>
14661 10:21:44.762814 write(2, "'\n", 2) = 2 <0.000043>
14661 10:21:44.762949 write(2, "gpgconf: ", 9) = 9 <0.000035>
14661 10:21:44.763056 write(2, "\tgeneral error\n", 15) = 15 <0.000032>
14661 10:21:44.763187 write(2, "gpgconf: ", 9) = 9 <0.000028>
14661 10:21:44.763287 write(2, "\tbad permissions\n", 17) = 17 <0.000043>
14661 10:21:44.763405 write(2, "gpgconf: ", 9) = 9 <0.000039>
14661 10:21:44.763508 write(2, "\tusing homedir as fallback\n", 27) = 27 <0.000031>
14661 10:21:44.763596 write(2, "gpgconf: error creating socket d"..., 40) = 40 <0.000024>
14661 10:21:44.763670 write(2, "\n", 1) = 1 <0.000023>
14661 10:21:44.763749 write(2, "gpgconf: fatal error (exit statu"..., 35) = 35 <0.000023>
14661 10:21:44.763822 write(2, ")\n", 2) = 2 <0.000024>
14661 10:21:44.763909 exit_group(1)     = ?
14661 10:21:44.764204 +++ exited with 1 +++

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Justus Winter
Daniel Kahn Gillmor <[hidden email]> writes:

> Thanks for this suggestion!  I'm hoping that the test suite would work
> in either situation:
>
>  a) where the build path is short enough, or

There is no such thing as a short enough build path.  The days of the
test suite using obj/tests/openpgp as GNUPGHOME are long gone.  We are
creating one GNUPGHOME per test, sometimes a test creates ephemeral home
diretories on its own.

> 14661 10:21:44.762330 stat("/run/user/1003", {st_mode=S_IFDIR|000, st_size=100, ...}) = 0 <0.000020>

/run/user/1003 is not writable.


Justus

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
On Mon 2017-03-20 12:04:39 -0400, Justus Winter wrote:

> Daniel Kahn Gillmor <[hidden email]> writes:
>
>> Thanks for this suggestion!  I'm hoping that the test suite would work
>> in either situation:
>>
>>  a) where the build path is short enough, or
>
> There is no such thing as a short enough build path.  The days of the
> test suite using obj/tests/openpgp as GNUPGHOME are long gone.  We are
> creating one GNUPGHOME per test, sometimes a test creates ephemeral home
> diretories on its own.
>
>> 14661 10:21:44.762330 stat("/run/user/1003", {st_mode=S_IFDIR|000, st_size=100, ...}) = 0 <0.000020>
>
> /run/user/1003 is not writable.
yes, i know.  Some build daemons, which use extremely minimalist chroots
that don't even have /run at all, let alone a writable /run/user/1003 or
whatever.  Should the test suites fail in those cases?  if so, what
do you think i should do about getting newer versions of GnuPG in
debian?  If not, what needs to change?

         --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
In reply to this post by Daniel Kahn Gillmor-7
On Sun 2017-03-19 18:48:59 -0400, Daniel Kahn Gillmor wrote:
> On Fri 2017-03-17 15:09:59 -0400, Justus Winter wrote:
>> Daniel Kahn Gillmor <[hidden email]> writes:
>> I bet gpgconf --create-socketdir fails.  Can you arrange make check to
>> be invoked with verbose=3?
>
> sure, will do.

2.1.19-2 includes the latest patches from the master branch, and has
verbose=3 set in the test suite.  You can see the failures and the logs
on different architectures linked from here:

  https://buildd.debian.org/status/logs.php?pkg=gnupg2&ver=2.1.19-2&suite=experimental

Let me know if you've got suggestions for how to proceed.

Thanks,

        --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Justus Winter
In reply to this post by Daniel Kahn Gillmor-7
Daniel Kahn Gillmor <[hidden email]> writes:

> On Mon 2017-03-20 12:04:39 -0400, Justus Winter wrote:
>> Daniel Kahn Gillmor <[hidden email]> writes:
>>
>>> Thanks for this suggestion!  I'm hoping that the test suite would work
>>> in either situation:
>>>
>>>  a) where the build path is short enough, or
>>
>> There is no such thing as a short enough build path.  The days of the
>> test suite using obj/tests/openpgp as GNUPGHOME are long gone.  We are
>> creating one GNUPGHOME per test, sometimes a test creates ephemeral home
>> diretories on its own.
>>
>>> 14661 10:21:44.762330 stat("/run/user/1003", {st_mode=S_IFDIR|000, st_size=100, ...}) = 0 <0.000020>
>>
>> /run/user/1003 is not writable.
>
> yes, i know.  Some build daemons, which use extremely minimalist chroots
> that don't even have /run at all, let alone a writable /run/user/1003 or
> whatever.  Should the test suites fail in those cases?  if so, what
> do you think i should do about getting newer versions of GnuPG in
> debian?  If not, what needs to change?
The test suite now uses longer gnupghomes.  This is a deliberate choice
I made.  Let me clarify my position.

First of all, I must stress that this is not a bug in the test suite.
The documentation does not state anywhere that gnupghomes must be "short
enough" (i.e. strlen (gnupghome) <= sizeof sun_path - 1 ('/') - max
strlen of any socket gpg uses - 1 ('\0')).  Furthermore, GnuPG does not
enforce such a restriction.  So until this changes, I consider the paths
the test suite uses slightly odd but valid.

We are not going to paper over any issues in GnuPG just to make the
tests happy.  Doing so defeats the purpose of testing the software, and
merely hides the real problem, making it pop up at our downstream users.

(E.g. https://notmuchmail.org/pipermail/notmuch/2017/024148.html)

GnuPG 2.x unreasonably restricting the length of gnupghomes, either by
documenting it or simply by failing, is a regression from GnuPG 1.4.

The test suite now relies on 'gpgconf --create-socketdir' to make
arbitrary gnupghomes work.

I believe that the problem of restricting the length of gnupghomes is
orthogonal to what 'gpgconf --create-socketdir' solves (even though
indeed separate socket directories solve it as a side effect), but I
have been unable to convince Werner of that.  So now we have to make
sure that 'gpgconf --create-socketdir' actually works everywhere and
robustly solves this problem too.


Justus

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Kristian Fiskerstrand-6
On 03/21/2017 10:24 AM, Justus Winter wrote:
> The test suite now relies on 'gpgconf --create-socketdir' to make
> arbitrary gnupghomes work.

Seems like this introduces a requirement on the systemd behavior with
/run/user/${UID}/gnupg that may or may not be present in other systems?

>
> I believe that the problem of restricting the length of gnupghomes is
> orthogonal to what 'gpgconf --create-socketdir' solves (even though
> indeed separate socket directories solve it as a side effect), but I
> have been unable to convince Werner of that.  So now we have to make
> sure that 'gpgconf --create-socketdir' actually works everywhere and
> robustly solves this problem too.


--
----------------------------
Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk
----------------------------
Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
----------------------------
"Be a yardstick of quality. Some people aren't used to an environment
where excellence is expected."
(Steve Jobs)


_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Werner Koch
In reply to this post by Justus Winter
On Tue, 21 Mar 2017 10:24, [hidden email] said:

> The test suite now uses longer gnupghomes.  This is a deliberate choice

I doubt that the need for long names for gnupghome just for testing is
justified.  In fact it is important to run test on a _local_ file
system.  This is the reason why switched to /var/run.  Now, when we
encountered problems creating a gnupg socket directory below /var/run we
should do the sensible thing, which is to fallback to standard Unix
behavior of having /tmp mounted locally.

> I believe that the problem of restricting the length of gnupghomes is
> orthogonal to what 'gpgconf --create-socketdir' solves (even though

We are using sockets for communication between the parts of GnuPG since
2002.  That are 15 years with only little trouble on some platforms.
The limited length of a socket file has never been a problem in
practice.  Well, things sometimes didn't build out of the box, but
moving the build to tmp has always been an option which has been in
active use for more than a decade, for example on some AIX
installations.

Note also that even GnuPG 1.x versions made use of sockets: For the EGD
and the Quintuple-agent (passphrase caching before gnupg 2.0)

ssh-agent also uses sockets without practical problems.  Bind and other
daemons have been using sockets for control operations for even longer
than ssh exists.  Stevens has enough example on how to use sockets and
warns about the limitations.  ustar, the only common tar standard, limit
the lengths of file names as well.

Please come up with a practical fix.


Shalom-Salam,

   Werner

--
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

attachment0 (233 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Justus Winter
Werner Koch <[hidden email]> writes:

> Please come up with a practical fix.

Fixed in 06f1f163e96f1039304fd3cf565cf9de1ca45849.


In the interest of moving the discussion forward, can we now agree on
the fact that 'gpgconf --create-socketdir' is not a sufficient solution
to the problem of overly long socket names?


Justus

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (497 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Werner Koch
On Tue, 21 Mar 2017 13:22, [hidden email] said:

> In the interest of moving the discussion forward, can we now agree on
> the fact that 'gpgconf --create-socketdir' is not a sufficient solution
> to the problem of overly long socket names?

Nope.  See also my comments on bug 2964.


Shalom-Salam,

   Werner

--
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

attachment0 (233 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
In reply to this post by Justus Winter
On Tue 2017-03-21 08:22:41 -0400, Justus Winter wrote:
> Werner Koch <[hidden email]> writes:
>
>> Please come up with a practical fix.
>
> Fixed in 06f1f163e96f1039304fd3cf565cf9de1ca45849.

thanks for this, Justus!  2.1.19-3 (which includes many of the
post-release patches after 2.1.19) is now building again on the debian
build daemons:

  https://buildd.debian.org/status/logs.php?pkg=gnupg2&ver=2.1.19-3&suite=experimental

I'll let that settle in for a day or two and then consider uploading it
to unstable, since it fixes several bugs from 2.1.18.

> In the interest of moving the discussion forward, can we now agree on
> the fact that 'gpgconf --create-socketdir' is not a sufficient solution
> to the problem of overly long socket names?

So we have a few different situations that make predictable socket
locations troubling in a universal sense.  Every possible candidate
location has at least one problem:

 a) $GNUPGHOME/S.gpg-agent : this might have a particularly long name,
    one that exceeds sun_path length.

 b) $GNUPGHOME/S.gpg-agent : this might be on a remote filesystem or a
    filesystem that cannot support unix-domain sockets (e.g. fat32 on a
    thumbdrive)

 c) anywhere under /tmp : this is not a predictable location that is
    safe to use, and inherently can't be on most systems where /tmp is
    shared and world-writable.

 d) /run/user/$(id -u)/gnupg/S.gpg-agent : We know that this user-owned
    directory isn't guaranteed to be available in every environment,
    including (sadly) chroots that are used by the debian build
    infrastructure.

Justus's clever use of relative paths resolves issue (a) and avoids the
problems associated with (c) and (d), but falls afoul of (b).

Where /run/user exists, using /run/user/$(id -u)/gnupg resolves issues
a, b, and c.  But as (d) points out it doesn't exist everywhere.

If we combine Justus's use of relative paths with the use of
/run/user/$(id -u)/gnupg where possible, then the only corner case we
cannot handle is:

 * $GNUPGHOME is on a filesystem that cannot support unix-domain
   sockets, and /run/user does not exist.

If anyone has a suggestion for how to handle this corner case, i'm all
ears.

        --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Kristian Fiskerstrand-6
On 03/21/2017 09:46 PM, Daniel Kahn Gillmor wrote:
> If anyone has a suggestion for how to handle this corner case, i'm all
> ears.

Throw an error and point to part of documentation for how to use
%Assuan% socket= redirection?

--
----------------------------
Kristian Fiskerstrand
Blog: https://blog.sumptuouscapital.com
Twitter: @krifisk
----------------------------
Public OpenPGP keyblock at hkp://pool.sks-keyservers.net
fpr:94CB AFDD 3034 5109 5618 35AA 0B7F 8B60 E3ED FAE3
----------------------------
Nosce te ipsum!
Know thyself!


_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
On Tue 2017-03-21 16:52:32 -0400, Kristian Fiskerstrand wrote:
> On 03/21/2017 09:46 PM, Daniel Kahn Gillmor wrote:
>> If anyone has a suggestion for how to handle this corner case, i'm all
>> ears.
>
> Throw an error and point to part of documentation for how to use
> %Assuan% socket= redirection?

When would this happen?  During an attempt to start up such a daemon?
During an attempt to communicate with such a daemon?  I agree that
explicit error messages with suggestions for how to help are better than
failing cryptically, but we need to think about where the error comes
from and how it gets to the user.

     --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Peter Lebbing
In reply to this post by Daniel Kahn Gillmor-7
I hope my comments help with the perspective. If not, never mind.

On 21/03/17 21:46, Daniel Kahn Gillmor wrote:
>  a) $GNUPGHOME/S.gpg-agent : this might have a particularly long name,
>     one that exceeds sun_path length.

Has this actually ever been an issue for someone? I understand the
problem can be artificially induced, but has anyone ever reported
hitting the limit accidentally?

>  c) anywhere under /tmp : this is not a predictable location that is
>     safe to use, and inherently can't be on most systems where /tmp is
>     shared and world-writable.

What's the problem when it is a sticky directory and you create a
directory with mode 0700 under it? I think OpenSSH's agent uses it.

> If anyone has a suggestion for how to handle this corner case, i'm all
> ears.

If /tmp is safe after all, that seems like a good alternative.

Peter.

--
I use the GNU Privacy Guard (GnuPG) in combination with Enigmail.
You can send me encrypted mail if you want some privacy.
My key is available at <http://digitalbrains.com/2012/openpgp-key-peter>


_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (499 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Werner Koch
In reply to this post by Kristian Fiskerstrand-6
On Tue, 21 Mar 2017 21:52, [hidden email]
said:

> Throw an error and point to part of documentation for how to use
> %Assuan% socket= redirection?

I like to remove this hack in 2.3 - it requires too much admin work and
thus it is easier to simply create /run/user, for example in rc.local:

--8<---------------cut here---------------start------------->8---
[ ! -d /run/user ] && mkdir /run/user
awk -F: </etc/passwd '$3 >= 1000 && $3 < 65000 {print $3}' \
  | ( while read uid rest; do
        if [ ! -d "/run/user/$uid" ]; then
          mkdir /run/user/$uid
          chown $uid /run/user/$uid
          chmod 700 /run/user/$uid
        fi
      done )
--8<---------------cut here---------------end--------------->8---

BTW, the use of /tmp is just fine as long as you create a sub directory.
This is how X has always worked.  In fact, the test suite does just
this.

Note also that the socket file system problem exists on many NFS or
Samba mounted home directories.  Not just on FAT file systems.


Salam-Shalom,

   Werner


--
Die Gedanken sind frei.  Ausnahmen regelt ein Bundesgesetz.

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

attachment0 (233 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
In reply to this post by Peter Lebbing
On Tue 2017-03-21 17:28:36 -0400, Peter Lebbing wrote:
> I hope my comments help with the perspective. If not, never mind.
>
> On 21/03/17 21:46, Daniel Kahn Gillmor wrote:
>>  a) $GNUPGHOME/S.gpg-agent : this might have a particularly long name,
>>     one that exceeds sun_path length.
>
> Has this actually ever been an issue for someone? I understand the
> problem can be artificially induced, but has anyone ever reported
> hitting the limit accidentally?

Yes.  I've seen the problem get hit repeatedly by software that depends
on GnuPG, and has a test suite that happens to be reasonably scoped.
Usually GnuPG itself is masked by one or two other layers of software
(e.g. gpgme, perl's GnuPG::Interface, etc) so even extracting the error
messages in a sane way isn't straightforward.  it just looks like
arbitrary test suite failures for someone who builds their software in
/home/users/mary/dayjob/software/open-source/libfoo-3.22, but no
failures for the developer who builds in /home/amy/src/foo.

It's a real concern, and this kind of idiosyncratic failure frustrates
developers who are trying to depend on GnuPG.

>>  c) anywhere under /tmp : this is not a predictable location that is
>>     safe to use, and inherently can't be on most systems where /tmp is
>>     shared and world-writable.
>
> What's the problem when it is a sticky directory and you create a
> directory with mode 0700 under it? I think OpenSSH's agent uses it.

The key word in (c) is "predictable".

The problem is communicating the location of the socket to use to the
clients.  If i'm a client using some particular GNUPGHOME, how do i find
the sockets of the associated agent?  We've tried the environment
variable approach (which OpenSSH's ssh-agent uses) and found a series of
problems with that -- how do you communicate that variable to each
client if you only start the agent later?

Also, gpg 2.1.x has a one-gpg-agent-per-GNUPGHOME model (and
one-dirmngr-per-GNUPGHOME) today, thanks to the "standard-socket".  I
don't know whether having two gpg-agents or two dirmngrs interacting
with the same GNUPGHOME will run into contention.

> If /tmp is safe after all, that seems like a good alternative.

how do you solve the predictability problem?

    --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel

signature.asc (847 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 2.1.19 testing failures on the debian build daemons

Daniel Kahn Gillmor-7
In reply to this post by Werner Koch
On Tue 2017-03-21 17:37:37 -0400, Werner Koch wrote:

> On Tue, 21 Mar 2017 21:52, [hidden email]
> said:
>
>> Throw an error and point to part of documentation for how to use
>> %Assuan% socket= redirection?
>
> I like to remove this hack in 2.3 - it requires too much admin work and
> thus it is easier to simply create /run/user, for example in rc.local:
>
> --8<---------------cut here---------------start------------->8---
> [ ! -d /run/user ] && mkdir /run/user
> awk -F: </etc/passwd '$3 >= 1000 && $3 < 65000 {print $3}' \
>   | ( while read uid rest; do
>         if [ ! -d "/run/user/$uid" ]; then
>  mkdir /run/user/$uid
>  chown $uid /run/user/$uid
>  chmod 700 /run/user/$uid
> fi
>       done )
> --8<---------------cut here---------------end--------------->8---

Putting this in rc.local won't fix the situation where there's a chroot
(which is the case for many debian build daemons, and probably other
targeted/minimalist build environments)

> BTW, the use of /tmp is just fine as long as you create a sub directory.
> This is how X has always worked.  In fact, the test suite does just
> this.

The trouble with /tmp is figuring out which directory to use.  If
$GNUPGHOME itself is in /tmp, that's one thing -- the env var itself
points to a location that needs to be owned by the user.

But if GNUPGHOME is unset, or if it is not itself in /tmp, how are the
daemons and their client supposed to agree on a rendevous point in /tmp
?

> Note also that the socket file system problem exists on many NFS or
> Samba mounted home directories.  Not just on FAT file systems.

sure, that's what i described as point (b):

>>>  b) $GNUPGHOME/S.gpg-agent : this might be on a remote filesystem or a
>>>     filesystem that cannot support unix-domain sockets (e.g. fat32 on a
>>>     thumbdrive)

Regards,

        --dkg

_______________________________________________
Gnupg-devel mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-devel
12
Loading...