Quantcast

Unattended use of gpg across a wide range of gpg versions, Ubuntu edition. --debug-quick-random taking evasive action.

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Unattended use of gpg across a wide range of gpg versions, Ubuntu edition. --debug-quick-random taking evasive action.

Dan Kegel-2
tl;dr: anyone know what's up with --debug-quick-random?  Also, handy
script for unattended key generation across many versions of gpg.

Hi all.  This topic has been beaten to death on many forums and in many
bug reports, but here's a user report from the field that sums up what
works.  It's mostly just stitching together known workarounds, plus
one little mystery
with --debug-quick-random in gpg 2.1.15 (the one on Ubuntu 17.04).
I'll list the problems, then at the bottom show the full solution I'm using.

I'm writing a test script that uses gpg, so I reviewed
https://www.gnupg.org/documentation/manuals/gnupg/Unattended-Usage-of-GPG.html
but it doesn't quite handle all the situations I ran into.
This kind of test script has to satisfy requirements like:
- work on current OS as well as last few LTS releases
- use the OS's default gpg
- work in both interactive and headless situations
- leave the user's normal environment unchanged
- work even in deeply nested directories
That means I can't follow some of the advice in the manual (e.g. "use
GPGME" or "use --quick-addkey").

For the purposes of testing, let's say I want to generate a key with the command
   gpg --gen-key
for use with apt on an Ubuntu 17.04 desktop, as well as in freshly
installed headless older systems.
(For instance, containers created with the commands
  lxc-create -n ubu1204 -t download -- --dist ubuntu --release precise
--arch amd64
  lxc-create -n ubu1404 -t download -- --dist ubuntu --release trusty
--arch amd64
  lxc-create -n ubu1604 -t download -- --dist ubuntu --release xenial
--arch amd64
  lxc-create -n ubu1704 -t download -- --dist ubuntu --release zesty
--arch amd64 )
Easy, right?

Challenges and solutions I ran into, rearranged in a less embarassing
order than I ran into them:

0. Googling for solutions to problems finds stale or incomplete info
from random people
Solution: RTFM.  Really.  Go find *the manual* for gpg and read it.

1. Running a test script that creates keys affects user's keyring
Solution: follow
https://www.gnupg.org/documentation/manuals/gnupg/Ephemeral-home-directories.html
i.e. create a directory for the test, and set GNUPGHOME to the
absolute path to that dir
Works on all systems

2. 'gpg --gen-key' prompts user for key parameters, and aborts if
/dev/tty can't be opened (e.g. with noninteractive ssh )
Solution: follow
https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html
i.e. create a file foo.dat containing the responses, e.g.
Key-Type: 1
Key-Length: 2048
Subkey-Type: 1
Subkey-Length: 2048
Name-Real: My Real Name
Name-Email: [hidden email]
Expire-Date: 30
and change the command to 'gpg --batch --gen-key foo.dat'
Works on ubuntu 16.04 and below

3. On ubuntu 16.04, which straddles gpg and gpg2, the command
 'gpg --export | gpg2 --import -'
appears to be required to get apt to notice a key you've generated
with gpg, but 'gpg2 --import' aborts with
gpg: can't connect to the agent: Invalid value passed to IPC
gpg: error getting the KEK: No agent running

Solution: 'sudo apt-get install gnupg-agent', then
use "gpg-agent --daemon -- gpgcommand..." to create a transient
gpg-agent just for the duration of the gpg command.
This works on Ubuntu 12.04 through 16.04.

4. also on ubuntu 17.04, the previous fix isn't quite enough.
gpg-agent fails with
gpg-agent[1631]: command 'GENKEY' failed: Inappropriate ioctl for
device <Pinentry>
gpg: agent_genkey failed: Inappropriate ioctl for device
which sounds like https://dev.gnupg.org/T2680
Evidently it wants a tty, which isn't going to be possible.
Solution:
echo allow-loopback-pinentry > $GNUPGHOME/gpg-agent.conf
and add --pinentry-mode loopback to the gpg command.
This requires ubuntu 17.04 and up; you can't use it with ubuntu 12.04
through 16.04.

5. gpg hangs with message
Not enough random bytes available.  Please do some other work...
Solutions:
a) stuff the system rng somewhat securely; e.g. on Ubuntu, 'sudo
apt-get install haveged'
b) tell gpg to use an insecure RNG, e.g.
if gpg --quick-random --version >/dev/null 2>&1 ; then
    echo quick-random >> "$GNUPGHOME"/gpg.conf
elif gpg --debug-quick-random --version >/dev/null 2>&1 ; then
    echo debug-quick-random >> "$GNUPGHOME"/gpg.conf
fi
Either works on all tested ubuntu versions up to ubuntu 16.04.

6. On Ubuntu 17.04, gpg (2.1.15) takes several minutes to run, complaining
gpg-agent[6385]: can't connect my own socket: IPC connect call failed
gpg-agent[6385]: this process is useless - shutting down
even with --debug-quick-random in gpg.conf (or gpg-agent.conf).
Oddly, the same two workarounds fix this, more or less:
a) stuff the system rng somewhat securely; e.g. on Ubuntu, 'sudo
apt-get install haveged'
b) tell gpg-agent to use an insecure RNG; only way is to pass
--debug-quick-random option on gpg-agent's commandline!
Neither conf file will do anymore.  That socket error is very odd, and
so is the fact
that tweaking the rng in these two ways makes it go away.  Bug?  Feature?

7. When running tests in directories with long names, gpg aborts with
socket name '/some/long/path is too long
Solution: GNUPGHOME must be shorter than UNIX_PATH_MAX, which can be
as short as 94 bytes on some systems.
(Fixed in latest gpg, but you have to work around it for older ones.)

8. gpg might write to the tty at some point
Solution: as documented, add --no-tty options on all gpg calls or in conf file

Here's a consensus script for unattended key generation demonstrating
most of the workarounds.
--- snip ---
#!/bin/sh
set -x
set -e

# Check to see if gpg requires agent & supports loopback prompt
if gpg --version | head -n 1 | grep ' 2\.'
then
   gpg_use_loopback=true
else
   gpg_use_loopback=false
fi

# Check to see if ubuntu 16.04-specific workaround needed for apt
if test -x /usr/bin/gpg2
then
   gpg_copy_to_gpg2_needed=true
else
   gpg_copy_to_gpg2_needed=false
fi

# Avoid 'socket name too long' error with older gpg
GNUPGHOME=/tmp/gpg-isolation-demo-unique-and-obscure-path
export GNUPGHOME
rm -rf $GNUPGHOME
mkdir -m700 $GNUPGHOME

# 1st half of workaround to allow bypassing pinentry prompts; insecure-ish
if $gpg_use_loopback
then
    echo allow-loopback-pinentry > $GNUPGHOME/gpg-agent.conf
fi

# Never send output to tty
echo no-tty >> $GNUPGHOME/gpg.conf
# Never ask questions
echo batch >> $GNUPGHOME/gpg.conf

# Work around lack of entropy; we don't need it for this test.  Insecure.
QUICK=""
if gpg --quick-random --version >/dev/null 2>&1 ; then
    echo quick-random >> "$GNUPGHOME"/gpg.conf
elif gpg --debug-quick-random --version >/dev/null 2>&1 ; then
    echo debug-quick-random >> "$GNUPGHOME"/gpg.conf
    QUICK=--debug-quick-random
fi

keyemail=[hidden email]
cat > gpg.in.tmp <<_EOF_
Key-Type: 1
Key-Length: 2048
Subkey-Type: 1
Subkey-Length: 2048
Name-Real: My Real Name
Name-Email: $keyemail
Expire-Date: 30
_EOF_

if $gpg_use_loopback
then
    time gpg-agent $QUICK --daemon -- \
    gpg --pinentry-mode loopback --passphrase '' --gen-key gpg.in.tmp
else
    gpg --passphrase '' --gen-key gpg.in.tmp < /dev/null

    # Extra step only needed for ubuntu 16.04's apt?
    if $gpg_copy_to_gpg2_needed
    then
        gpg --passphrase '' --armor --export-secret-keys $keyemail \
        | gpg-agent --daemon gpg2 --passphrase "" --import -
    fi
fi
--- snip ---

_______________________________________________
Gnupg-users mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-users
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Unattended use of gpg across a wide range of gpg versions, Ubuntu edition. --debug-quick-random taking evasive action.

Dan Kegel-2
addendum: demo of how to delete a key unattended with gpg2
Documented in earlier thread,
http://marc.info/?l=gnupg-users&m=146287358008663&w=2

-- snip --
#!/bin/sh
# Script to demonstrate unattended creation, export, and deletion of a
secret key with gpg 2.x
set -ex

cat > test-script.sh << "_EOF_"
set -e
set -x
passphrase=""
gpg --batch --passphrase "$passphrase" --quick-gen-key 'test user
<[hidden email]>'

gpg --batch --passphrase "$passphrase" --pinentry-mode loopback
--export-secret-key --armor 'test user <[hidden email]>' > key.dat

# 1st fingerprint is for the primary, 2nd is for the secondary?
fingerprint=$(gpg -k --with-colons [hidden email] | awk -F: '/^fpr:/
{print $10}' | head -n 1)

gpg  --batch --passphrase "$passphrase" --pinentry-mode loopback --yes
--delete-secret-and-public-key $fingerprint
_EOF_
chmod +x test-script.sh

rm -rf /tmp/gpgtest-*
export GNUPGHOME=$(mktemp -d /tmp/gpgtest-XXXXXXX.tmp)
echo "allow-loopback-pinentry" > $GNUPGHOME/gpg-agent.conf
gpg-agent --daemon ./test-script.sh
rm -rf $GNUPGHOME
-- snip --

On Sat, Apr 29, 2017 at 9:14 PM, Dan Kegel <[hidden email]> wrote:

> tl;dr: anyone know what's up with --debug-quick-random?  Also, handy
> script for unattended key generation across many versions of gpg.
>
> Hi all.  This topic has been beaten to death on many forums and in many
> bug reports, but here's a user report from the field that sums up what
> works.  It's mostly just stitching together known workarounds, plus
> one little mystery
> with --debug-quick-random in gpg 2.1.15 (the one on Ubuntu 17.04).
> I'll list the problems, then at the bottom show the full solution I'm using.
>
> I'm writing a test script that uses gpg, so I reviewed
> https://www.gnupg.org/documentation/manuals/gnupg/Unattended-Usage-of-GPG.html
> but it doesn't quite handle all the situations I ran into.
> This kind of test script has to satisfy requirements like:
> - work on current OS as well as last few LTS releases
> - use the OS's default gpg
> - work in both interactive and headless situations
> - leave the user's normal environment unchanged
> - work even in deeply nested directories
> That means I can't follow some of the advice in the manual (e.g. "use
> GPGME" or "use --quick-addkey").
>
> For the purposes of testing, let's say I want to generate a key with the command
>    gpg --gen-key
> for use with apt on an Ubuntu 17.04 desktop, as well as in freshly
> installed headless older systems.
> (For instance, containers created with the commands
>   lxc-create -n ubu1204 -t download -- --dist ubuntu --release precise
> --arch amd64
>   lxc-create -n ubu1404 -t download -- --dist ubuntu --release trusty
> --arch amd64
>   lxc-create -n ubu1604 -t download -- --dist ubuntu --release xenial
> --arch amd64
>   lxc-create -n ubu1704 -t download -- --dist ubuntu --release zesty
> --arch amd64 )
> Easy, right?
>
> Challenges and solutions I ran into, rearranged in a less embarassing
> order than I ran into them:
>
> 0. Googling for solutions to problems finds stale or incomplete info
> from random people
> Solution: RTFM.  Really.  Go find *the manual* for gpg and read it.
>
> 1. Running a test script that creates keys affects user's keyring
> Solution: follow
> https://www.gnupg.org/documentation/manuals/gnupg/Ephemeral-home-directories.html
> i.e. create a directory for the test, and set GNUPGHOME to the
> absolute path to that dir
> Works on all systems
>
> 2. 'gpg --gen-key' prompts user for key parameters, and aborts if
> /dev/tty can't be opened (e.g. with noninteractive ssh )
> Solution: follow
> https://www.gnupg.org/documentation/manuals/gnupg/Unattended-GPG-key-generation.html
> i.e. create a file foo.dat containing the responses, e.g.
> Key-Type: 1
> Key-Length: 2048
> Subkey-Type: 1
> Subkey-Length: 2048
> Name-Real: My Real Name
> Name-Email: [hidden email]
> Expire-Date: 30
> and change the command to 'gpg --batch --gen-key foo.dat'
> Works on ubuntu 16.04 and below
>
> 3. On ubuntu 16.04, which straddles gpg and gpg2, the command
>  'gpg --export | gpg2 --import -'
> appears to be required to get apt to notice a key you've generated
> with gpg, but 'gpg2 --import' aborts with
> gpg: can't connect to the agent: Invalid value passed to IPC
> gpg: error getting the KEK: No agent running
>
> Solution: 'sudo apt-get install gnupg-agent', then
> use "gpg-agent --daemon -- gpgcommand..." to create a transient
> gpg-agent just for the duration of the gpg command.
> This works on Ubuntu 12.04 through 16.04.
>
> 4. also on ubuntu 17.04, the previous fix isn't quite enough.
> gpg-agent fails with
> gpg-agent[1631]: command 'GENKEY' failed: Inappropriate ioctl for
> device <Pinentry>
> gpg: agent_genkey failed: Inappropriate ioctl for device
> which sounds like https://dev.gnupg.org/T2680
> Evidently it wants a tty, which isn't going to be possible.
> Solution:
> echo allow-loopback-pinentry > $GNUPGHOME/gpg-agent.conf
> and add --pinentry-mode loopback to the gpg command.
> This requires ubuntu 17.04 and up; you can't use it with ubuntu 12.04
> through 16.04.
>
> 5. gpg hangs with message
> Not enough random bytes available.  Please do some other work...
> Solutions:
> a) stuff the system rng somewhat securely; e.g. on Ubuntu, 'sudo
> apt-get install haveged'
> b) tell gpg to use an insecure RNG, e.g.
> if gpg --quick-random --version >/dev/null 2>&1 ; then
>     echo quick-random >> "$GNUPGHOME"/gpg.conf
> elif gpg --debug-quick-random --version >/dev/null 2>&1 ; then
>     echo debug-quick-random >> "$GNUPGHOME"/gpg.conf
> fi
> Either works on all tested ubuntu versions up to ubuntu 16.04.
>
> 6. On Ubuntu 17.04, gpg (2.1.15) takes several minutes to run, complaining
> gpg-agent[6385]: can't connect my own socket: IPC connect call failed
> gpg-agent[6385]: this process is useless - shutting down
> even with --debug-quick-random in gpg.conf (or gpg-agent.conf).
> Oddly, the same two workarounds fix this, more or less:
> a) stuff the system rng somewhat securely; e.g. on Ubuntu, 'sudo
> apt-get install haveged'
> b) tell gpg-agent to use an insecure RNG; only way is to pass
> --debug-quick-random option on gpg-agent's commandline!
> Neither conf file will do anymore.  That socket error is very odd, and
> so is the fact
> that tweaking the rng in these two ways makes it go away.  Bug?  Feature?
>
> 7. When running tests in directories with long names, gpg aborts with
> socket name '/some/long/path is too long
> Solution: GNUPGHOME must be shorter than UNIX_PATH_MAX, which can be
> as short as 94 bytes on some systems.
> (Fixed in latest gpg, but you have to work around it for older ones.)
>
> 8. gpg might write to the tty at some point
> Solution: as documented, add --no-tty options on all gpg calls or in conf file
>
> Here's a consensus script for unattended key generation demonstrating
> most of the workarounds.
> --- snip ---
> #!/bin/sh
> set -x
> set -e
>
> # Check to see if gpg requires agent & supports loopback prompt
> if gpg --version | head -n 1 | grep ' 2\.'
> then
>    gpg_use_loopback=true
> else
>    gpg_use_loopback=false
> fi
>
> # Check to see if ubuntu 16.04-specific workaround needed for apt
> if test -x /usr/bin/gpg2
> then
>    gpg_copy_to_gpg2_needed=true
> else
>    gpg_copy_to_gpg2_needed=false
> fi
>
> # Avoid 'socket name too long' error with older gpg
> GNUPGHOME=/tmp/gpg-isolation-demo-unique-and-obscure-path
> export GNUPGHOME
> rm -rf $GNUPGHOME
> mkdir -m700 $GNUPGHOME
>
> # 1st half of workaround to allow bypassing pinentry prompts; insecure-ish
> if $gpg_use_loopback
> then
>     echo allow-loopback-pinentry > $GNUPGHOME/gpg-agent.conf
> fi
>
> # Never send output to tty
> echo no-tty >> $GNUPGHOME/gpg.conf
> # Never ask questions
> echo batch >> $GNUPGHOME/gpg.conf
>
> # Work around lack of entropy; we don't need it for this test.  Insecure.
> QUICK=""
> if gpg --quick-random --version >/dev/null 2>&1 ; then
>     echo quick-random >> "$GNUPGHOME"/gpg.conf
> elif gpg --debug-quick-random --version >/dev/null 2>&1 ; then
>     echo debug-quick-random >> "$GNUPGHOME"/gpg.conf
>     QUICK=--debug-quick-random
> fi
>
> keyemail=[hidden email]
> cat > gpg.in.tmp <<_EOF_
> Key-Type: 1
> Key-Length: 2048
> Subkey-Type: 1
> Subkey-Length: 2048
> Name-Real: My Real Name
> Name-Email: $keyemail
> Expire-Date: 30
> _EOF_
>
> if $gpg_use_loopback
> then
>     time gpg-agent $QUICK --daemon -- \
>     gpg --pinentry-mode loopback --passphrase '' --gen-key gpg.in.tmp
> else
>     gpg --passphrase '' --gen-key gpg.in.tmp < /dev/null
>
>     # Extra step only needed for ubuntu 16.04's apt?
>     if $gpg_copy_to_gpg2_needed
>     then
>         gpg --passphrase '' --armor --export-secret-keys $keyemail \
>         | gpg-agent --daemon gpg2 --passphrase "" --import -
>     fi
> fi
> --- snip ---

_______________________________________________
Gnupg-users mailing list
[hidden email]
http://lists.gnupg.org/mailman/listinfo/gnupg-users
Loading...