Gpg decryption fails(no secret key found) after upgrade from gpg4win 3.1.1->4.3.0

acornanomaly · February 9, 2024, 7:23am

So, I have a weird problem with a system I was using. I had installed the gpg4win 4.3.0 package on the system, and after importing the key I want to use, I am unable to decrypt files. It identifies the files as having been encrypted with my key’s subkey, but it still comes back as “no secret key found”.

After comparing to the system I have set up and working on another computer(which is using gpg4win 3.1.1), I decided to just remove everything 4.3.0 from the new system, install 3.1.1, and compare.

After installing 3.1.1, and importing my key, I am able to successfully decrypt:

C:\temp>gpg -vvv -d -o test.pdf test.pdf.pgp
gpg: using character set 'utf-8'
gpg: armor: BEGIN PGP MESSAGE
# off=0 ctb=c1 tag=1 hlen=2 plen=140 new-ctb
:pubkey enc packet: version 3, algo 1, keyid 681D100FD32A7C6B
        data: [1023 bits]
gpg: public key is 681D100FD32A7C6B
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: pinentry launched (9952 qt 1.1.1-beta5 - - -)
gpg: WARNING: cipher algorithm AES not found in recipient preferences
gpg: public key encrypted data: good DEK
# off=142 ctb=d2 tag=18 hlen=2 plen=0 partial new-ctb
:encrypted data packet:
        length: unknown
        mdc_method: 2
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: encrypted with 1024-bit RSA key, ID 681D100FD32A7C6B, created 2016-10-24
      "My Name <myname@company.net>"
gpg: AES encrypted data
# off=163 ctb=a3 tag=8 hlen=1 plen=0 indeterminate
:compressed packet: algo=2
# off=165 ctb=cb tag=11 hlen=2 plen=0 partial new-ctb
:literal data packet:
        mode b (62), created 1706894702, name="",
        raw data: unknown length
gpg: original file name=''
gpg: decryption okay

However, once I upgrade to 4.3.0, even though I haven’t touched the keys, it fails to decrypt the same file using the same key:

C:\temp>gpg -vvv -d -o test.pdf test.pdf.pgp
gpg: using character set 'utf-8'
gpg: enabled compatibility flags:
gpg: armor: BEGIN PGP MESSAGE
# off=0 ctb=c1 tag=1 hlen=2 plen=140 new-ctb
:pubkey enc packet: version 3, algo 1, keyid 681D100FD32A7C6B
        data: [1023 bits]
gpg: public key is 681D100FD32A7C6B
# off=142 ctb=d2 tag=18 hlen=2 plen=0 partial new-ctb
:encrypted data packet:
        length: unknown
        mdc_method: 2
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: encrypted with rsa1024 key, ID 681D100FD32A7C6B, created 2016-10-24
      "My Name <myname@company.net>"
gpg: public key decryption failed: No secret key
gpg: decryption failed: No secret key

I just don’t understand what I’m getting wrong, that the same key isn’t working after upgrade?

And it’s not pinentry, this still happens even if I completely remove the passphrase from the key, and I have another key on this same system that DOES work on the new version, even with pinentry.

cklassen · February 9, 2024, 7:42am

Maybe gpg looks into the wrong directory. You can check that with gpg --list-secret-keys. The first line of the output shows you where gpg is searching for the keys.

bernhard · February 9, 2024, 8:32am

As 3.1.1 is from 2018 and comes with GnuPG 2.2.7, some algorithms have been deprecated since then. Maybe you are running into one of those. This would explain why a never version would not deal with the old keys by default.

Check into the GPG Esoteric Options (Using the GNU Privacy Guard) and try options like --allow-old-cipher-algos or --allow-weak-digest-algos or --allow-weak-key-signatures for testing.

eebb · February 9, 2024, 10:17am

To understand whats going on I’d like to know what gpg --list-secret-keys looks like for the key in question

acornanomaly · February 10, 2024, 1:22am

bernhard, thank you for directing me to that page. I will be going through the options to see if I can find a discrepancy.

@cklassen and @eebb, the output of list-secret-keys appears to be the same. (I also include --version calls to demonstrate that they are different gpg versions.)

C:\temp\gpg>"C:\Program Files (x86)\GnuPG\bin\gpg.exe" --version
gpg (GnuPG) 2.4.4
libgcrypt 1.10.3
Copyright (C) 2024 g10 Code GmbH
License GNU GPL-3.0-or-later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: C:\Users\chris\AppData\Roaming\gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

C:\temp\gpg>"C:\Program Files (x86)\GnuPG\bin\gpg.exe" --list-secret-keys
C:\Users\chris\AppData\Roaming\gnupg\pubring.kbx
------------------------------------------------
sec   rsa1024 2016-10-24 [SC]
      D239F1F32C678261C2F381E70521D2004C296FA5
uid           [ unknown] My Name <myname@company.net>
ssb   rsa1024 2016-10-24 [ER]

And here’s gpg version 2.2.7(included with gpg4win 3.1.1):

C:\temp\gpg>C:\gpg-2.2.7\bin\gpg.exe --version
gpg (GnuPG) 2.2.7
libgcrypt 1.8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: C:/Users/chris/AppData/Roaming/gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
        CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2

C:\temp\gpg>C:\gpg-2.2.7\bin\gpg.exe --list-secret-keys
C:/Users/chris/AppData/Roaming/gnupg/pubring.kbx
------------------------------------------------
sec   rsa1024 2016-10-24 [SC]
      D239F1F32C678261C2F381E70521D2004C296FA5
uid           [ unknown] My Name <myname@company.net>
ssb   rsa1024 2016-10-24 [E]

bernhard · February 12, 2024, 9:11am

@acornanomaly can you use the secret key at all in the new installation?

For example can you sign a file with that secret key?

Another way to find more diagnostic messages is to turn them on for gpg-agent (e.g. set a log file in gpg-agent.conf see the documentation).

Or to use --debug-all for the gpg call (careful this may contain sensible data, so do not share the output, just try to see what is going one around the secret key for the new version.)

acornanomaly · February 12, 2024, 10:56pm

Digging into things more, I find I come back with more confusion.

I can sign a file with that identity, but not with that subkey in particular, as signature usage is not enabled on that subkey, and I can’t seem to do so. (change-usage only gives me the option to enable/disable encryption usage.)

More importantly, though, I can encrypt files with that subkey as a recipient, and even the newer version will be able to decrypt them.

Because of this, I’m starting to think there’s a structural error in the files I’m receiving and trying to decrypt, but I can’t imagine what error would cause “no secret key found” only on newer versions of GPG.

Doing more testing, I found out that there are two different departments within our client that are encrypting files for us using that key. However, we are only able to decrypt the files from one of the two departments on the newest version of GPG. The files from the other one will not decrypt with the newest version, but they WILL decrypt if we fall back to the older version of GPG.

Trying to compare results using --debug-all, I admit I don’t know enough about the internal operation of GPG to deduce much. I do notice that, for the files that are able to successfully decrypt, there are multiple “get_keygrip” results, with different keys(?) specified. With the one that fails, there is only one.

Still trying to determine more.

eebb · February 13, 2024, 1:57pm

You can not sign with an encryption subkey. Bernhard meant with any key of that keypair.

The issue seems really strange. Do the departments use different software to encrypt the files? Which one?

Try --ignore-mdc-error for the decryption with the newer version. Does it work with that?

And --debug lookup could also be better for gathering information in your case, all is a bit much…

bernhard · February 13, 2024, 2:31pm

With the latest report we know now that crypto operations like signing and decryption work in the new installation. That is a good.

Now my leading hypothesis is: The non-working files are encrypted with an algorithms that is deprecated. So you could try the different --ignore* and --allow* options on the command line.

acornanomaly · February 13, 2024, 6:37pm

Well, according to the gpg -vvv output, the files that work are encrypted with CAST5, and the files that don’t work are encrypted with AES. And no, I don’t have that backwards, I made sure to double-check. Interestingly, in gpg 2.4.4, it will identify the working file as a CAST5 encrypted file as it decrypts it, but it won’t identify the non-working file as an AES file. This might be because it doesn’t get far enough to even bother trying to identify the file, though.

And yes, the different departments of our client are using different software to produce these encrypted files. I don’t know which software; the ones that are working have a comment in the ASCII armor saying “Version: BCPG v1.70”, the ones that aren’t working don’t have anything besides the BEGIN PGP MESSAGE and END PGP MESSAGE banners.

The concerning thing for me is that the one that’s not working is their newer setup that they’re migrating to!

I also made an attempt with the flags --allow-old-cipher-algos --allow-weak-digest-algos --allow-weak-key-signatures --ignore-mdc-error --ignore-crc-error all enabled, to no avail.

And below I’ve included results from running gpg 2.4.4 with --debug lookup, attempting to decrypt both the working and non-working files.

Working:

PS C:\temp\gpg> gpg --debug lookup -d -o test.txt .\working.txt.pgp
gpg: reading options from '[cmdline]'
gpg: enabled debug flags: lookup
gpg: enabled compatibility flags:
gpg: armor header: Version: BCPG v1.70
gpg: public key is 681D100FD32A7C6B
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: LONG_KID: '681D100FD32A7C6B'
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => Success
gpg: DBG: finish_lookup: exact search requested and found
gpg: DBG: finish_lookup: checking key 4C296FA5 (one)(req_usage=0)
gpg: DBG:       using key D32A7C6B
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: encrypted with rsa1024 key, ID 681D100FD32A7C6B, created 2016-10-24
      "MyName <myname@company.com>"
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: FIRST
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => Success
gpg: DBG: finish_lookup: checking key 4C296FA5 (all)(req_usage=0)
gpg: DBG:       using key 4C296FA5
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: LONG_KID: '681D100FD32A7C6B'
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => Success
gpg: DBG: finish_lookup: checking key 4C296FA5 (all)(req_usage=0)
gpg: DBG:       using key 4C296FA5
gpg: WARNING: cipher algorithm CAST5 not found in recipient preferences
gpg: CAST5.CFB encrypted data
gpg: original file name='7ef8fb75-c9e6-11ee-ccbd-000000b9bb12'
gpg: Signature made 02/12/24 14:37:10 Central Standard Time
gpg:                using RSA key 456E8188B6C46B3A
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: LONG_KID: '456E8188B6C46B3A'
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => EOF
gpg: Can't check signature: No public key
gpg: secmem usage: 0/32768 bytes in 0 blocks

(I don’t have their public signature imported on this test system that I’m checking all of this on, but I still get a successful decryption.)

Not working:

PS C:\temp\gpg> gpg --debug lookup -d -o test.pdf .\not-working.pdf.pgp
gpg: reading options from '[cmdline]'
gpg: enabled debug flags: lookup
gpg: enabled compatibility flags:
gpg: public key is 681D100FD32A7C6B
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: LONG_KID: '681D100FD32A7C6B'
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => Success
gpg: DBG: finish_lookup: exact search requested and found
gpg: DBG: finish_lookup: checking key 4C296FA5 (one)(req_usage=0)
gpg: DBG:       using key D32A7C6B
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: encrypted with rsa1024 key, ID 681D100FD32A7C6B, created 2016-10-24
      "MyName <myname@company.com>"
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: FIRST
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => Success
gpg: DBG: finish_lookup: checking key 4C296FA5 (all)(req_usage=0)
gpg: DBG:       using key 4C296FA5
gpg: DBG: keydb_search: 1 search descriptions:
gpg: DBG: keydb_search   0: NEXT
gpg: DBG: internal_keydb_search: searching keybox (resource 0 of 1)
gpg: DBG: internal_keydb_search: searched keybox (resource 0 of 1) => EOF
gpg: public key decryption failed: No secret key
gpg: decryption failed: No secret key
gpg: secmem usage: 0/32768 bytes in 0 blocks

EDIT: And to provide further information, this does look like a change that happened with the gpg 2.3.x line. I went to try out other versions of gpg to see what works.

gpg 2.2.42 works, but gpg 2.3.0 does not.

bernhard · February 14, 2024, 10:15am

gpg 2.2.42 works, but gpg 2.3.0 does not.

[Announce] GnuPG 2.3.0 released has

gpg: Do not use any 64-bit block size cipher algorithm for
encryption. Use AES as last resort cipher preference instead of
3DES. This can be reverted using --allow-old-cipher-algos.

But you have ruled that out already.

Two things you can still try next:

a) enable more diagnostic on gpg-agent (as mentioned before)
b) try a --list-packets -v in both versions (but be careful to not to just post it, it should be okay in the non-working case, but double check)

eebb · February 15, 2024, 2:42pm

Could you find out which software the other side uses? The “new” software, with which it does not work?

acornanomaly · February 15, 2024, 8:54pm

I’ll work on getting gpg-agent log output next. For now, this is what shows up between the two versions for --list-packets -v(with the data redacted, but it’s the same between the two):

PS C:\temp\gpg> C:\gpg\gpg-2.2.42\bin\gpg.exe --list-packets -v  .\not-working.pdf.pgp
gpg: enabled compatibility flags:
gpg: public key is 681D100FD32A7C6B
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: WARNING: cipher algorithm AES not found in recipient preferences
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: encrypted with 1024-bit RSA key, ID 681D100FD32A7C6B, created 2016-10-24
      "My Name <myname@company.com>"
gpg: AES encrypted data
# off=0 ctb=c1 tag=1 hlen=2 plen=140 new-ctb
:pubkey enc packet: version 3, algo 1, keyid 681D100FD32A7C6B
        data: 22E2..2A99
# off=142 ctb=d2 tag=18 hlen=2 plen=0 partial new-ctb
:encrypted data packet:
        length: unknown
        mdc_method: 2
# off=163 ctb=a3 tag=8 hlen=1 plen=0 indeterminate
:compressed packet: algo=2
# off=165 ctb=cb tag=11 hlen=2 plen=0 partial new-ctb
:literal data packet:
        mode b (62), created 1707507901, name="",
        raw data: unknown length

And gpg 2.3.0:

PS C:\temp\gpg> C:\gpg\gpg-2.3.0\bin\gpg.exe --list-packets -v .\not-working.pdf.pgp
gpg: Note: RFC4880bis features are enabled.
gpg: public key is 681D100FD32A7C6B
gpg: using subkey 681D100FD32A7C6B instead of primary key 0521D2004C296FA5
gpg: encrypted with rsa1024 key, ID 681D100FD32A7C6B, created 2016-10-24
      "My Name <myname@company.com>"
gpg: public key decryption failed: No secret key
gpg: decryption failed: No secret key
# off=0 ctb=c1 tag=1 hlen=2 plen=140 new-ctb
:pubkey enc packet: version 3, algo 1, keyid 681D100FD32A7C6B
        data: 22E2..2A99
# off=142 ctb=d2 tag=18 hlen=2 plen=0 partial new-ctb
:encrypted data packet:
        length: unknown
        mdc_method: 2

As for the software the client uses, I will try to find that out, but I’m not in direct communication with them.

EDIT: Ok, with a log file enabled(and verbose turned on in gpg-agent.conf), I get the following outputs for 2.2.42 and 2.3.0(with gpg-agent being killed between attempts):

2024-02-15 15:20:24 gpg-agent[1092] listening on socket 'C:\Users\chris\AppData\Local\gnupg\S.gpg-agent'
2024-02-15 15:20:24 gpg-agent[1092] listening on socket 'C:\Users\chris\AppData\Local\gnupg\S.gpg-agent.extra'
2024-02-15 15:20:24 gpg-agent[1092] listening on socket 'C:\Users\chris\AppData\Local\gnupg\S.gpg-agent.browser'
2024-02-15 15:20:24 gpg-agent[1092] listening on socket 'C:\Users\chris\AppData\Local\gnupg\S.gpg-agent.ssh'
2024-02-15 15:20:24 gpg-agent[1092] gpg-agent (GnuPG) 2.2.42 started
2024-02-15 15:20:25 gpg-agent[1092] handler 0x2 for fd 132 started
2024-02-15 15:20:26 gpg-agent[1092] handler 0x2 for fd 132 terminated
2024-02-15 15:21:28 gpg-agent[1092] handler 0x3 for fd 616 started
2024-02-15 15:21:28 gpg-agent[1092] socket is still served by this server
2024-02-15 15:21:28 gpg-agent[1092] handler 0x3 for fd 616 terminated

2024-02-15 15:24:47 gpg-agent[28096] listening on socket 'C:\\Users\\chris\\AppData\\Roaming\\gnupg\\S.gpg-agent'
2024-02-15 15:24:47 gpg-agent[28096] listening on socket 'C:\\Users\\chris\\AppData\\Roaming\\gnupg\\S.gpg-agent.extra'
2024-02-15 15:24:47 gpg-agent[28096] listening on socket 'C:\\Users\\chris\\AppData\\Roaming\\gnupg\\S.gpg-agent.browser'
2024-02-15 15:24:47 gpg-agent[28096] listening on socket 'C:\\Users\\chris\\AppData\\Roaming\\gnupg\\S.gpg-agent.ssh'
2024-02-15 15:24:47 gpg-agent[28096] gpg-agent (GnuPG) 2.3.0 started
2024-02-15 15:24:49 gpg-agent[28096] handler 0x2 for fd 132 started
2024-02-15 15:24:49 gpg-agent[28096] Assuan get_peercred failed: General IPC error
2024-02-15 15:24:49 gpg-agent[28096] handler 0x2 for fd 132 terminated

The one thing I do note is the line “Assuan get_peercred failed: General IPC error”, but I’m not sure what that means.

Is there anything else I should try to log?

EDIT2: I’ve turned on debug-all, and have some results with the channel communications with gpg-agent, but I’m not sure if it points to anything.

These have all been redacted to remove sensitive info(I believe), but still show what I’m seeing.

The first is the not working file in 2.3.0:

2024-02-15 15:35:28 gpg-agent[10372] gpg-agent (GnuPG) 2.3.0 started
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> OK Pleased to meet you
2024-02-15 15:35:30 gpg-agent[10372] Assuan get_peercred failed: General IPC error
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- RESET
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> OK
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- GETINFO version
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> D 2.3.0
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> OK
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- OPTION allow-pinentry-notify
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> OK
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- OPTION agent-awareness=2.1.0
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> OK
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- SCD SERIALNO
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> ERR 67108924 Not supported <GPG Agent>
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- HAVEKEY ..024D ..8E77
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 -> OK
2024-02-15 15:35:30 gpg-agent[10372] DBG: chan_0x00000084 <- [eof]

I’m a little confused why the gpg client seems to just end the connection there?

Here’s a working file in gpg 2.3.0:

2024-02-15 15:55:19 gpg-agent[20876] gpg-agent (GnuPG) 2.3.0 started
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK Pleased to meet you
2024-02-15 15:55:21 gpg-agent[20876] Assuan get_peercred failed: General IPC error
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- RESET
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- GETINFO version
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> D 2.3.0
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- OPTION allow-pinentry-notify
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- OPTION agent-awareness=2.1.0
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- SCD SERIALNO
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> ERR 67108924 Not supported <GPG Agent>
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- HAVEKEY ..024D ..8E77
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- RESET
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- SETKEY ..8E77
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- SETKEYDESC Please+enter+the+passphrase+to+unlock+the+OpenPGP+secret+key:%0A%22My+Name+<MyName@company.com>%22%0A1024-bit+RSA+key,+ID+681D100FD32A7C6B,%0Acreated+2016-10-24+(main+key+ID+0521D2004C296FA5).%0A
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- PKDECRYPT
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> S INQUIRE_MAXLEN 4096
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> INQUIRE CIPHERTEXT
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_00000088 <- [ 44 20 28 37 3a 65 6e 63 2d 76 61 6c 28 33 3a 72 ...(147 byte(s) skipped) ]
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- END
2024-02-15 15:55:21 gpg-agent[20876] DBG: keygrip: ..8E77
2024-02-15 15:55:21 gpg-agent[20876] DBG: cipher:  2837 .. [MANUALLY DELETED]

2024-02-15 15:55:21 gpg-agent[20876] DBG: rsa_decrypt data: [MANUALLY DELETED]

2024-02-15 15:55:21 gpg-agent[20876] DBG: rsa_decrypt    => Success
2024-02-15 15:55:21 gpg-agent[20876] DBG: plain: [data="MANUALLY DELETED"]
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_00000088 -> [ 44 20 28 35 3a 76 61 6c 75 65 31 32 37 3a 02 42 ...(127 byte(s) skipped) ]
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 -> OK
2024-02-15 15:55:21 gpg-agent[20876] DBG: chan_0x00000088 <- [eof]

And finally, the not working file in gpg 2.2.42:

2024-02-15 15:36:30 gpg-agent[23268] gpg-agent (GnuPG) 2.2.42 started
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK Pleased to meet you
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- RESET
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- GETINFO version
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> D 2.2.42
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- OPTION allow-pinentry-notify
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- OPTION agent-awareness=2.1.0
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- KEYINFO ..8E77
2024-02-15 15:36:32 gpg-agent[23268] DBG: agent_get_cache '..8E77'.0 (mode 2) ...
2024-02-15 15:36:32 gpg-agent[23268] DBG: ... miss
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> S KEYINFO ..8E77 D - - - C - - -
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- HAVEKEY ..024D ..8E77
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- KEYINFO ..8E77
2024-02-15 15:36:32 gpg-agent[23268] DBG: agent_get_cache '..8E77'.0 (mode 2) ...
2024-02-15 15:36:32 gpg-agent[23268] DBG: ... miss
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> S KEYINFO ..8E77 D - - - C - - -
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- RESET
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- SETKEY ..8E77
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- SETKEYDESC Please+enter+the+passphrase+to+unlock+the+OpenPGP+secret+key:%0A%22My+Name+<myname@company.com>%22%0A1024-bit+RSA+key,+ID+681D100FD32A7C6B,%0Acreated+2016-10-24+(main+key+ID+0521D2004C296FA5).%0A
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- PKDECRYPT
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> S INQUIRE_MAXLEN 4096
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> INQUIRE CIPHERTEXT
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_00000088 <- [ 44 20 28 37 3a 65 6e 63 2d 76 61 6c 28 33 3a 72 ...(143 byte(s) skipped) ]
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- END
2024-02-15 15:36:32 gpg-agent[23268] DBG: keygrip: .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. .. 8E 77
2024-02-15 15:36:32 gpg-agent[23268] DBG: cipher:  28 37 .. [MANUALLY DELETED]

2024-02-15 15:36:32 gpg-agent[23268] DBG: rsa_decrypt data: [MANUALLY DELETED]

2024-02-15 15:36:32 gpg-agent[23268] DBG: rsa_decrypt    => Success
2024-02-15 15:36:32 gpg-agent[23268] DBG: plain: [data="MANUALLY DELETED"]
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_00000088 -> [ 44 20 28 35 3a 76 61 6c 75 65 31 32 37 3a 02 94 ...(129 byte(s) skipped) ]
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 -> OK
2024-02-15 15:36:32 gpg-agent[23268] DBG: chan_0x00000088 <- [eof]

bernhard · February 23, 2024, 7:55am

Thanks for the additional logs and sorry that we do not have a good idea so far either. I agree with your analysis that something seems to be at odds at the gpg side.

So a gpg --debug-all gives the maximum of debugging information (and should be carefully cleaned before handing out to others).

I think you should try to ask the other department about the software, as there is the small chance that it is producing something which is not fully compatible with OpenPGP (v4) or LibrePGP (draft). Knowing the software would give us an idea how to recreate this.

There is the possibility that this is a defect in GnuPG somewhere, in order to examine this in detail , we would need an encrypted file with one public keys we have access to, that shows the problem. For this you would need the cooperation of the other departement. Here is a set of completely open to everyone testing keypair (that I have been using for a completely different product): csaf_distribution/docs/test-keys at main · csaf-poc/csaf_distribution · GitHub
If you could get someone to encrypt something that can be fully published showing the bad behaviour, then we and everybody can try to decrypt it with the private key parts and reproduce and analyse the bad behaviour.

Sorry for all the hassle, but if a defect is deep down either within the software doing the encryption or within the modern version of GnuPG, it is hard to diagnose by nature.

Best Regards,
Bernhard

acornanomaly · February 28, 2024, 10:37pm

I understand, and I thank everyone for their help so far.

I’m going to be attempting to pull a full gpg --debug-all session and sanitize it so I could show what I’m seeing, and maybe see if anything pops out.

I’m also going to see if I can get into contact with this client and get a couple of test files encrypted with a test key that I can share, as suggested. This is likely going to take a little while.

Thanks again!

bernhard · February 29, 2024, 8:34am

Thanks for your understanding and help to see if this is a defect somewhere!

pateljag · May 1, 2024, 12:05pm

I am seeing the similar issue. We are moving our SFTP to Azure and installed latest version of GPG. Decryption works perfectly fine on old environment but in new one it fails.

C:\Users\Public\Documents\git_projects\utility-scripts>python gpg_decrypt_file.pyw "E:\python-scripts\PythonGnuPgEncryption\log\*** Automation Team Prod Key.asc" "\\****\BA$\EDW\PROD\VEDA\Landing\PAFMO_ERROR_REPORT_20240429.txt.pgp" "\\******\BA$\EDW\PROD\VEDA\Archive\20240429\PAFMO_ERROR_REPORT_20240429.txt" "passphrase" "E:\python-scripts\PythonGnuPgEncryption\homeNew" 
Line 1369: {'fingerprint': '3512B0876D588C448B47F2D9C4AB46B396A22AF2', 'ok': '1', 'text': 'Not actually changed\nEntirely new key\n'}
None
Line 1380: False
None
Line 1381: no secret key
None
Line 1382: [GNUPG:] ENC_TO 91AD24AA41847B1C 1 0
[GNUPG:] ENC_TO D0624BFF67C80621 1 0
[GNUPG:] KEY_CONSIDERED 3512B0876D588C448B47F2D9C4AB46B396A22AF2 0
gpg: encrypted with rsa2048 key, ID D0624BFF67C80621, created 2019-10-21
      "***** Automation Team Prod Key <dl-automation@****com.au>"
gpg: encrypted with RSA key, ID 91AD24AA41847B1C
gpg: problem with fast path key listing: No such file or directory - ignored
[GNUPG:] NO_SECKEY D0624BFF67C80621
[GNUPG:] NO_SECKEY 91AD24AA41847B1C
gpg: public key decryption failed: No secret key
[GNUPG:] ERROR pkdecrypt_failed 33554449
[GNUPG:] BEGIN_DECRYPTION
[GNUPG:] DECRYPTION_FAILED
gpg: decryption failed: No secret key
[GNUPG:] END_DECRYPTION

bernhard · May 2, 2024, 7:20am

Hi @pateljag
sorry to hear that you have having problems.

While the symptoms maybe the same, I’d prefer a new topic because the chance is high that you are experiencing something else.

In the new post, please answer this questions to give us more details:
What versions of Gpg4win/GnuPG are you using? Which works, which doesnt?

Can you reproduce the problem on the command line with manual commands?
(To exclude effects from the python scripts you are using.)

Does gpg -v --list-secret-keys see your secret keys in both settings?
Can you use the secret key for other purposese in both settings?
Do diagnostics (adding -v then one or two more, then --debug X give you more insights?)

Best,
Bernhard