Signature validation issue for Elliptic Curve signed files

[ertankucukoglu]

Hello,

I've been using xmlsec (Win32) for validating XML file signatures for a few years.
This year authority switched from using RSA to EC and new EC signed files are not passing as valid.
I have double checked multiple times and authority provided tool (java and not command prompt friendly) is identifying them as valid.

I have not enough knowledge to say what the problem is, or if the xmlsec does not support it.
I appreciated it if someone could look at what is wrong.

I cropped only the signature part from some XML files for reference.
RSA sample: https://pastebin.pl/view/ef9bd386
EC sample: https://pastebin.pl/view/8bcfdd79

Thanks & Regards,
Ertan

[lsh123]

1/ What error do you get?
2/ What version of XML sec library do you use?
3/ What crypto library and version do you use?

I honestly don’t have any way to help you since I can’t reproduce the issue without certificates, etc. Best way is that if you can create self contained repro test case with xmlsec command line utility.

[ertankucukoglu]

Hi,
Sorry for missing details.
I tested with the available latest release using what is provided in the package. No external crypto libraries used. This is Windows environment.
Below is the error I receive (for the EC sample from my initial post)

PS C:\Users\Ertan\Downloads\xmlsec1-1.3.9-win64\xmlsec\bin> .\xmlsec.exe --verify --insecure --verbose .\EC-8001216299-202508-K-000000.xml
func=xmlSecOpenSSLX509NameReadCallback:file=..\src\openssl\x509vfy.c:line=1842:obj=unknown:subj=X509_NAME_add_entry_by_txt:error=4:crypto library function failed:name=CN; type=0; openssl error: error:06800097:asn1 encoding routines::string too long
func=xmlSecX509NameRead:file=..\src\x509.c:line=366:obj=unknown:subj=xmlSecX509NameReadCallback:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509NameRead:file=..\src\openssl\x509vfy.c:line=1872:obj=unknown:subj=xmlSecX509NameRead:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509FindCertCtxInitialize:file=..\src\openssl\x509vfy.c:line=1371:obj=unknown:subj=xmlSecOpenSSLX509NameRead:error=1:xmlsec library function failed:subject=CN=WAW INTERNATIONAL MEDİKAL KOZMETİK DIŞ TİCARET VE LİMİTED ŞİRKETİ, SERIALNUMBER=8001216299
func=xmlSecOpenSSLX509FindCertCtxInitializeFromValue:file=..\src\openssl\x509vfy.c:line=1427:obj=unknown:subj=xmlSecOpenSSLX509FindCertCtxInitialize:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509FindKeyByValue:file=..\src\openssl\x509vfy.c:line=287:obj=unknown:subj=xmlSecOpenSSLX509FindCertCtxInitializeFromValue:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509NameReadCallback:file=..\src\openssl\x509vfy.c:line=1842:obj=unknown:subj=X509_NAME_add_entry_by_txt:error=4:crypto library function failed:name=CN; type=0; openssl error: error:06800097:asn1 encoding routines::string too long
func=xmlSecX509NameRead:file=..\src\x509.c:line=366:obj=unknown:subj=xmlSecX509NameReadCallback:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509NameRead:file=..\src\openssl\x509vfy.c:line=1872:obj=unknown:subj=xmlSecX509NameRead:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509FindCertCtxInitialize:file=..\src\openssl\x509vfy.c:line=1371:obj=unknown:subj=xmlSecOpenSSLX509NameRead:error=1:xmlsec library function failed:subject=CN=WAW INTERNATIONAL MEDİKAL KOZMETİK DIŞ TİCARET VE LİMİTED ŞİRKETİ, SERIALNUMBER=8001216299
func=xmlSecOpenSSLX509FindCertCtxInitializeFromValue:file=..\src\openssl\x509vfy.c:line=1427:obj=unknown:subj=xmlSecOpenSSLX509FindCertCtxInitialize:error=1:xmlsec library function failed:
func=xmlSecOpenSSLX509StoreFindCertByValue:file=..\src\openssl\x509vfy.c:line=249:obj=unknown:subj=xmlSecOpenSSLX509FindCertCtxInitializeFromValue:error=1:xmlsec library function failed:
func=xmlSecOpenSSLEvpSignatureVerify:file=..\src\openssl\signatures.c:line=1074:obj=ecdsa-sha384:subj=unknown:error=18:data do not match:details=Signature verification failed
Verification status: FAILED
Failure reason: SIGNATURE
SignedInfo References (ok/all): 2/2
Manifests References (ok/all): 0/0
Error: failed to verify file ".\EC-8001216299-202508-K-000000.xml"
PS C:\Users\Ertan\Downloads\xmlsec1-1.3.9-win64\xmlsec\bin>

I will also try to get permission for providing a failing XML file in complete.

Thanks & Regards,
Ertan

[lsh123]

The error says that OpenSSL doesn’t like some string being too long but looking at it seems like there is string encoding issues. Since you are on Windows, try to run it with MSCng (add —crypto mscng to the command line)

[ertankucukoglu]

Windows messes when it comes to UTF8. Console text is not true UTF8 as far as I know. Processing is not a problem though. So, I believe encoding problem is only visual and is not important.

When I try added command line parameter error is different now

PS C:\Users\Ertan\Downloads\xmlsec1-1.3.9-win64\xmlsec\bin> .\xmlsec.exe --verify --insecure --crypto mscng --verbose .\EC-8001216299-202508-K-000000.xml
func=xmlSecMSCngSignatureVerify:file=..\src\mscng\signatures.c:line=594:obj=ecdsa-sha384:subj=unknown:error=18:data do not match:details=BCryptVerifySignature: the signature was not verified
func=xmlSecTransformVerifyNodeContent:file=..\src\transforms.c:line=1582:obj=ecdsa-sha384:subj=xmlSecTransformVerify:error=1:xmlsec library function failed:
func=xmlSecDSigCtxVerify:file=..\src\xmldsig.c:line=372:obj=unknown:subj=xmlSecTransformVerifyNodeContent:error=1:xmlsec library function failed:
Verification status: ERROR
SignedInfo References (ok/all): 2/2
Manifests References (ok/all): 0/0
Error: failed to verify file ".\EC-8001216299-202508-K-000000.xml"
PS C:\Users\Ertan\Downloads\xmlsec1-1.3.9-win64\xmlsec\bin>

Thanks.

[lsh123]

So for mscng seems like everything but signature worked. I don’t really know why (I assume file was not modified, etc). It’s very hard to debug those errors without a rep to example though.

For OpenSSL, can you share top line in the file with encoding?

[ertankucukoglu]

Here is the tested file as whole. I extracted it from compressed archive (program generated) and shared in here. File contents are not modified in anyway and signature should be valid. It is enough for me the verify using --insecure parameter.

8001216299-202508-K-000000.xml

Thanks.

[lsh123]

Great, I will check it out and let you know what I find (I am on the road so might take a couple days)

[lsh123]

I have good / bad news for you. Good news -- I made it work. Bad news -- to use the fix, you will need to rebuild the library with the patch (see PR #996) :

$ OPENSSL_CONF=openssl3.cnf ./apps/xmlsec1 --verify --insecure --verbose 8001216299-202508-K-000000.xml 
Verification status: OK
SignedInfo References (ok/all): 2/2
Manifests References (ok/all): 0/0

$ cat openssl3.cnf 
# Comment out the next line to ignore configuration errors
config_diagnostics = 1

openssl_conf = openssl_init

[openssl_init]
stbl_section = stbl

[stbl]
commonName = max:100

1. The original issue with long name was indeed long name error since OpenSSL limits the CN size to 64 chars. This is easily fixable with a custom openssl.cnf (see example above). MSCNG doesn't have this issue hence this was not an error when you tried it.

2. Unfortunately the next problem was not fixable with configs -- OpenSSL uses string case sensitive compare for DNs (see sn_cmp function in OpenSSL codebase) and your file has "SERIALNUMBER" instead of "serialNumber" expected by OpenSSL. But it's a trivial fix since some other fields had this problems in the past as well. MSCNG doesn't have this issue either.

3. Unfortunately it is not the end of it. Next problem was that the software you use encodes the signature value incorrectly. The XMLDSig spec (section 3.3 in https://www.ietf.org/rfc/rfc4050.txt) specifies that signature value is "the concatenation of two octet-streams that respectively result from the octet-encoding of the values r and s". Instead the software you use put ASN1 encoding of the two integers. This is the second (bigger part of the fix).

So the fix in PR #996 addresses both 2/ and 3/ for OpenSSL. You can follow https://github.com/lsh123/xmlsec/blob/master/scripts/build_windows.sh and build XMLSec from git master or apply path to the 1.3.9 tarball or ... (this way you will also be building binaries in your own environment and don't use ones I provide which is a good thing anyway).

I still need to finish the fixes for other crypto libraries (including mscng) though. I will probably add some config options to also generate ECDSA signatures that way so other software that expects ASN1 would handle it correctly as well. This will take a bit more time.

[ertankucukoglu]

Hello,

Thank you for you help.
I remember trying to build the library in the past. I failed. I will try to do it again, but I will likely end up waiting for a new release.

Regards,
Ertan

[lsh123]

I would still highly recommend for you to build the binaries yourself... said that here is both source code and the binaries that include all of the changes from this issue. Note that I've added --enable-asn1-signatures-hack option to control if ASN1 signatures are supported in both singing and verifying paths (see PR #997 for more details). Let me know if these fixes work for you!

xmlsec1-1.3.10-preview-1.tar.gz
xmlsec1-1.3.10-preview-1-win64.zip