This issue tracker has been migrated to GitHub, and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: "MUPCA Root" Certificates - treated as invalid and cause error, but are walid and necessary
Type: crash Stage: resolved
Components: SSL, Windows Versions: Python 3.9
process
Status: closed Resolution: third party
Dependencies: 35665 Superseder: Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute
View: 35665
Assigned To: Nosy List: MDM-1, Pedjas, paul.moore, pukkandan, steve.dower, tim.golden, zach.ware
Priority: normal Keywords:

Created on 2021-09-28 15:49 by MDM-1, last changed 2022-04-11 14:59 by admin. This issue is now closed.

Files
File name Uploaded Description Edit
Untitled.png MDM-1, 2021-09-28 15:49 MUPCA Root - Certificates in qestion
135081521-83d466d9-c71d-465c-96a2-652c2549c461.png MDM-1, 2021-09-28 15:58 Error 1
135087546-e6fd1b05-6858-4e0f-ad3f-857c614bb15b.png MDM-1, 2021-09-28 15:58 Error 2
Messages (9)
msg402784 - (view) Author: Dimitrije Milović (MDM-1) * Date: 2021-09-28 15:49
I just commented to the issue here https://bugs.python.org/issue35665?@ok_message=issue%2035665%20files%20edited%20ok&@template=item, but noticed "closed" so better start a new one issue, and to further update the importance of those certificates...

I came to this issue (still persistent with all python versions since 3.6) while using yt-dlp: https://github.com/yt-dlp/yt-dlp/issues/1060

I obviously have the SAME problem than the guys in your link since I am from Serbia too, and those certificates "MUPCA Root" are (unfortunately-badly executed) crucial (issued by the ministry of interior - police 🙄) ones to be able too read ID cards and use personal signing certificates, and they're are all valid...
So the option to remove the faulty certificates, is a no go to me (or anyone in Serbia using their ID card - individuals, companies and entrepreneurs like me)...

Please help!
msg402786 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-09-28 16:11
This needs to be a feature request against the script that you're running. They have the option of not verifying TLS certificates if they choose not to, but they are explicitly enabling the checks right now.

We can't add a command line option to disable it for them. You'll need to find the place where they work and request it there.
msg402787 - (view) Author: Dimitrije Milović (MDM-1) * Date: 2021-09-28 16:26
OK, will let the yt-dlp author pukkandan on GitHub know.
Thanks for the quick answer.

The only downside in this will be that this will have to be donne for many programs and scripts in the future; and for more and more persons (using our ID certificate will be only more preponderant as time passes).
So is it impossible to just somehow circumvent or add as exclusions those certificates? I could send them all to you..?
msg402788 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-09-28 16:29
Python doesn't include any trusted certificates - it reads them from the operating system. So you'll need to get the operating system vendors to include it if you want it to be trusted by default.

Additionally, some libraries include a copy of Mozilla's bundle (usually via the certifi package) and override the operating system. You'd need them to also include it.
msg402789 - (view) Author: pukkandan (pukkandan) Date: 2021-09-28 16:38
Hi,

I am the maintainer of the above mentioned project. I was planning to implement a patch for this. But I asked OP to report the issue here anyway since I do not believe this is the intended behavior. 

For context, the issue is occurring when using the `ssl.create_default_context` function and not by manually adding the verify flag. For this, the default (in my opinion) should be to ignore any invalid certificates. Even the comment in the relevent code (https://github.com/python/cpython/blob/84975146a7ce64f1d50dcec8311b7f7188a5c962/Lib/ssl.py#L772-L774) seem to agree with my sentiment. 

I ask that you please reconsider your stance on this issue. Thanks
msg402790 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-09-28 16:42
Adding Christian, as he's our expert in this area, and was also driving the other bug.
msg402791 - (view) Author: pukkandan (pukkandan) Date: 2021-09-28 16:49
Also, the pictures uploaded by the OP are misleading since they are from a version of the code that was specifically intended for debugging the issue. the problem can be better seen in this comment https://github.com/yt-dlp/yt-dlp/issues/1060#issuecomment-925843378

```py
C:\Windows\system32>py
Python 3.9.7 (tags/v3.9.7:1016ef3, Aug 30 2021, 20:19:38) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> len(ssl.enum_certificates('ROOT'))
68
>>> len(ssl.enum_certificates('CA'))
39
>>> ssl.create_default_context()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Program Files\Python39\lib\ssl.py", line 750, in create_default_context
    context.load_default_certs(purpose)
  File "C:\Program Files\Python39\lib\ssl.py", line 574, in load_default_certs
    self._load_windows_store_certs(storename, purpose)
  File "C:\Program Files\Python39\lib\ssl.py", line 566, in _load_windows_store_certs
    self.load_verify_locations(cadata=certs)
ssl.SSLError: not enough data: cadata does not contain a certificate (_ssl.c:4159)
>>> exit()
```
msg402801 - (view) Author: (Pedjas) Date: 2021-09-28 19:42
This hurts number of Python applications, even those published by large players. Basically, any attempt to read any certificate (for example to load any https url) fails due to this issue.

For example:

- QGIS fails to load map tiles on https links. Python issue with certificates.

- AutoDesk Fusion 360 cannot be installed. On install, it requires online activation. Activation is done using https link. That does not work as Python fails on certificates.

And that is a bug of Python. If you check code that causes this issue you will notice problem in code.

1.  When some certificate is needed Python loops and tries to load each and every certificate installed instead of loading only certificate that is actually needed and skip others.

2. No exception handling. When trying to load "bad" certificate, Python just crashes instead of graciously handle (skip) issue.

This problem occurs only with Python. No other application has such issue when handling certificates. MUPCA certificate works fine with every other application.

This issue can be easily solved with one simple if and one simple exception handler: loop through certificate. Only if certificate is the one needed try to load it. Enclose loading code within exception, if it fails, report descriptive error, and skip further. Do not allow Python to crash.
msg402813 - (view) Author: Steve Dower (steve.dower) * (Python committer) Date: 2021-09-28 21:57
Looks like you should take the discussion to issue35665, and this one can stay closed.
History
Date User Action Args
2022-04-11 14:59:50adminsetgithub: 89475
2021-09-28 21:57:36steve.dowersetmessages: + msg402813
2021-09-28 20:22:48christian.heimessetassignee: christian.heimes ->

nosy: - christian.heimes
2021-09-28 19:42:15Pedjassetnosy: + Pedjas
messages: + msg402801
2021-09-28 16:54:09christian.heimessetdependencies: + Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute
superseder: Function ssl.create_default_context raises exception on Windows 10 when called with ssl.Purpose.SERVER_AUTH) attribute
2021-09-28 16:49:28pukkandansetmessages: + msg402791
2021-09-28 16:42:09steve.dowersetnosy: + christian.heimes
messages: + msg402790

assignee: christian.heimes
components: + SSL
2021-09-28 16:38:28pukkandansetnosy: + pukkandan
messages: + msg402789
2021-09-28 16:29:26steve.dowersetresolution: remind -> third party
messages: + msg402788
2021-09-28 16:26:14MDM-1setresolution: third party -> remind
messages: + msg402787
2021-09-28 16:11:10steve.dowersetstatus: open -> closed
resolution: third party
messages: + msg402786

stage: resolved
2021-09-28 15:58:23MDM-1setfiles: + 135087546-e6fd1b05-6858-4e0f-ad3f-857c614bb15b.png
2021-09-28 15:58:15MDM-1setfiles: + 135081521-83d466d9-c71d-465c-96a2-652c2549c461.png
2021-09-28 15:49:22MDM-1create