手元のpythonで書いてるクローラーが動かなくなって、かなり困りました。
エラーログ
% pip3 install chromedriver-binary-auto
Collecting chromedriver-binary-auto
Using cached chromedriver-binary-auto-0.2.6.tar.gz (5.2 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: chromedriver-binary-auto
Building wheel for chromedriver-binary-auto (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [85 lines of output]
running bdist_wheel
running build
running build_py
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1037, in _send_output
self.send(msg)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 975, in send
self.connect()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/http/client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1070, in _create
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/private/var/folders/0d/gycxv2pn19xfmt_vlmjj42l40000gn/T/pip-install-ptykwzd5/chromedriver-binary-auto_28a8726f20db439990f5c8bfe1c90a3a/chromedriver_binary/utils.py", line 97, in get_latest_release_for_version
response = urlopen(release_url, context=ssl_context)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 519, in open
response = self._open(req, data)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
result = func(*args)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/private/var/folders/0d/gycxv2pn19xfmt_vlmjj42l40000gn/T/pip-install-ptykwzd5/chromedriver-binary-auto_28a8726f20db439990f5c8bfe1c90a3a/setup.py", line 64, in <module>
setup(
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/__init__.py", line 107, in setup
return distutils.core.setup(**attrs)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 969, in run_commands
self.run_command(cmd)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 343, in run
self.run_command("build")
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 131, in run
self.run_command(cmd_name)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
self.distribution.run_command(command)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/dist.py", line 1244, in run_command
super().run_command(command)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
cmd_obj.run()
File "/private/var/folders/0d/gycxv2pn19xfmt_vlmjj42l40000gn/T/pip-install-ptykwzd5/chromedriver-binary-auto_28a8726f20db439990f5c8bfe1c90a3a/setup.py", line 33, in run
chromedriver_version = get_latest_release_for_version(chrome_major)
File "/private/var/folders/0d/gycxv2pn19xfmt_vlmjj42l40000gn/T/pip-install-ptykwzd5/chromedriver-binary-auto_28a8726f20db439990f5c8bfe1c90a3a/chromedriver_binary/utils.py", line 102, in get_latest_release_for_version
raise RuntimeError('Failed to find release information: {}'.format(release_url))
RuntimeError: Failed to find release information: https://chromedriver.storage.googleapis.com/LATEST_RELEASE_114
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for chromedriver-binary-auto
Running setup.py clean for chromedriver-binary-auto
Failed to build chromedriver-binary-auto
ERROR: Could not build wheels for chromedriver-binary-auto, which is required to install pyproject.toml-based projects
直接の原因は↓この部分のようです。SSL通信に失敗しており、DL先のサーバー証明書が古いらしいです。
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)>DL先のサーバー証明書が新しくなるのを待とうかと思ったのですが、いつになるかわかりません。
ネットで調べると↓このようにtrusted-hostを指定することで、サーバー証明書の検証をスキップできると書いていたので、やってみました。
pip3 install --trusted-host pypi.org --trusted-host files.pythonhosted.org chromedriver-binary-auto原理的にいけるかと思ったのですが、それでも同様のエラーが出て失敗してしまいました。
解決方法
最終的には、こちらの解決策2の方法でやっと解決できました。
https://qiita.com/tommy19970714/items/96edba36dfde468e26f3
やったこと↓
% pip3 install --upgrade certifi
Requirement already satisfied: certifi in /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages (2022.12.7)
Collecting certifi
Downloading certifi-2023.5.7-py3-none-any.whl (156 kB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 157.0/157.0 kB 1.5 MB/s eta 0:00:00
Installing collected packages: certifi
Attempting uninstall: certifi
Found existing installation: certifi 2022.12.7
Uninstalling certifi-2022.12.7:
Successfully uninstalled certifi-2022.12.7
Successfully installed certifi-2023.5.7
% python3
Python 3.10.1 (v3.10.1:2cd268a3a9, Dec 6 2021, 14:28:59) [Clang 13.0.0 (clang-1300.0.29.3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import certifi
>>> certifi.where()
'/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/certifi/cacert.pem'
>>> ^D
% export SSL_CERT_FILE=/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/certifi/cacert.pem
% pip3 install chromedriver-binary-auto
Collecting chromedriver-binary-auto
Using cached chromedriver-binary-auto-0.2.6.tar.gz (5.2 kB)
Preparing metadata (setup.py) ... done
Building wheels for collected packages: chromedriver-binary-auto
Building wheel for chromedriver-binary-auto (setup.py) ... done
Created wheel for chromedriver-binary-auto: filename=chromedriver_binary_auto-0.2.6-py3-none-any.whl size=8652837 sha256=abfe4ec8056736c22c02aafd59477d66edee95601921cf9f0faca482136828e2
Stored in directory: /Users/noricgeographic/Library/Caches/pip/wheels/7a/c4/44/3dbbc417f9762769d8068f30d91f849984c638609b83898dc2
Successfully built chromedriver-binary-auto
Installing collected packages: chromedriver-binary-auto
Successfully installed chromedriver-binary-auto-0.2.6正直、何をやっているかわからないのですが、chromedriver-binary-autoがインストールでき、自家製のクローラーも動いてくれました。
chrome-driverはChromeとのバージョンが一致しないと動かないので、Chromeの自動アップデートはOFFにしておいた方が良いと思いました。