Selenium Webdriver – How to Setup Webdriver Manager to Use Chromedriver in a MacOS App Compiled with PyInstaller

pyinstallerselenium-chromedriverselenium-webdriverwebdriverwebdrivermanager-python

In my python script I am currently using webdriver_manager (text) to install the chrome webdriver. This webdriver is used to create a Selenium driver to navigate websites and collect HTML data. This script works perfectly when running in python, but when I use PyInstaller, with the "–onefile" flag to convert it into a .app, the code does not work as expected. I am on an intel based Mac.

I think it might be an issue with MacOS and chromedriver, since on windows I have been able to compile the script to a .exe and have the webdriver_manager work as expected.

I tried multiple strategies. Here is a brief description of the solution I tried with an error code attached. The full function is located at the bottom.

In the first try block, in chrome_driver_setup() I tried the default installation method following the webdriver-manager documentation.
INFO:webdriverSetup:ChromeDriver not working. trying ChromeDriverManager().install() Message: Unable to obtain driver for chrome; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

In the second try block I attempt to use the service command from Selenium.
INFO:__main:running INFO:webdriverSetup:Error. Attempting other default chrome webdriver setup: Message: Unable to obtain driver for chrome; For documentation on this error, please visit:

In the third try block I attempt to use the chrome_autoinstaller library text following the documentation guides.

INFO:webdriverSetup:Error. Attempting autoinstaller: Message: Unable to obtain driver for chrome; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

In the fourth try block I used an opener
INFO:root:Downloading chromedriver (127.0.6533.57)... INFO:webdriverSetup:Error. Attempting opener thing: Message: Unable to obtain driver for chrome; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

In the fifth try block, I noticed the webdriver-manager, when ran as a .app, would not create the driver as an executable file and it would append THIRD_PARTY_NOTICES to the chrome driver making it impossible to run. Here is the error attached
NFO:WDM:====== WebDriver manager ====== INFO:WDM:Get LATEST chromedriver version for google-chrome INFO:WDM:Get LATEST chromedriver version for google-chrome INFO:WDM:Driver [/Users/bennet/.wdm/drivers/chromedriver/mac64/127.0.6533.57/chromedriver-mac-x64/THIRD_PARTY_NOTICES.chromedriver] found in cache INFO:webdriverSetup:ChromeDriverManager, /Users/bennet/.wdm/drivers/chromedriver/mac64/127.0.6533.57/chromedriver-mac-x64/chromedriver INFO:webdriverSetup:Tried everything and it's not working. Now attempting to remove driver and reinstall. Message: Unable to obtain driver for chrome; For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors/driver_location

All of these cases do not work when the script is converted to a .app on Mac but does when it is in script format. If anyone could help with this issue, I would greatly appreciate it!

Full function attached here

def get_driver():
    if sys.platform in ["Linux", "darwin"]:
        try:
            return chrome_driver_setup()
        except Exception as e:
            logger.info(f"Tried everything and it's not working. Now attempting to remove driver and reinstall. \n {e}")
            driver_path = shutil.which('chromedriver')
            # Remove the folder and everything inside it if the path exists
            if driver_path:
                driver_folder = os.path.dirname(driver_path)
                shutil.rmtree(driver_folder)
            return chrome_driver_setup()
    elif sys.platform == "win32":
        return bing_driver_setup()

def chrome_driver_setup():
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")  # Run Chrome in headless mode
    chrome_options.add_argument("--no-sandbox")  # Bypass OS security model
    chrome_options.add_argument("--disable-dev-shm-usage")  # Overcome limited resource problems

    def make_executable(path):
        # Change the file permissions to make it executable
        st = os.stat(path)
        os.chmod(path, st.st_mode | stat.S_IEXEC)

    def is_executable(file_path):
        # Check if the file is executable
        return os.path.isfile(file_path) and os.access(file_path, os.X_OK)   
        
    def create_ssl_context():
        context = ssl.create_default_context(cafile=certifi.where())
        return context

    def create_opener():
        # Create a custom opener with our SSL context
        context = create_ssl_context()
        opener = urllib.request.build_opener(urllib.request.HTTPSHandler(context=context))
        urllib.request.install_opener(opener)
    

    # I know this is disgusting. Please blame ChromeDriver and Pyinstaller with MacOS because they don't work well together
    try:      
        print("Attempting default chrome webdriver setup")
        driver = webdriver.Chrome(options=chrome_options)
        return driver
    except Exception as e:
        try: 
            logger.info(f"Error. Attempting other default chrome webdriver setup: \n {e}")
            service = ChromeService()
            driver = webdriver.Chrome(service=service, options=chrome_options)
        except Exception as e:
            try: 
                logger.info(f"Error. Attempting autoinstaller: \n {e}")
                chromedriver_autoinstaller.install()
                driver = webdriver.Chrome()
                return driver
            except Exception as e:
                try:
                    logger.info(f"Error. Attempting opener thing: \n {e}")
                    create_opener()
                    driver = webdriver.Chrome(options=chrome_options)
                    return driver
                except Exception as e:
                    logger.info(f"ChromeDriver not working. trying ChromeDriverManager().install() \n {e}")
                    executable_path = ChromeDriverManager().install()
                    # need to do this because it installs incorrectly sometimes
                    if executable_path.endswith("THIRD_PARTY_NOTICES.chromedriver"):
                        executable_path = executable_path.replace("THIRD_PARTY_NOTICES.chromedriver", "chromedriver")
                    logger.info(f"ChromeDriverManager, {executable_path}")
                    if not is_executable(executable_path):
                        logger.warning(f"The file at {executable_path} is not executable. Attempting to fix permissions.")
                        make_executable(executable_path)
                    driver = webdriver.Chrome(options=chrome_options)
                    return driver

Best Answer

Found a solution to my problem.

As users user26521526 and zhivago-dr recomended I created a function to adjust the file.

However, while it worked as a python script once it compiled to a .app it did not work. This happened because Mac .app files require their contents to be in specific Application support folders. Using ammar alkotb's answer from the question i can't install the driver in the location specified by python's webdriver-manager, I was able to change the chromedriver install to ~/Library/Application Support/ . Where '~' represents the home directory.

This method works after compiling with pyinstaller.

Thank you for your help and support user26521526 and zhivago-dr.

If anyone runs into this issue in the future I have attached my code snippet below as reference.

from webdriver_manager.core.driver_cache import DriverCacheManager
import os
import sys
import stat
from selenium import webdriver
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager

def chrome_driver_setup():
    chrome_options = webdriver.ChromeOptions()
    chrome_options.add_argument("--headless")  # Run Chrome in headless mode
    chrome_options.add_argument("--no-sandbox")  # Bypass OS security model
    chrome_options.add_argument("--disable-dev-shm-usage")  # Overcome limited resource problems

    def make_executable(path):
        # Change the file permissions to make it executable
        st = os.stat(path)
        os.chmod(path, st.st_mode | stat.S_IEXEC)

    def is_executable(file_path):
        # Check if the file is executable
        return os.path.isfile(file_path) and os.access(file_path, os.X_OK)   

    def third_party_notice_correction(executable_path):
        # need to do this because it installs incorrectly sometimes
        if executable_path.endswith("THIRD_PARTY_NOTICES.chromedriver"):
            executable_path = executable_path.replace("THIRD_PARTY_NOTICES.chromedriver", "chromedriver")
        logger.info(f"ChromeDriverManager, {executable_path}")
        if not is_executable(executable_path):
            logger.warning(f"The file at {executable_path} is not executable. Attempting to fix permissions.")
            make_executable(executable_path)
    try:
        #Mac needs the apps resources to be in a specific location for .app applications
        logger.info(f"ChromeDriver not working. Trying Chad Mac Path Method \n {e}")
        install_path = get_path()
        
        #Using Cache manager to specify path to be in the Mac Resource Path
        cache_manager=DriverCacheManager(install_path)
        
        executable_path = ChromeDriverManager(cache_manager=cache_manager).install()
        
        third_party_notice_correction(executable_path)
        
        driver = webdriver.Chrome(service=ChromeService(executable_path), options=chrome_options)
        return driver
        
    except Exception as e:
        logger.info(f"All Driver Initialization Attempts Failed \n {e}")
Related Question