All of my Selenium scripts are raising errors after Chrome updated to version 127 because I always have to select a default search engine when the browser is being launched.
I use ChromeDriver 127.0.6533.72.
Is anyone experiencing the same issue?
pythonselenium-webdriver
All of my Selenium scripts are raising errors after Chrome updated to version 127 because I always have to select a default search engine when the browser is being launched.
I use ChromeDriver 127.0.6533.72.
Is anyone experiencing the same issue?
Basically, the way the Selenium detection works, is that they test for predefined JavaScript variables which appear when running with Selenium. The bot detection scripts usually look anything containing word "selenium" / "webdriver" in any of the variables (on window object), and also document variables called $cdc_
and $wdc_
. Of course, all of this depends on which browser you are on. All the different browsers expose different things.
For me, I used Chrome, so, all that I had to do was to ensure that $cdc_
didn't exist anymore as a document variable, and voilĂ (download chromedriver source code, modify chromedriver and re-compile $cdc_
under different name.)
This is the function I modified in chromedriver:
function getPageCache(opt_doc) {
var doc = opt_doc || document;
//var key = '$cdc_asdjflasutopfhvcZLmcfl_';
var key = 'randomblabla_';
if (!(key in doc))
doc[key] = new Cache();
return doc[key];
}
(Note the comment. All I did I turned $cdc_
to randomblabla_
.)
Here is pseudocode which demonstrates some of the techniques that bot networks might use:
runBotDetection = function () {
var documentDetectionKeys = [
"__webdriver_evaluate",
"__selenium_evaluate",
"__webdriver_script_function",
"__webdriver_script_func",
"__webdriver_script_fn",
"__fxdriver_evaluate",
"__driver_unwrapped",
"__webdriver_unwrapped",
"__driver_evaluate",
"__selenium_unwrapped",
"__fxdriver_unwrapped",
];
var windowDetectionKeys = [
"_phantom",
"__nightmare",
"_selenium",
"callPhantom",
"callSelenium",
"_Selenium_IDE_Recorder",
];
for (const windowDetectionKey in windowDetectionKeys) {
const windowDetectionKeyValue = windowDetectionKeys[windowDetectionKey];
if (window[windowDetectionKeyValue]) {
return true;
}
};
for (const documentDetectionKey in documentDetectionKeys) {
const documentDetectionKeyValue = documentDetectionKeys[documentDetectionKey];
if (window['document'][documentDetectionKeyValue]) {
return true;
}
};
for (const documentKey in window['document']) {
if (documentKey.match(/\$[a-z]dc_/) && window['document'][documentKey]['cache_']) {
return true;
}
}
if (window['external'] && window['external'].toString() && (window['external'].toString()['indexOf']('Sequentum') != -1)) return true;
if (window['document']['documentElement']['getAttribute']('selenium')) return true;
if (window['document']['documentElement']['getAttribute']('webdriver')) return true;
if (window['document']['documentElement']['getAttribute']('driver')) return true;
return false;
};
According to answer, there are multiple methods to remove them. One of them is simply opening chromedriver.exe
with a HEX-editor and removing all occurences of $cdc_
No, there is no other alternative than updating the ChromeDriver binary versions, while the Chrome Browser keeps on getting updated automatically.
Each Chrome Browser is released after certain feature additions, modifications and removals from the existing features. To comply with the current set of Browser Features, Chrome Team releases the compatible ChromeDriver binary time to time. These ChromeDriver binaries are capable to interact with the Chrome Browser. Certain version of a ChromeDriver binary supports a specific range of Chrome Browser versions (some of the recent) as below:
ChromeDriver v84.0.4147.30 (2020-05-28)
Supports Chrome version 84
ChromeDriver v83.0.4103.39 (2020-05-05)
Supports Chrome version 83
ChromeDriver v82 was intensionally skipped.
ChromeDriver v81.0.4044.138 (2020-05-05)
Supports Chrome version 81
ChromeDriver v80.0.3987.106 (2020-02-13)
Supports Chrome version 80
ChromeDriver v79.0.3945.36 (2019-11-18)
Supports Chrome version 79
ChromeDriver v78.0.3904.70 (2019-10-21)
Supports Chrome version 78
ChromeDriver v77.0.3865.40 (2019-08-20)
Supports Chrome version 77
ChromeDriver v76.0.3809.126 (2019-08-20)
Supports Chrome version 76
ChromeDriver v75.0.3770.8 (2019-04-29)
Supports Chrome version 75
ChromeDriver v74.0.3729.6 (2019-03-14)
Supports Chrome version 74
ChromeDriver v73.0.3683.68 (2019-03-06)
Supports Chrome version 73
ChromeDriver v2.46 (2019-02-01)
Supports Chrome v71-73
ChromeDriver v2.45 (2018-12-10)
Supports Chrome v70-72
ChromeDriver v2.44 (2018-11-19)
Supports Chrome v69-71
ChromeDriver v2.43 (2018-10-16)
Supports Chrome v69-71
ChromeDriver v2.42 (2018-09-13)
Supports Chrome v68-70
ChromeDriver v2.41 (2018-07-27)
Supports Chrome v67-69
ChromeDriver v2.40 (2018-06-07)
Supports Chrome v66-68
ChromeDriver v2.39 (2018-05-30)
Supports Chrome v66-68
ChromeDriver v2.38 (2018-04-17)
Supports Chrome v65-67
ChromeDriver v2.37 (2018-03-16)
Supports Chrome v64-66
ChromeDriver v2.36 (2018-03-02)
Supports Chrome v63-65
ChromeDriver v2.35 (2018-01-10)
Supports Chrome v62-64
ChromeDriver v2.34 (2017-12-10)
Supports Chrome v61-63
ChromeDriver v2.33 (2017-10-03)
Supports Chrome v60-62
ChromeDriver v2.32 (2017-08-30)
Supports Chrome v59-61
ChromeDriver v2.31 (2017-07-21)
Supports Chrome v58-60
ChromeDriver v2.30 (2017-06-07)
Supports Chrome v58-60
ChromeDriver v2.29 (2017-04-04)
Supports Chrome v56-58
To keep your script/program interactive with the updated Chrome Browser you have to keep the version of ChromeDriver binary in sync with the Chrome Browser as per the compatibility.
Best Answer
Just need to add this Chrome Options:
options.addArguments("--disable-search-engine-choice-screen");