I'm using python requests_html to scrape a website behind a login. I've done this previously without issue, but now the site appends a query paramenter of sessionCacheKey to each url to make a valid request. The sessionCacheKey variable is stored in the browsers local and session storage. Does anyone know how or if it is possible to access local or session storage with python/requests_html? I have tried manually copying the session variable from chrome into my python request and it works. I just need to know how I can access it and store it in a variable in python to use for all future requests. Thanks for any help.
Below is the script assigning sessionCacheKey to local and session storage.
'setSessionCacheKey': function(){
externalLoginCheck();
var currentSessionCacheKey = getQueryParamByName('sessionCacheKey', window.location.href);
window.sessionStorage.setItem('sessionCacheKey', currentSessionCacheKey);
window.localStorage.setItem('sessionCacheKey', currentSessionCacheKey);
if (duplicateTabConditions()) {
var newSessionCacheKey = getNewSessionCacheKey(currentSessionCacheKey);
$.ajax({
url: '/AM/RemoteProxy/SessionManagerProxy.cfc?method=copySessionCacheToNewKey&SourceSessionCacheKey=' + currentSessionCacheKey + '&DestinationSessionCacheKey=' + newSessionCacheKey,
success: function (result) {
reloadLocation = removeParam("sessionCacheKey", window.location.href);
var separator = reloadLocation.match(/\?/) ? '&' : '?';
window.location = reloadLocation + separator + 'sessionCacheKey=' + newSessionCacheKey;
},
async: false
});
}