Recently, I have started to occupy the AWS platform, but when trying to occupy Sagemaker, the following error and I don't know if it is because of Sagemaker or it has something to do with ´´Parquet Dataset´´
session = sagemaker.Session()
region = boto3.Session().region_name
role = get_execution_role()
import pyarrow.parquet as pq
import s3fs
s3 = s3fs.S3FileSystem()
bucket = 's3://xx'
df = pq.ParquetDataset(bucket, filesystem=s3).read_pandas().to_pandas()
Up to this point everything loads fine, but I have the following error
attributeerror: 'AioClientCreator' object has no attribute '_register_lazy_block_unknown_fips_pseudo_regions'
I do not know, what could be the error
Same here. According to this issue, it's having to do with botocore removing this function.
This function in question (_register_lazy_block_unknown_fips_pseudo_regions) was intentionally removed as it was used to prevent unknown/new region endpoint variants from actually being used to make API calls while allowing the client to be constructed, and seems to have been added to aibotocore inadvertently.
I was able to resolve the issue by using botocore==1.22.5.
Edit:
I've since tried again to get everything working together. Here is my Pipfile which seems to work without throwing errors like those reported by @Drwhit. I believe that error is coming from this issue.
s3fs = {version = "~=2021.4", extras = ["boto3"]}
aiohttp = "==3.7.2"
botocore = "==1.22.8"
aiobotocore = "==2.0.1"
For making this work, in an environment with some libraries already installed, I needed to issue some uninstall commands, before installing the old boto3 version:
pip uninstall boto3
pip uninstall botocore
pip uninstall aiobotocore
Then issued this one:
pip install boto3==1.17.106
...and finally worked.
Should any conflict issue arises when installing that specific boto3 version, just uninstall the package that is in conflict (happened to me with a version of sagemaker). If you need it, reinstall after installing boto3 old version. No guarantee that will work at the end; trial and error will help.
When done, issue pip list, so you can get the complete list of versions that work, and put it with the appropriate syntax changes into a requirements.txt file, so you can build your environment later.