Business
Jobs
  • About Us
  • Solutions
    • Job Postings
      Post your job and receive qualified candidates in 48h.
    • Candidate Assessments
      500+ technical and psychological tests, plus anti-fraud.
    • Headhunting
      Tailor-made executive search from start to finish.
    • Payroll + EOR
      Payroll dispersal and EOR across 15+ LATAM countries.
  • Pricing
  • Jobs

0

525
Views
SKIP/OFFSET/ScanRange of rows in AWS S3 Select

I wonder if anyone figured out how to skip rows in S3 Select?

SELECT S.* FROM s3object S SKIP 100 LIMIT 200
--or
SELECT * from s3object s LIMIT 5, 10
--or
SELECT * from s3object s limit 5 OFFSET 10

It looks like you can limit number of records returned

s3 = boto3.client('s3')
bucket = bucket
file_name = file

sql_stmt = """SELECT S.* FROM s3object S LIMIT 10"""


req = s3.select_object_content(
    Bucket=bucket,
    Key=file,
    ExpressionType='SQL',
    Expression=sql_stmt,
    InputSerialization = {'CSV': {'FileHeaderInfo': 'USE'}},
    OutputSerialization = {'CSV': {}},
)

There was also a request made to add OFFSET/SKIP to s3api, but it was closed.

Also you can specify ScanRange in bytes, but what happens if object is compressed?

Is it range in bytes of compressed object or uncompressed?

If uncompressed how does S3 Select handle partial records?

Update: You cannot use ScanRange on gzip file:

botocore.exceptions.ClientError: An error occurred (UnsupportedScanRangeInput) when calling the SelectObjectContent operation: Scan range queries are not supported on objects with type GZIP.

over 4 years ago · Santiago Trujillo
Answer question
Find remote jobs

Discover the new way to find a job!

Top jobs
Top job categories
Business
Post vacancy Pricing Sales
Legal
Terms and conditions Privacy policy
© 2026 PeakU Inc. All Rights Reserved.
Andres GPT
Show me some job opportunities
There's an error!