I want to write a spark application in which I read from multiple hive tables and writes to multiple hive table . However here is the quirk.

All the hive tables are external hive tables with data residing on S3.

Some s3 buckets require s3:sse encryption when writing and some buckets do not .

From my understand , I can only set the property fs.s3a.server-side-encryption-algorithm at a global level .

How do I go about this . I am pretty certain this is a common use case.

I guess the question I am asking is the following.

consider that I am reading from a hive table and writing to a hive table within a spark application . consider that both hive tables are external with data on S3. however , the table reads from a non encrypted bucket and writes into a bucket that requires encryption.