Like it was said on the comments, if you're not using multipart uploads, you should get that out of the box.

If you are using multipart uploads, there are two things to consider:

1. An abandoned multipart upload consumes storage space in a holding area, but does not leave an incomplete object in the bucket itself. ListObjects, GET, etc., cannot see incomplete multipart uploads, and any existing object remains undisturbed. The parts of an incomplete multipart upload are only accessible using the multipart API, so, in the only relevant sense, all object creation and overwrite operations -- multipart or not -- are automatically and always 100% atomic. More information on that on the multipart upload overview

2. You can create a bucket lifecycle policy that cleans up incomplete uploads (AbortIncompleteMultipartUpload policy), as detailed in the lifecycle guidelines:

   Amazon S3 supports a bucket lifecycle rule that you can use to direct Amazon S3 to abort multipart uploads that don't complete within a specified number of days after being initiated. When a multipart upload is not completed within the time frame, it becomes eligible for an abort operation and Amazon S3 aborts the multipart upload (and deletes the parts associated with the multipart upload).

   You can attach a rule to only certain prefixes or to the entire bucket, which sounds more appropriate to your use case. You can find more details about bucket lifecycles in the S3 lifecycle rules docs.