No S3 objects found under S3 URL

Q: Why is S3 object URL Access Denied?

If youre trying to host a static website using Amazon S3, but youre getting an Access Denied error, check the following requirements: Objects in the bucket must be publicly accessible. S3 bucket policy must allow access to the s3:GetObject action. The AWS account that owns the bucket must also own the object.

Q: How do I find my S3 bucket URL?

Get an S3 Objects URL # Navigate to the AWS S3 console and click on your buckets name. Use the search input to find the object if necessary. Click on the checkbox next to the objects name. Click on the Copy URL button.

TL;DR

S3 pre-signed URLs grant temporary access to objects in AWS S3 buckets without the need to grant explicit permissions.

Inhaltsverzeichnis Show

Pre-Signed URLs
Implementation
Generating S3 pre-signed URLs — The Producer
Accessing S3 objects using pre-signed URLs — The Consumer
Further Reading
Why is S3 object URL Access Denied?
What is object URL in S3?
How do I find my S3 bucket URL?
Why can't I access a specific folder or Amazon S3 bucket?

Supports only GET and PUT requests.
Request headers must exactly match both when creating and using the URLs.

Motivation

It’s a best practice to keep S3 buckets private and only grant public access when absolutely required. Then how can you grant your client access to an object without changing the bucket ACL, creating roles, or providing a user on your account? That’s where S3 pre-signed URLs come in.

Pre-Signed URLs

S3 pre-signed URLs are a form of an S3 URL that temporarily grants restricted access to a single S3 object to perform a single operation — either PUT or GET — for a predefined time limit.

To break it down:

It is secure — the URL is signed using an AWS access key
It grants restricted access — only one of GET or PUT is allowed for a single URL
Only to a single object — each pre-signed URL corresponds to one object
With a time-constrained — the URL expires after a set timeout

In the next section, we’ll use these properties to generate an S3 pre-signed URL. So let’s jump over to some code samples.

Key Points

Building a solution with S3 pre-signed URLs is not without its pitfalls. Here are some pointers to save you valuable time:

You must send the same HTTP headers — when accessing a pre-signed URL — as you used when you generated it. For example, if you generate a pre-signed URL with the Content-Type header, then you must also provide this header when you access the pre-signed URL. Beware that some libraries - for example, Axios - attach default headers, such as Content-Type, if you don't provide your own.
The default pre-signed URL expiration time is 15 minutes. Make sure to adjust this value to your specific needs. Security-wise, you should keep it to the minimum possible — eventually, it depends on your design.
To upload a large file — larger than 10MB — you need to use multi-part upload. I’ll cover this topic in my next post.
Pre-signed URLs support only the getObject, putObject and uploadPart functions from the AWS SDK for S3. It's impossible to grant any other access to an object or a bucket, such as listBucket.
Because of the previously mentioned AWS SDK functions limitation, you can’t use pre-signed URLs as Lambda function sources, since Lambda requires both listBucket and getObject access to an S3 object to use as a source.

Implementation

Now that we have all the information we require let’s get to coding.

Each code sample section contains two parts, the producer of the URL and the consumer. I use the JavaScript AWS SDK to generate the pre-signed URLs and the Axios NPM package to access these URLs from the consumer using basic HTTP requests.

In the examples, I use the constants BUCKET_NAME and OBJECT_NAME to represent my bucket and object - replace these with your own as you see fit. Also, there is a placeholder for theaccesskeyId and secretAccessKey, you can read about it here.

Generating S3 pre-signed URLs — The Producer

var AWS = require('AWS') var cuid = require('cuid') const s3 = new AWS.S3({ accessKeyId: /* Bucket owner access key id */, secretAccessKey: /* Bucket owner secret */, sessionToken: `session-${cuid()}` }) var params = { Bucket: BUCKET_NAME, Key: OBJECT_NAME, ContentType: 'application/json', Expires: 120 // In seconds } const readSignedUrl = await s3.getSignedUrlPromise('getObject', params).promise() // Read access const writeSignedUrl = await s3.getSignedUrlPromise('putObject', params).promise() // Write access

Accessing S3 objects using pre-signed URLs — The Consumer

const axios = require('axios') // Read object axios.get( readSignedUrl, { // The url which was generated headers: { 'Content-Type': 'application/json' }, }) // Write object axios.put( writeSignedUrl, // The url which was generated JSON.stringify(JSON_OBJECT), { // Using your own json object headers: { 'Content-Type': 'application/json' }, })

Why is S3 object URL Access Denied?

If you're trying to host a static website using Amazon S3, but you're getting an Access Denied error, check the following requirements: Objects in the bucket must be publicly accessible. S3 bucket policy must allow access to the s3:GetObject action. The AWS account that owns the bucket must also own the object.

What is object URL in S3?

A presigned URL is a URL that you can provide to your users to grant temporary access to a specific S3 object. Using the URL, a user can either READ the object or WRITE an Object (or update an existing object). The URL contains specific parameters which are set by your application.

How do I find my S3 bucket URL?

Get an S3 Object's URL # Navigate to the AWS S3 console and click on your bucket's name. Use the search input to find the object if necessary. Click on the checkbox next to the object's name. Click on the Copy URL button.

Why can't I access a specific folder or Amazon S3 bucket?

Check the following permissions for any settings that are denying your access to the prefix or object: Ownership of the prefix or object. Restrictions in the bucket policy. Restrictions in your AWS Identity and Access Management (IAM) user policy.