I’ve been hanging around at repost.aws for a while because the best way to keep your skills fresh is to see what challenges other people have in real life and check if can solve those. And maybe learn something new during the process. That is also how this post got started.

I am a complete newby to AWS and HTML and want to create a pdf file sharing solution for my social group. I have created a s3 bucket with my pdf’s in a folder. when I specify a single file the web page works ok ie

<iframe src="Songs/my file name.pdf""> </iframe>

I cannot find a way to specify all the files using a wild card and I cannot specify each file individually so how can it be done.

When you ignore some of the context, what is left is the question how to list objects from S3 bucket on a webpage or an application. This may first sound trivial but keep reading …

Server-side solutions

Obvious solution would be rendering the page on server-side. You could write a piece of code to list the bucket content and transform that into something that can be presented in browser or application. As everyone should know by now, inserting AWS API key (or any secrets) into your code base is serious anti-pattern. Luckily this is very easy to avoid using instance profiles or IAM roles that take transparently care of assigning temporary API keys for your code to use.

And even if you would be running server outside of AWS, there is similar functionality (with little bit more setup effort) called IAM Roles Everywhere you could use instead of static API keys.

All above makes server-side implementation too trivial to be interesting. But what if you don’t want to have any of application code running on your servers, including serverless solutions ;-)

Problem with client-side secrets

Challenge with client-side solutions isn’t in writing the code that can run in a browser or creating a client application. Just pick the SDK for your favourite language and go build! Problem is how to authenticate for AWS API without exposing your API key & secret to the client. By default you should assume anything distributed with client will be exposed to the public. You could craft the IAM policy very carefully so it would allow access to things that are already public but even then there is a risk that you made a small mistake in IAM policy (we all know how easy that is) or someone accidentially alters the policy later.

Amazon Cognito helps in building end-user identity and access management. It can also provide temporary access to AWS APIs similar to instance profiles and IAM roles for server-side solutions. Cognito also allows you to define a separate IAM role with limited permissions for guest users who are not authenticated. This removes unnecessary login step, but still enables you to use temporary, limited privilege credentials to access AWS resources.

While Cognito helps when you want to provide access to AWS resources for authenticated users, identity pool and permission setup is bit complex if you only want to provide anonymous access to S3 (or similar).

Anonymous access

For S3 there is also truly anonymous access, meaning you don’t need any API key to access public buckets and objects. Easiest way to test this is with AWS S3 CLI. When you add -no-sign-request option, no credentials are used but I can still get the list of objects.

% aws s3 ls s3://s3.carriagereturn.nl/ --region eu-north-1 --no-sign-request
2024-07-31 18:42:27     119409 hello.jpg
2024-07-31 18:41:17         15 hello.json
2024-07-31 18:41:17         12 hello.txt

And to verify I didn’t have any credentialsi, the same command fails when I try to sign the request.

% aws s3 ls s3://s3.carriagereturn.nl/ --region eu-north-1                  
An error occurred (InvalidAccessKeyId) when calling the ListObjectsV2 operation: The AWS Access Key Id you provided does not exist in our records.

For above to work, bucket policy must allow s3:ListBucket and s3:GetObject.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "PublicRead",
            "Effect": "Allow",
            "Principal": "*",
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::s3.carriagereturn.nl",
                "arn:aws:s3:::s3.carriagereturn.nl/*"
            ]
        }
    ]
}

As the challenge was to create a webpage with dynamic listing of bucket content I created a small demo project using AWS Javascript SDK V3 and anonymous access to S3 API.

You can get the code from GitHub repository and build it yourself. If things worked as planned you should see page similar to this screenshot.

In earlier versions of Javascript SDK there was s3.makeUnauthenticatedRequest method but in current versions that doesn’t exists and documentation isn’t very helpful either :-(

Work-a-round is to define signer as shown below.

    // Initialize an S3 service with anonymous credentials.
    const s3 = new S3({
        region: region,
        signer: {
            sign: async (request) => request
        }
        ...

Summary

If you think you must include static credentials with any client, or server, side code, think again. As shown above, there are multiple ways to avoid using long term AWS API secrets. In some cases you can also make the code work without any API authentication. Yes, it is only limited S3 use-cases but when that is all you need, anonymous access can simplify your application architecture and avoid running application code on server-side, making the application truly serverless ;-)

PS. If you are looking for a bit more ambitious implementation of S3 access from a browser, using Cognito authentication, take a look at public-file-browser-for-amazon-s3.