Category Archives: Guides

Upload an Image to S3 Using Post and a Presigned Url

Hi everyone,

Today I’ve been converting my “PUT” upload to S3 to a “POST”. The main motivator for this was to restrict the file size of uploads using a signed policy. Unfortunately this was a pretty tedious process and the error responses from S3 were very vague. Thankfully it’s working now and here’s what I needed to do!

Here’s an excerpt from my nodejs lambda function that generates the presigned url:

/* Creates a pre-signed upload url and then stores a reference in a table */
exports.createUploadUrl = async params => {

    var { databaseHandler, bucketHandler } = params;

    // Create id for upload
    var uploadId = uuidv4();

    // Retrieve pre-signed url
    var bucketDataPromise = createPresignedPostPromise({
        Bucket: process.env.BUCKET_UPLOADS,
        Expires: 60 * 60,
        Conditions: [            
            ["content-length-range", 0, 300000], // 300kb
            [ "eq", "$key", uploadId],
            [ "eq", "$Content-Type", 'image/jpeg' ],

    // var ddbData = await ddbDataPromise;
    var bucketData = await bucketDataPromise;

    // Wait for database handler to complete operation and then return
    return Helpers.generateResponse({
        data: {
            uploadData: bucketData,
            additionalFields: {
                key: uploadId,
                "Content-Type": 'image/jpeg',
        statusCode: 200

You can then retrieve the upload details using a request like the following (Python):

resp = requests.put("", data=open(path, 'rb'))

This will return a response similar to the following:

    "messages": [],
    "data": {
        "uploadUrl": {
            "url": "YOUR UPLOAD URL",
            "fields": {
                "bucket": "YOUR BUCKET",
                "X-Amz-Algorithm": "AWS4-HMAC-SHA256",
                "X-Amz-Credential": "XXX",
                "X-Amz-Date": "20190107T125044Z",
                "X-Amz-Security-Token": "SECURITY_TOKEN",
                "Policy": "YOUR BASE64 ENCODED POLICY",
                "X-Amz-Signature": "SIGNATURE"
        "uploadId": "UPLOAD_ID"

And once you have those details you can use them to upload the image:

# Attempt to retrieve upload url etc
json = resp.json()
data = json["data"]
uploadUrl = data["uploadData"]["url"]
uploadFields = data["uploadData"]["fields"]

    print("Uploading image...",end='')
    headers = {'Content-Type': 'multipart/form-data' }
    files = { 'file': open(path, 'rb')}
    resp =, data=uploadFields, files=files)

    # Only show content if there's an error
    if resp.status_code == 200:

except Exception as e:
    print("\r\nFailed to upload image.")
    print("Upload data:")

Hopefully that’s been able to help you out, but feel free to let me know in the comments below if you need more info!

Thanks to these links for the info:

Cognito Auth with AWS SAM

Hi everyone,

I’ve spent today implementing Cognito with AWS SAM and it took quite a while to work out what needed to be done – unfortunately there’s a lot of conflicting doco out there. Posting a sample template just in case it’s able to help anyone else out.

The first thing to do is to explicitly define you rest API. By default AWS SAM will generate one with a default logical id of ServerlessRestApi. You’ll need to override this:

  # See links for more info
  # Referencing cognito authorizer:
  # Logical id is auto generated:
  # Sample template.yml:
    Type: "AWS::Serverless::Api"
      StageName: Prod
      Auth: # We will eventually define other auth options here such as Usage Plans/Api Keys, AWS_IAM, and Resource Policies
        DefaultAuthorizer: MyCustomCognitoAuthorizer
            UserPoolArn: !GetAtt MyCustomCognitoUserPool.Arn # Can be a string, or array
            # Identity: # Optional
              # Header: ... # Optional; Default: Authorization
              # ValidationExpression: ...  # Optional; ensures the request header matches a pattern before checking in with the Authorizer endpoint; is there a default we can set for Cognito User Pools Auth?

You’ll also need to create a user pool and client:

# Creating a cognito user pool -
    Type: AWS::Cognito::UserPool
      UserPoolName: !Ref CognitoUserPoolName
      # LambdaConfig:
        # PreSignUp: !GetAtt PreSignupLambdaFunction.Arn
          MinimumLength: 8
        - email
        - AttributeDataType: String
          Name: email
          Required: false

    Type: AWS::Cognito::UserPoolClient
      UserPoolId: !Ref MyCustomCognitoUserPool
      ClientName: !Ref CognitoUserPoolClientName
      GenerateSecret: false

You then add the api id and the auth attribute to each of your function properties. If you’ve used the default authorizer property when defining the associated api you can override it by using authorizer: none.

    Type: AWS::Serverless::Function
      Handler: breed.getBreedHandler
      Policies: arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
      Runtime: nodejs8.10
          TABLE_CAT_BREED: !Ref CatBreedTable
          Type: Api
            Path: /breed/{breedId}
            Method: get
            RestApiId: !Ref MyCustomCustomApi
              Authorizer: NONE

Hopefully that’s able to help you out. If you run into any trouble I found the following link pretty useful:

AWS SAM Cheat Sheet/Quick Reference

Hi everyone,

This will just be a living cheat sheet or quick reference for using AWS SAM with nodejs. I’ll add to it as I find things that are useful or I might need again.

General AWS SAM Info:

Description Example More
Validate template sam validate
Start AWS Sam Locally
Access config variable in Lambda const CAT_BREED_TABLE = process.env.CAT_BREED_TABLE;
Dynamically reference another resource in template.yml CAT_BREED_TABLE: !Ref DynamoDbCatBreedTable

Dynamo DB Local Info:

Description Example More
Create table aws dynamodb create-table –table-name CatBreeds –attribute-definitions AttributeName=CatBreedId,AttributeType=S –key-schema AttributeName=CatBreedId,KeyType=HASH –provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
List Tables aws dynamodb list-tables –endpoint-url
List All Rows in Table aws dynamodb scan –table-name PestinatorTable –endpoint-url

Node.js Info:

Description Example More
Run tests npm test

Create DynamoDB Table – AWS CLI

Hi everyone,

A quick example of how to create a dynamodb table using the AWS CLI:

aws dynamodb create-table --table-name CatBreeds --attribute-definitions AttributeName=CatBreedId,AttributeType=S --key-schema AttributeName=CatBreedId,KeyType=HASH --provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5

For more info the following AWS page helped me:

Update Wireless Info on Raspberry Pi

Hi everyone,

Just a quick post on how to configure wireless info on a raspberry pi. To start with, open the config file:

sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

You can then add any number of connections to the file with varying priorities:

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev

        ssid="Mobile Network"

        ssid="Home Network"

Then save the file. Your raspberry pi will now connect to the network with the highest priority first (home network), when that’s not available it will use the network with the lower priority (mobile network).

The official doco is actually pretty good for this use case if you need more info:

Fetch As Google Error – ReactJs

Hi everyone,

I ran into the following error while using “fetch as Google” and none of my pages were being indexed correctly:

Uncaught TypeError: undefined is not a function

It took a while to find a solution but after some Googling I found that GoogleBot currently uses Chrome v41. You can download the mini installer of Chromium v41 which will run in parallel with your existing Chrome version:

After downloading Chromium you can just debug normally via the console. In my case, Chrome v41 didn’t like the following command:


If you’re using a front-end framework like ReactJS, AngularJS or Vue you’ll often just need to add babel polyfill:

npm install babel-core --save-dev

Then add this as the first line in your entry point e.g. index.js:

import 'babel-core/polyfill';

It’s important that the polyfill import is added as the first line otherwise anything added before it won’t work.

If you’re getting an error about Headers, you may also need to install the following:

npm install whatwg-fetch --save

Add then add this import just below your babel polyfill:

import 'whatwg-fetch'

Thanks to tomekrudzki on Reddit for the link to Chromium v41: Chrome 41 the key to successful website rendering



.NET Core 2.1 is Returning JSON with Lowercase Properties

Hi everyone,

Just testing out .Net Core 2.1 and noticed that all of my JSON properties are being returned in lowercase despite being defined as uppercase. It turns out that this is configurable:

// Change from this

// To this
        .AddJsonOptions(options => options.SerializerSettings.ContractResolver = new DefaultContractResolver());

Check out these links for more info:

Dynamic Robots.txt with Web Api 2

Hi everyone,

For a project I’m currently working on I needed a dynamic robots.txt. Because our test environment is public facing we want to keep it from being indexed by Google etc. It took a bit of Googling to find a solution that worked, but in the end it was actually pretty simple.

Here’s the action in one of the API Controllers:

    public class UtilitiesController : CustomBaseApiController
        public HttpResponseMessage GetRobotsFile()
            var resp = new HttpResponseMessage(HttpStatusCode.OK);
            var stringBuilder = new StringBuilder();

            if (Helpers.IsProduction())
                // Allow bots in production
                stringBuilder.AppendLine("user-agent: *");
                stringBuilder.AppendLine("disallow: ");
                // Don't allow bots in non-production environments
                stringBuilder.AppendLine("user-agent: *");
                stringBuilder.AppendLine("disallow: *");

            resp.Content = new StringContent(stringBuilder.ToString());

            return resp;

Also need to add the following to your web.config so that the robots.txt file can processed by the routing handler. Without this IIS will attempt to serve it as a static file and will return a 404 when it’s not found:


In production you’ll end up with the following:

user-agent: *

And any other environments:

user-agent: *
disallow: *

Thanks to these answers on stackoverflow for the info:

Including an Externally Hosted Script in ReactJs

Hi everyone,

Just a quick example of how to include an externally hosted js file in a reactjs app. In index.html add your script tag as you would in a normal app:

Then in the file you plan to use the library you can add a const and reference it normally:

const promisepay = window.promisepay;

Thanks to this link for the info: