Category Archives: Guides

View Raw SQL in EF Core – Simple Option

Hi everyone,

I’ve been looking for a simple way of viewing the raw output of sql in my local environment without having to make code changes and came across the following config setup:

{
“Logging”: {
“LogLevel”: {
“Default”: “Debug”,
“System”: “Information”,
“Microsoft”: “Information”
}
}

This will show the sql statements in your output window without requiring any additional code:
sql-output

Thanks to this stackoverflow post for the answer: https://stackoverflow.com/a/54704006/522859

Simple React-Leaflet Location Picker Setup

Hi everyone,

Just thought I’d share a very simple react-leaflet setup in case it’s able to help anyone.

screencast-localhost_3889-2019.08.24-14_01_41.gif

Add the css and javascript for leaflet to public > index.html:

  
https://unpkg.com/leaflet@1.5.1/dist/leaflet.js">https://unpkg.com/leaflet@1.5.1/dist/leaflet.js

Install react-leaflet via npm or yarn, also add the dependencies:

npm install react-leaflet # npm
npm install leaflet react react-dom # npm

yarn add react-leaflet # Yarn
yarn add leaflet react react-dom # Yarn

Create the map component:

import React, { useState } from 'react'
import { Map, TileLayer, Marker, Popup } from 'react-leaflet'


const MapSimple = () => {

    const [location, setLocation] = useState({
        lat: 51.505,
        lng: -0.09,
        zoom: 13,
    });
    const position = [location.lat, location.lng];
    
    const setMarkerPosition = e => {
        setLocation({ ...e.latlng, zoom: 19 });
        console.log(`My location is: ${JSON.stringify(e.latlng, null, 3)}`)
    };

    return (
        
            <TileLayer
                attribution='© OpenStreetMap contributors'
                url="https://{s}.tile.openstreetmap.org/{z}/{x}/{y}.png"
            />
            
                
                   A sample popup.
                
            
        
    )
}

export default MapSimple;

NetTopologySuite Circular Reference with .net core 2.2

Hey everyone,

Testing out a spatial project with .net core and I ran into the following error:

Microsoft.CSharp.RuntimeBinder.RuntimeBinderException : The best overloaded method match for ‘Xunit.Assert.Equal(string, string)’ has some invalid arguments

at CallSite.Target(Closure , CallSite , Type , Nullable`1 , Object )
at UpdateDelegates.UpdateAndExecuteVoid3[T0,T1,T2](CallSite site, T0 arg0, T1 arg1, T2 arg2)
at DiscussionsControllerIntegrationTests.CreateTestDiscussion(HttpClient client, ApplicationDbContext db, DiscussionCreateWebRequest payload) in DiscussionsControllerIntegrationTests.cs line: 56
at DiscussionsControllerIntegrationTests.b__1_0(ApplicationDbContext db) in DiscussionsControllerIntegrationTests.cs line: 27
at IntegrationTestBase.RunTest(Func`2 testToExecute) in IntegrationTestBase.cs line: 54
at DiscussionsControllerIntegrationTests.CreateDiscussionIsSuccessful() in DiscussionsControllerIntegrationTests.cs line: 25
at — End of stack trace from previous location where exception was thrown —

Thankfully the solution is pretty straight forward. Install GeoJSON:

Install-Package NetTopologySuite.IO.GeoJSON

Then merge the following with your existing addMvc call in startup.cs:

services.AddMvc(options =>
            {
                // Prevent the following exception: 'This method does not support GeometryCollection arguments'
                // See: https://github.com/npgsql/Npgsql.EntityFrameworkCore.PostgreSQL/issues/585
                options.ModelMetadataDetailsProviders.Add(new SuppressChildValidationMetadataProvider(typeof(Point)));
                options.ModelMetadataDetailsProviders.Add(new SuppressChildValidationMetadataProvider(typeof(Coordinate)));
                options.ModelMetadataDetailsProviders.Add(new SuppressChildValidationMetadataProvider(typeof(LineString)));
                options.ModelMetadataDetailsProviders.Add(new SuppressChildValidationMetadataProvider(typeof(MultiLineString)));
            })
                .AddJsonOptions(options =>
                {
                    foreach (var converter in NetTopologySuite.IO.GeoJsonSerializer.Create(new GeometryFactory(new PrecisionModel(), 4326)).Converters)
                    {
                        options.SerializerSettings.Converters.Add(converter);
                    }
                })
                .SetCompatibilityVersion(CompatibilityVersion.Version_2_2);

String to Color – Avatar Helpers

Hi everyone,

A quick post on a helper function I came across for generating avatars. This function generates a random color for the specified string:


export const stringToColor = string => {

let hash = 0;
let i;

for (i = 0; i < string.length; i += 1) {
hash = string.charCodeAt(i) + ((hash << 5) - hash);
}

let colour = '#';

for (i = 0; i > (i * 8)) & 0xff;
colour += `00${value.toString(16)}`.substr(-2);
}

return colour;
}

Thanks to oliviertassinari: https://github.com/oliviertassinari/SplitMe/blob/c25fe62187a856386fd0c43c51859d6f973d651e/src/main/member/Avatar.js#L8-L26

Move Selection to Single Line – Visual Code

Hi everyone,

A quick post on how to merge a multiline block into a single line using Visual Studio Code:

-- From this
CREATE TABLE posts
(
    post_id INT IDENTITY(1,1) PRIMARY KEY,
    title VARCHAR(100),
    description VARCHAR(MAX),
    user_id VARCHAR(255) NOT NULL,
    created DATETIME NOT NULL,
    updated DATETIME NOT NULL,
    version INT NOT NULL,
    up_votes INT NOT NULL,
    down_votes INT NOT NULL,
    updated DATETIME NOT NULL
);

-- To this
CREATE TABLE posts ( post_id INT IDENTITY(1,1) PRIMARY KEY, title VARCHAR(100), description VARCHAR(MAX), user_id VARCHAR(255) NOT NULL, created DATETIME NOT NULL, updated DATETIME NOT NULL, version INT NOT NULL, up_votes INT NOT NULL, down_votes INT NOT NULL, updated DATETIME NOT NULL )

Simply highlight your code block, press F1, and then type “join lines”. Pressing enter will join everything. If you’re on a mac you can also use ctrl+j.

Overriding Global Variables – AWS SAM Local

Hi everyone,

Today I’ve added local overrides to the global variables in my template.yml file. This was pretty finicky – a lot of conflicting suggestions out there unfortunately. These are the settings that ended up working for me.

local-env-var-overrides.json:

{
    "Parameters": {
        "RDS_HOSTNAME_TEST": "192.168.0.31"
    }
}

template.yaml

Globals:
  Function:
    Environment:
      Variables:        
        RDS_HOSTNAME_TEST:
          Type: String
          Default: 'DEFAULT'

aws sam local command:

sam local start-api --env-vars "local-env-var-overrides.json"

create.js

exports.createTrapHandler = async (event, context, callback) => {

	try {

        console.log("RDS_HOSTNAME_TEST: " + process.env.RDS_HOSTNAME_TEST)
        // Outputs: 192.168.0.31

This link ended up being pretty useful: https://www.npmjs.com/package/aws-sam-local

Get User Id in Lambda node.js

Hi everyone,

A quick post on where to find the user id (sub) in a lambda requested that has been authenticated with a congito authorizer.

You’ll be able to find everything you need in the event object under requestContext > authorizer > claims:


exports.viewContextHandler = async (event, context, callback) => {
    console.log(JSON.stringify(event.requestContext));
}

"requestContext": {
    "resourceId": "XXXXX",
    "authorizer": {
        "claims": {
            "at_hash": "XXXXX",
            "sub": "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX",
            "aud": "XXXXX12341234512345XXXXX",
            "email_verified": "true",
            "token_use": "id",
            "auth_time": "1547371205",
            "iss": "https://cognito-XXXXX.com/XXXXX",
            "cognito:username": "XXXXX-XXXXX-XXXXX-XXXXX-XXXXX",
            "exp": "Sun Jan 13 10:20:05 UTC 2019",
            "iat": "Sun Jan 13 09:20:05 UTC 2019",
            "email": "XXXXX@XXXXX.XXXXX"
        }
    },
}

Upload an Image to S3 Using Post and a Presigned Url

Hi everyone,

Today I’ve been converting my “PUT” upload to S3 to a “POST”. The main motivator for this was to restrict the file size of uploads using a signed policy. Unfortunately this was a pretty tedious process and the error responses from S3 were very vague. Thankfully it’s working now and here’s what I needed to do!

Here’s an excerpt from my nodejs lambda function that generates the presigned url:

/* Creates a pre-signed upload url and then stores a reference in a table */
exports.createUploadUrl = async params => {

    var { databaseHandler, bucketHandler } = params;

    // Create id for upload
    var uploadId = uuidv4();

    // Retrieve pre-signed url
    var bucketDataPromise = createPresignedPostPromise({
        Bucket: process.env.BUCKET_UPLOADS,
        Expires: 60 * 60,
        Conditions: [            
            ["content-length-range", 0, 300000], // 300kb
            [ "eq", "$key", uploadId],
            [ "eq", "$Content-Type", 'image/jpeg' ],
        ]
    });

    // var ddbData = await ddbDataPromise;
    var bucketData = await bucketDataPromise;

    // Wait for database handler to complete operation and then return
    return Helpers.generateResponse({
        data: {
            uploadData: bucketData,
            additionalFields: {
                key: uploadId,
                "Content-Type": 'image/jpeg',
            },
        },
        statusCode: 200
    });
}

You can then retrieve the upload details using a request like the following (Python):

resp = requests.put("https://f86caxqe9f.execute-api.ap-southeast-2.amazonaws.com/Prod/images", data=open(path, 'rb'))

This will return a response similar to the following:

{
    "messages": [],
    "data": {
        "uploadUrl": {
            "url": "YOUR UPLOAD URL",
            "fields": {
                "bucket": "YOUR BUCKET",
                "X-Amz-Algorithm": "AWS4-HMAC-SHA256",
                "X-Amz-Credential": "XXX",
                "X-Amz-Date": "20190107T125044Z",
                "X-Amz-Security-Token": "SECURITY_TOKEN",
                "Policy": "YOUR BASE64 ENCODED POLICY",
                "X-Amz-Signature": "SIGNATURE"
            }
        },
        "uploadId": "UPLOAD_ID"
    }
}

And once you have those details you can use them to upload the image:

# Attempt to retrieve upload url etc
json = resp.json()
data = json["data"]
uploadUrl = data["uploadData"]["url"]
uploadFields = data["uploadData"]["fields"]
uploadFields.update(data["additionalFields"])

try:
    print("Uploading image...",end='')
    headers = {'Content-Type': 'multipart/form-data' }
    files = { 'file': open(path, 'rb')}
    resp = requests.post(uploadUrl, data=uploadFields, files=files)

    # Only show content if there's an error
    if resp.status_code == 200:
        print("Uploaded")
    else:  
        print("ERROR")                          
        print(repr(resp))
        print(repr(resp.content))

except Exception as e:
    print("\r\nFailed to upload image.")
    print("Upload data:")
    print(repr(uploadFields))
    print("Exception:")
    print(repr(e))
    traceback.print_exc()

Hopefully that’s been able to help you out, but feel free to let me know in the comments below if you need more info!

Thanks to these links for the info:
https://stackoverflow.com/a/35923104/522859
https://stackoverflow.com/a/49311831/522859

Cognito Auth with AWS SAM

Hi everyone,

I’ve spent today implementing Cognito with AWS SAM and it took quite a while to work out what needed to be done – unfortunately there’s a lot of conflicting doco out there. Posting a sample template just in case it’s able to help anyone else out.

The first thing to do is to explicitly define you rest API. By default AWS SAM will generate one with a default logical id of ServerlessRestApi. You’ll need to override this:

Resources:
  # See links for more info
  # Referencing cognito authorizer: https://github.com/awslabs/serverless-application-model/issues/512#issuecomment-411284092
  # Logical id is auto generated: https://github.com/awslabs/serverless-application-model/blob/master/docs/internals/generated_resources.rst#api
  # Sample template.yml: https://github.com/awslabs/serverless-application-model/blob/release/v1.8.0/examples/2016-10-31/api_cognito_auth/template.yaml
  MyCustomApi:
    Type: "AWS::Serverless::Api"
    Properties:
      StageName: Prod
      Auth: # We will eventually define other auth options here such as Usage Plans/Api Keys, AWS_IAM, and Resource Policies
        DefaultAuthorizer: MyCustomCognitoAuthorizer
        Authorizers:
          MyCustomCognitoAuthorizer:
            UserPoolArn: !GetAtt MyCustomCognitoUserPool.Arn # Can be a string, or array
            # Identity: # Optional
              # Header: ... # Optional; Default: Authorization
              # ValidationExpression: ...  # Optional; ensures the request header matches a pattern before checking in with the Authorizer endpoint; is there a default we can set for Cognito User Pools Auth?

You’ll also need to create a user pool and client:

# Creating a cognito user pool - https://github.com/awslabs/serverless-application-model/blob/master/examples/2016-10-31/api_cognito_auth/template.yaml
  MyCustomCognitoUserPool:
    Type: AWS::Cognito::UserPool
    Properties:
      UserPoolName: !Ref CognitoUserPoolName
      # LambdaConfig:
        # PreSignUp: !GetAtt PreSignupLambdaFunction.Arn
      Policies:
        PasswordPolicy:
          MinimumLength: 8
      UsernameAttributes:
        - email
      Schema:
        - AttributeDataType: String
          Name: email
          Required: false

  MyCustomCognitoUserPoolClient:
    Type: AWS::Cognito::UserPoolClient
    Properties:
      UserPoolId: !Ref MyCustomCognitoUserPool
      ClientName: !Ref CognitoUserPoolClientName
      GenerateSecret: false

You then add the api id and the auth attribute to each of your function properties. If you’ve used the default authorizer property when defining the associated api you can override it by using authorizer: none.

GetBreedFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: breed.getBreedHandler
      Policies: arn:aws:iam::aws:policy/AmazonDynamoDBFullAccess
      Runtime: nodejs8.10
      Environment:
        Variables:
          TABLE_CAT_BREED: !Ref CatBreedTable
      Events:
        GetEvent:
          Type: Api
          Properties:
            Path: /breed/{breedId}
            Method: get
            RestApiId: !Ref MyCustomCustomApi
            Auth:
              Authorizer: NONE

Hopefully that’s able to help you out. If you run into any trouble I found the following link pretty useful: https://github.com/awslabs/serverless-application-model/blob/release/v1.8.0/examples/2016-10-31/api_cognito_auth/template.yaml

AWS SAM Cheat Sheet/Quick Reference

Hi everyone,

This will just be a living cheat sheet or quick reference for using AWS SAM with nodejs. I’ll add to it as I find things that are useful or I might need again.

General AWS SAM Info:

Description Example More
Validate template sam validate
Start AWS Sam Locally https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-quick-start.html
Access config variable in Lambda const CAT_BREED_TABLE = process.env.CAT_BREED_TABLE; https://stackoverflow.com/a/48491845/522859
Dynamically reference another resource in template.yml CAT_BREED_TABLE: !Ref DynamoDbCatBreedTable https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html

Dynamo DB Local Info:

Description Example More
Create table aws dynamodb create-table –table-name CatBreeds –attribute-definitions AttributeName=CatBreedId,AttributeType=S –key-schema AttributeName=CatBreedId,KeyType=HASH –provisioned-throughput ReadCapacityUnits=5,WriteCapacityUnits=5
–endpoint-url http://192.168.0.31:8000
https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.CLI.html
List Tables aws dynamodb list-tables –endpoint-url http://192.168.0.31:8000 https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Tools.CLI.html
List All Rows in Table aws dynamodb scan –table-name PestinatorTable –endpoint-url http://192.168.0.31:8000 https://stackoverflow.com/a/52236600/522859

Node.js Info:

Description Example More
Run tests npm test