Category Archives: Guides

Support for password authentication was removed on August 13, 2021. Please use a personal access token instead – Fix for Mac

Leave a reply

Hey everyone,

If you’re like me and a bit slack with your personal projects you might’ve started receiving the following error today:

admin@Admins-iMac ui % git push
remote: Support for password authentication was removed on August 13, 2021. Please use a personal access token instead.
remote: Please see https://github.blog/2020-12-15-token-authentication-requirements-for-git-operations/ for more information.
fatal: unable to access 'https://github.com/Buzzology/referrer.git/': The requested URL returned error: 403

As the message says, Github wants you to start using a Personal Access Token (PAT) instead of password authentication. Luckily, the fix is pretty straight forward – you’ll need to create a Personal Access Token and then update your keychain.

Step #1: Create a Personal Access Token (PAT)

To create the personal access token, login to Github and go to the following page: https://github.com/settings/tokens

You can also get to this page via the following:

  1. Click on your profile dropdown
  2. Click settings
  3. Click Develop Settings (on the left)
  4. Click Personal access tokens (on the left)

Once you’re on the Personal Access Tokens page you should see something like the following:

Click the Generate new token button, set an expiry and then copy the generated value (you’ll need it in the next step).

Step #2: Updating your keychain

Now that you’ve got your Personal Access Token you need to replace the password that you’ve currently got stored in your keychain. To start, open search and bring up Keychain Access:

If you’ve got quite a few keys there you can filter them by searching for github. You’ll then need to double click on each of the entries and replace the stored password with your personal access token:

Note that you’ll first need to click Show Password.

Now that your keychain is updated, close and then re-open any of your terminals and you should be good to go.

admin@Admins-iMac ui % git push
Enumerating objects: 110, done.
Counting objects: 100% (110/110), done.
Delta compression using up to 4 threads
Compressing objects: 100% (91/91), done.
Writing objects: 100% (93/93), 15.30 KiB | 2.19 MiB/s, done.
Total 93 (delta 64), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (64/64), completed with 14 local objects.
To https://github.com/Buzzology/referrer.git
   0d2ecf0..97f2716  master -> master
admin@Admins-iMac ui %

Thanks to the following Stackoverflow post for the additional info: https://stackoverflow.com/a/67650257/522859

Custom Error Message for PriceInCents – Vue3 and Vuelidate

Leave a reply

Hey everyone,

This is a quick post to show how you can add a custom error message when using Vuelidate in Vue3. In my case I have a price field that should not be greater than $1000. The issue is that I store the amount in cents which results in the following error:

This is obviously a little ambiguous to the user who isn’t aware that everything is running in cents behind the scenes. In order to avoid the issue I used a helper in @vuelidate/validates:

// Import the following in your component
import {
  required,  
  maxValue,
  helpers, // Helpers provides access to the message customisation
} from "@vuelidate/validators";

...

// Add the following validation rule
priceInCents: {
          required,
          maxValue: helpers.withMessage("Price cannot be more than $1000.00" , maxValue(100000)),
        },

With the new rule in place the error is much more useful:

While it doesn’t seem to show up on Google too well this functionality is documented by Vuelidate on the following page: https://vuelidate-next.netlify.app/custom_validators.html#custom-error-messages

Create a pre-signed upload url for AWS S3 using Golang

Leave a reply

Hi everyone,

This is just a quick post on how to create a pre-signed upload url for AWS S3 using Golang.

The generate the presigned url, you’ll need something like the following:

package main

import (
	"fmt"
	"github.com/joho/godotenv"
	log "github.com/sirupsen/logrus"
	"os"
	"time"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3"
)

func main() {

	// Load env vars
	err := godotenv.Load(".env")
	if err != nil {
		log.Fatalf("Error loading .env file: %v", err)
	}

	// Load the bucket name
	s3Bucket := os.Getenv("S3_BUCKET")
	if s3Bucket == "" {
		log.Fatal("an s3 bucket was unable to be loaded from env vars")
	}

	// Prepare the S3 request so a signature can be generated
	svc := s3.New(session.New())
	r, _ := svc.PutObjectRequest(&s3.PutObjectInput{
		Bucket: aws.String(s3Bucket),
		Key:    aws.String("test-file.jpg"),
	})

	// Create the pre-signed url with an expiry
	url, err := r.Presign(15 * time.Minute)
	if err != nil {
		fmt.Println("Failed to generate a pre-signed url: ", err)
		return
	}

	// Display the pre-signed url
	fmt.Println("Pre-signed URL", url)
}

Note that we’re using godotenv to load AWS environment variables containing a few AWS keys. You can get godotenv by running the following:

go get github.com/joho/godotenv

I then have a file called “.env” sitting in the root of my working directory:

S3_BUCKET=<YOUR_BUCKET_NAME>
AWS_ACCESS_KEY_ID=<YOUR_AWS_ACCESS_KEY_ID>
AWS_SECRET_ACCESS_KEY=<YOUR_AWS_SECRET_ACCESS_KEY>
AWS_REGION=<YOUR_AWS_REGION>

Once you’ve got all of that setup you can run the script, it should output a link in your console window similar to the following:

Pre-signed URL https://YOUR_BUCKET.s3.YOUR_REGION.amazonaws.com/test-file.jpg?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=CREDS&X-Amz-Date=20210717T073809Z&X-Amz-Expires=900&X-Amz-SignedHeaders=host&X-Amz-Signature=GENERATED_SIG

To test the url you can use something like Postman with the following configuration:

Simply pasting the url into the path should populate the headers for you. As for the body, select “binary” and browse for an image. When you’re ready, click “Send”.

You should get a 200 OK response and now be able to see your uploaded image in your destination bucket. Unless you’ve changed the key it should be under the name “test-file.jpg”.

One of the main advantages of using a pre-signed url is that it allows you to upload images directly to AWS and bypass your backend server completely. You can also use it to sign image retrievals. This allows you to give the links a limited life-span – great for preventing hot-linking.

Thanks to the following GitHub post for pointing me in the right direction: https://github.com/aws/aws-sdk-go/issues/467#issuecomment-171468806

Viewing SQL Logs in MySql

Leave a reply

Hi everyone,

I’ve recently switched from PostgreSQL and MSSQL to MySQL. I ran into a bit of an issue today where I needed to see the queries I was generating for an insert statement. For MSSQL I’d normally use SQL profiler.

After a bit of Googling I came across the following solution for MySQL:

-- Enable the logging
SET GLOBAL log_output = 'TABLE';
SET GLOBAL general_log = 'ON';

-- View the results
SELECT *
FROM mysql.general_log

Running this in Sequel Pro displays an output similar to the following:

Sequel Pro screenshot of logging output for MySQL

Thanks to the following StackOverflow post for the info: https://stackoverflow.com/a/678310/522859

Azure CDN not updating after deployment – Azure

Leave a reply

Hey everyone,

I’ve setup a deployment pipeline for the JellyWidgets.com react app that I’m currently messing around on. Unfortunately, while the deployment appeared to be successful it the CDN wasn’t updating.

This turned out to be a fairly simply fix – I needed to purge the cache after each deployment. This can be done manually using the Azure portal:

Simply click the purge button on your CDN’s profile page. For a more permanent fix you can also setup a pipeline step like the following:

      - task: AzureCLI@2
        displayName: 'Purge the CDN.'
        inputs:
          azureSubscription: $(azureSubscription)
          scriptType: 'pscore'
          scriptLocation: 'inlineScript'
          inlineScript: 'az cdn endpoint purge --resource-group widgets-prod --name $(resourceGroup) --profile-name JellyWidgets --content-paths "/*" --no-wait'
          workingDirectory: '$(Build.SourcesDirectory)/UI/infrastructure'
          failOnStandardError: true

Note that if you don’t add the “no-wait” step it can take a long time for the purge to complete.

See the following urls for more info:

AKS Template for Azure Bicep

Leave a reply

Hi everyone,

I’ve added AKS to my Azure environment today using Bicep. This is the Bicep template I ended up using in case it’s useful:

@description('The name of the Managed Cluster resource.')
param clusterName string

@description('The location of the Managed Cluster resource.')
param location string = resourceGroup().location

@description('Optional DNS prefix to use with hosted Kubernetes API server FQDN.')
param dnsPrefix string

@minValue(0)
@maxValue(1023)
@description('Disk size (in GB) to provision for each of the agent pool nodes. This value ranges from 0 to 1023. Specifying 0 will apply the default disk size for that agentVMSize.')
param osDiskSizeGB int = 0

@minValue(1)
@maxValue(50)
@description('The number of nodes for the cluster.')
param agentCount int = 1

@description('The size of the Virtual Machine. Currently the cheapest is Standard_B2s: https://azureprice.net/?currency=AUD®ion=eastus2')
param agentVMSize string = 'Standard_B2s'

@description('User name for the Linux Virtual Machines.')
param linuxAdminUsername string

@description('Configure all linux machines with the SSH RSA public key string. Your key should include three parts, for example \'ssh-rsa AAAAB...snip...UcyupgH azureuser@linuxvm\'')
param sshRSAPublicKey string

@allowed([
  'Linux'
])
@description('The type of operating system.')
param osType string = 'Linux'

resource clusterName_resource 'Microsoft.ContainerService/managedClusters@2020-03-01' = {
  name: clusterName
  location: location
  properties: {
    dnsPrefix: dnsPrefix
    agentPoolProfiles: [
      {
        name: 'agentpool'
        osDiskSizeGB: osDiskSizeGB
        count: agentCount
        vmSize: agentVMSize
        osType: osType
        type: 'VirtualMachineScaleSets'
        mode: 'System' // https://github.com/Azure/AKS/issues/1568#issuecomment-619642862
      }
    ]
    linuxProfile: {
      adminUsername: linuxAdminUsername
      ssh: {
        publicKeys: [
          {
            keyData: sshRSAPublicKey
          }
        ]
      }
    }
  }
  identity: {
    type: 'SystemAssigned'
  }
}

output controlPlaneFQDN string = reference(clusterName).fqdn

See this link for more info on what each of the parameters do: https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-rm-template

One thing to note is that reversing this template didn’t immediately work. I had to add the node pool with a mode of “system”. See the following github thread for more info: https://github.com/Azure/AKS/issues/1568#issuecomment-619642862

Container Registry Template for Azure Bicep

Leave a reply

Hi everyone,

Just another Bicep template I’m using – this one is for a basic container registry:

param namePrefix string
param location string = resourceGroup().location
param tier string = 'Basic'

var registryName = '${namePrefix}${uniqueString(resourceGroup().id)}'

resource container_registry 'Microsoft.ContainerRegistry/registries@2020-11-01-preview' = {
  name: registryName
  location: location
  sku: {
    name: tier
  }
  properties: {
    adminUserEnabled: false
    policies: {
      quarantinePolicy: {
        status: 'disabled'
      }
      trustPolicy: {
        type: 'Notary'
        status: 'disabled'
      }
      retentionPolicy: {
        days: 7
        status: 'disabled'
      }
    }
    encryption: {
      status: 'disabled'
    }
    dataEndpointEnabled: false
    publicNetworkAccess: 'Enabled'
    networkRuleBypassOptions: 'AzureServices'
    zoneRedundancy: 'Disabled'
    anonymousPullEnabled: false
  }
}

output id string = container_registry.id

See the following definition if you need more info on some of the properties: https://docs.microsoft.com/en-us/azure/templates/microsoft.containerregistry/2019-12-01-preview/registries?tabs=json

Key Vault Template for Azure Bicep

Leave a reply

Hi everyone,

I’m currently mucking around with Bicep as a replacement for plain ARM templates. I’ve just created a key vault and thought I’d share it in case anyone else finds it useful:

param namePrefix string
param location string = resourceGroup().location
param tenantId string

var name = '${namePrefix}${uniqueString(resourceGroup().id)}'

resource key_vault 'Microsoft.KeyVault/vaults@2019-09-01' = {
  name: name
  location: location
  properties: {
    sku: {
      family: 'A'
      name: 'standard'
    }    
    tenantId: tenantId
  }
}

resource my_test_secret 'Microsoft.KeyVault/vaults/secrets@2019-09-01' = {
  name: '${key_vault.name}/my-test-secret'
  properties: {
    attributes: {
      enabled: true
    }
  }
}

If you want to add any other properties checkout the arm template definition: https://docs.microsoft.com/en-us/azure/templates/microsoft.keyvault/2019-09-01/vaults?tabs=json

Setup MongoDB on an Azure VM

Leave a reply

Hi everyone,

I’m currently using a small linux vm to host a MongoDB instance on Azure. These are the steps I followed:

## MongoDB

### Install
https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

1. Import the public key: `wget -qO - https://www.mongodb.org/static/pgp/server-4.4.asc | sudo apt-key add -`
2. Create a list file:  `echo "deb [ arch=amd64,arm64 ] https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-4.4.list`
   - This is for Ubuntu 18.04, check version using `lsb_release -a` 
3. Reload the local package database: `sudo apt-get update`
4. Install MongoDb: `sudo apt-get install -y mongodb-org`
5. Pin the version to prevent automatic upgrades 
```
echo "mongodb-org hold" | sudo dpkg --set-selections
echo "mongodb-org-server hold" | sudo dpkg --set-selections
echo "mongodb-org-shell hold" | sudo dpkg --set-selections
echo "mongodb-org-mongos hold" | sudo dpkg --set-selections
echo "mongodb-org-tools hold" | sudo dpkg --set-selections
```

### Allow remote access
https://www.digitalocean.com/community/tutorials/how-to-configure-remote-access-for-mongodb-on-ubuntu-20-04

1. sudo nano /etc/mongod.conf
2. Find the `network interfaces` section
3. Find the `bindIp` value
4. Append the public ip of your vm to this address list `bindIp: 0.0.0.0`
   - *Note that this is the VM ip not the ip of the machine you plan to connect from.*
5. Restart MongoDB `sudo systemctl restart mongod`

### Add authentication
1. Connect to the instance from an ssh connection: `mongo --port 27017'
2. Create the user:
https://stackoverflow.com/a/38921949/522859
```
db.createUser(
  {
    user: "YOUR_USERNAME",
    pwd: "YOUR_PASSWORD",
    roles: [ { role: "root", db: "admin" } ]
  }
)
```
3. Restart with access control: `mongod --auth --port 27017 --dbpath /data/db1`
4. Authenticate as the user adminstrator: `mongo --port 27017 -u "YOUR_USERNAME" -p "YOUR_PASSWORD"  --authenticationDatabase "admin"`
5. Locate the commented out security heading and uncomment it.
6. Add `authorized: enabled`
```
security:
  authorization: enabled
```
7. Restart with access control: `mongod --auth --port 27017 --dbpath /data/db1`

Setup PostgreSQL on an Azure VM

Leave a reply

Hi everyone,

I’m currently testing out a small linux vm with PostgreSQL on Azure. Just sharing the steps for future reference and hopefully it’ll be able to help you out as well.

# Setup PostgreSQL on Azure VM

## Run updates
sudo apt-get update
sudo apt-get install postgresql

## Setup postgresql.conf
1. sudo vim /etc/postgresql/10/main/postgresql.conf
2. Add or uncomment `listen_addresses = '*'`

## Update pg_hba.conf
1. sudo nano /etc/postgresql/10/main/pg_hba.conf
2. Add the following line `host    all             all             151.XXX.XXX.XXX/32        md5` *\*where the ip is your public ip*
3. Restart pgsql `invoke-rc.d postgresql restart`

## Setup a PostgreSQL user
https://serverfault.com/a/248162/148152  

1. sudo -u postgres psql postgres
2. postgres=# \password postgres
3. Enter a new password and then confirm it.

## Done
You should now be able to login remotely (I was using pgAdmin from Windows for this).

Good luck!