Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

GCP -Google Cloud Platform

Google Cloud Platform

Self intro

Registration

I strongly recommend you use your own private Google account for this and you create your own Google Cloud account and you don't use your employers account. If for no other reason, then because in that corporate account you might have certain limitations that will make it impossible to do some of the exercises.

Google provides you with some credit to experiment with their services without actually paying money, but they still require a valid credit card to create the account. If you don't want to provide that then I am afraid you won't be able to try this.

Remove services!

  • At the end of the session don't forget to stop and even to remove services!

  • Otherwise they might eat your credit.

  • Stop and even remove instances.

  • Remove public IP address registations.

  • Remove snapshots.

  • Remove images.

  • Remove instance groups.

Menu of the console

Some other insteresting entries

  • IAM - Identity Access Management, where you can allow other people (and devices) to have certain rights in your account.
  • Compute Engine - Virtual Machines (we'll start with this)
  • Kubernetes - to run Docker images
  • Storage - various types of managed databases and filesystems

Create VM Instance

  • Name (fixed)

  • Region, Zone (fixed)

  • Machine Type

  • Price

  • Boot disk / OS image (fixed)

There are only very few options that are "fixed" once you create the instance. Most of them can be changed later. Some can be changed while the instance is running, others require it to be shut down and restarted.

Look at the prices. They charge by the second, so you can turn on and off instances without using a lot of the credit you have.

VM Instance Example

  • Launch the smallest possible instance (micro)

Access VM in browser

  • SSH - Open in browser window

Install packages - http

  • Check htop and install it
  • curl http://localhost/
  • Install nginx and try curl again
sudo apt-get install htop
sudo apt-get install nginx
curl http://localhost/
  • Allow http traffic (while the instance is running)

Stop instances

  • Stop instance will take more than 1:30 minutes
  • Refresh

Exercise 1

  • Sign up to Google Cloud.
  • Create a micro instance with your favorite Linux distribution.
  • Access it using the web-based SSH client.
  • Try curl on localhost. It will fail as there is no web server.
  • Install nginx.
  • Try curl again.
  • Change the default web page. (See /etc/nginx )
  • Install your favorite server or command line tool.
  • Shut down the instance.

Google Cloud SDK - command line tools

gcloud init

List instances

gcloud compute instances list

Access instance using gcloud

gcloud compute ssh NAME

Snapshots

Extra disk

  • Compute Engine - VM Instances
  • Edit the VM where we would like to attach the disk "Additional disks"
  • "Attach existing disk"
$ sudo lsblk
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0  10G  0 disk
└─sda1   8:1    0  10G  0 part /
sdb      8:16   0  20G  0 disk

sudo mkfs.ext4 -m 0 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/sdb

Mount disk

sudo mkdir /mnt/data
sudo mount -o discard,defaults /dev/sdb /mnt/data
df -h

Add a file

sudo chmod a+w /mnt/data
touch /mnt/data/YOURNAME

sudo blkid /dev/sdb
/dev/sdb: UUID="51b624c7-81ad-4c17-849c-142908acff50" TYPE="ext4"

Edit /etc/fstab and add:

UUID=51b624c7-81ad-4c17-849c-142908acff50 /mnt/data ext4 discard,defaults,nofail 0 2

then you should be able to

sudo mount /mnt/data
sudo umount /mnt/data

Delete instance

Create instance based on a snapshot

  • Create instance
  • Boot disk - Change - Snapshots

Exercise 2

  • Install and configure Google Cloud SDK.
  • Access the instance using gcloud.
  • Create an instance using on the console or using gcloud compute instances create
  • Create a file on the instance (just to see it remained on the new instance).
  • Create a snapshot.
  • Create a new instance based on the snapshot.
  • Add a new disk to the instance.

Meta-data

  • Some are provided by Google.
  • Some can be set by us either project-wide or per instance.

Access Metadata

  • On the instance:
curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/
curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/hostname
curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/name

Metadata - key-value pairs

  • Compute Engine - Metadata - Metadata - Edit
  • Add 'organization' = 'Some value'
curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/project/attributes/organization

Metadata on the instance

  • Compute Engine - VM instances - (instance) - edit
  • Custom metadata
  • Set 'hello' : 'world'
curl -H "Metadata-Flavor: Google" http://metadata.google.internal/computeMetadata/v1/instance/attributes/hello

Metadata - ssh keys

  • If you don't have one yet, create an shh key on your computer

  • Compute Engine - Metadata - SSH Keys - Edit

  • Paste the public key there

  • Try to ssh to the instance from a regular SSH client.

alias xscp='scp -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'
alias xssh='ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no'

Labels

  • Labels allow us to organize resources.
  • Can help us allocate expenses to project/clients/etc.

Static or Fixed IP Address

Exercise 3

  • Make sure you have a server running!.
  • If you don't have it yet, create an ssh key on your local computer.
  • (Windows users can use the git-bash terminal.)
  • Add the public key to the project-wise Meta-Data
  • Try to ssh to the remote servers public IP using the command-line ssh client.
  • Add a label to the instance.
  • Record the current public IP address.
  • Shut down the instance.
  • Start it again, observe the change in public IP address.
  • Create a fixed IP address.
  • Shut down/start again.
  • Remove the fixed IP address (as that costs money when unused).

Create preemptible instance

  • Similar to spot instance in Amazon.
  • Can be tuned off any time.
  • Max life-span is 24 hours.
  • ~ 20% of the cost.

Instance Group

  • Automatic scaling.

  • Create and Image

  • Create an Instance template

  • Create an Instance group

Create Image

  • Create an instance.
  • Install htop.
  • Copy the following file.
import random
import time
import sys

if len(sys.argv) != 2:
    exit('Pass the load')

load = int(sys.argv[1])

while True:
    z = 0
    for _ in range(load):
        x = random.random()
        z += z
    time.sleep(0.000001)
  • Shut down the instance.
  • Create an image based on the disk of the instance.
  • Start an instance based on the image, verify it and then destroy it.

Create Instance template

Create Instance Group

Exercise 4

  • Create an instance with the loader script.

  • Create an image.

  • Cerate a template for the smallest instance possible.

  • Create an instance group with 1-10 nodes.

  • There should be one instance running.

  • Connect to it in two ssh sessions, one with htop, the other runs the load generator.

  • See how increasing load make the system increase the number of running instances.

  • Don't forget to destroy the instance group!

QA