Introduction to Google Cloud Platform

This page describes how to:

This page assumes that you have created an account and obtained the free credits on GCP.

Discovering the GCP console and creating a VM

Connecting to the console

To connect to the GCP console, use the following URL: https://console.cloud.google.com/

The front page of the console is a dashboard that gives several types of information about the GCP projects of the user including:

Accessing the GCP services

GCP provides all kind of services to its user. The service to instantiate and manage VMs is called Google Compute Engine (GCE).

To access the page corresponding to GCE, use the navigation menu (accessible at the top left of the console). An alternative is to directly access it through this link: https://console.cloud.google.com/compute/

Initially you should not see any VM running and the homepage should offer you to create one.

Creating a first VM in GCP using the console

We will now try to create our first VM using the console.

Creation of the VM

Some information about the creations of VMs can be found here: https://cloud.google.com/compute/docs/instances/create-start-instance

To create the VM, simply click on the create button in the homepage of GCE.

In the create form, several options can be configured, including:

For this first instance that we are going to create, we would like it to have the following specification:

Note that an estimate cost of the VM is given on the right side of the panel. This cost is updated automatically when changing the specifications.

The VM can now be created.

About regions and zones

In the previous step, we were asked to define a region and a zone for your VM. In a nutshell:

To obtain detailed information about regions and zones (including the full list of existing locations), please refer to https://cloud.google.com/compute/docs/regions-zones/.

About the price of VMs

As mentioned earlier, the price of a VM depends on the resources it consumes. The prices may also very depending on the region, as well as the reservation model (on-demand or long-term commitment or preemptible instances). All information about prices can be found here: https://cloud.google.com/compute/vm-instance-pricing

Observing the created VM

After a few seconds, the newly created VM should appear in GCE. We suggest you to check the changes that appeared on your GCP dashboard. You should be able to observe that:

Connecting to the VM

To connect to the newly created VM, we should go back to the GCE homepage. In the SSH menu at the end of the line describing your instance, select Open in browser window. After a few seconds, you obtain a new window with a terminal in your VM.

To check that the VM is running the OS version that we selected, run in this terminal:

lsb_release -a

The two following commands can also be run to check the number of CPUs and the amount of memory associated with the VM:

cat /proc/cpuinfo | grep "cpu cores"
cat /proc/meminfo | grep "MemTotal"

If cpu cores does not return the expected number of cores, it might be due to hyper-threading. In this case, look at the field siblings instead.

Deleting the VM through the console

Once you have run these commands, you can close the terminal. Note however that it does not mean that the VM has been deleted.

!! IMPORTANT !!: The VMs you create (and this comment is more generally applicable to any resource that you allocate in the cloud) is not deleted until you explicitly ask for it to be deleted. It is not even deleted when you disconnect from GCP. As long as a VM is not deleted, you continue paying for it.

To delete the instance that has been created, select the instance in the homepage of GCE and click on the DELETE button at the top of the page.

Creating VM instances using the command line interface

One of the main ways to interact with GCP is by using the command line interface (gcloud) gcloud. This tool is very useful as it allows automatizing many tasks that are often executed when working with GCP. The documentation about gcloud can be found here: https://cloud.google.com/sdk/gcloud/

In the following, we are going to see how to use gcloud to create a VM instance and to allow SSH connections on the created instance using a pre-existing key.

Note also that the documentation about all gcloud commands is accessible here: https://cloud.google.com/sdk/gcloud/reference/

Installing gcloud

Instructions to install gcloud on your system are available here.

Configuring gcloud

To obtain information about the default project used by gcloud, run:

gcloud config list

You may have created multiple projects, for instance, to manage multiple Google Cloud Education credits. To obtain the list of your projets, run:

gcloud projects list

To change the project in which you work:

gcloud config set project [PROJECT_ID]

Warning: Don’t forget to delete resources in the current project before moving to a new project, if need be.

Creating a VM instance using gcloud

Before creating the instance, we need to select the operating system (OS) image we should create the VM from. To obtain the list of available images, run:

gcloud compute images list

Assuming we would like an Ubuntu image, we can be a bit more specific using:

gcloud compute images list | grep ubuntu

or (type the following command verbatim – do not replace the string PROJECT):

gcloud compute images list --filter="PROJECT:(ubuntu-os-cloud)"

You should be able to observe than an image from the ubuntu-1804-lts family is available.

To decide which machine type we are going to use, we can also list the available types.

gcloud compute machine-types list

Here we obtain all the machine types available in all regions. To be a bit more specific, and obtain only a list of the machine types that can be interesting for us, we can run the following command (type this command verbatim – do not replace the string [CPUS]):

gcloud compute machine-types list --filter="zone:(us-east1-b)" --sort-by=[CPUS]

The f1-micro type should be available.

To decide which region you would like to have information about, you may want to select the default region associated with your project, as it is where your VMs will be created by gcloud by default. To obtain this information, use the command:

gcloud config configurations list

Finally, we can start our VM instance and select our options, including the VM name (my-second-instance):

gcloud compute instances create my-second-instance --image-project=ubuntu-os-cloud --image-family=ubuntu-1804-lts --machine-type=f1-micro

Once the instance is started, the command displays the main information about the newly created instance.

At this moment, it is time to check in the GCE dashboard that the instance has been created.

Enabling SSH to VM instances created in GCP

Among the information items returned by the create command, there is the public IP address (called EXTERNAL_IP) of the created instance. However, by default, it is not possible to connect to VM instances using SSH . To confirm this, you can run:

ssh [USERNAME]@[EXTERNAL_IP_ADDRESS]

Here, your USERNAME is your google login where . and @ are replaced with _ (for example, thomas_ropars_gmail_com).

Note that this limitation is not specific to the instance created with gcloud. It was also not possible to connect from outside the console to the instances created through the console.

To enable SSH connections to your VMs, two steps are required:

  1. Modifying the metadata associated with the project to allow SSH connections to the VMs
  2. Associating an SSH key with your account

These steps can be run through the command line tool or the web interface.

Configuring SSH through the command line

To allow SSH connections, run the following command:

gcloud compute project-info add-metadata --metadata enable-oslogin=TRUE

Before adding an SSH key with your account, you need to create one.

Once you have the ssh key pair, you can add the public key to your profile:

gcloud compute os-login ssh-keys add --key-file [PATH_TO_KEY]/[PUB_KEY_FILE] --ttl 0

An expiration deadline for the key can be given. The parameter --ttl 0 states that the key never expires. More information about this command can be found here: https://cloud.google.com/sdk/gcloud/reference/compute/os-login/ssh-keys/add

Configuring SSH through the web interface

Through the web interface, you can associate a SSH key to your project. To upload your public key, in the menu accessible at the top left, go to:

Compute Engine > SETTINGS > Metadata > SSH Keys

Deleting a VM instance

Before testing the new SSH configuration, we can delete the first instance that we have created since it was not configured properly. To delete the instance, simply run:

gcloud compute instances delete my-second-instance

Note: This command may take some time to complete.

Connecting to an instance using SSH

To verify that we can now connect to the instance that we created, we need to create a new instance using the same command as before. We can now try to connect to the new instance using:

ssh -i [PATH_TO_PRIVATE_KEY] [USERNAME]@[EXTERNAL_IP_ADDRESS]

You are now able to create VM instances on GCP and to connect to these instances using GCP. Hence, you are able to run jobs on these instances.

Note that the option -i [PATH_TO_PRIVATE_KEY] is only required is you have chosen to create a ssh key pair dedicated to connect to GCP, and that this key pair is not stored in the default $HOME\.ssh\ directory.

Obtaining the list of existing instances

To obtain the list of currently running instances for your project, you can run:

gcloud compute instances list

Reminder: Don’t forget to delete all VM instances before stopping working with GCP !!

Using Google Cloud Storage

In addition to computing resources, data should also be managed to run applications in the Cloud. Google Cloud Storage is the storage service (more precisely, object storage service) for data manipulated by Cloud applications.

In general, the life-cycle of data is independent from the life-cycle of VM instances. For example, we might want to have access to the results of a computation long after the computation has been done. We might also want to upload some data for which we am not going to launch all the analysis immediately. Last but not least, we might want some data to be accessible to users outside of our project. Hence, data are generally not stored in the VM instances.

The documentation about GCS is accessible here: https://cloud.google.com/storage/docs/concepts

Th GCS service works in the following way:

In the following part, we are going to create our first bucket, and we are going to upload a file in this bucket.

To create and manage GCS buckets and objects, another command-line tool is provided by Google: gsutil.

Creating a bucket

The documentation about the creation of buckets using gsutil is available here: https://cloud.google.com/storage/docs/gsutil/commands/mb.

Buckets need to be associated with a project for billing. Hence, we need to know the name of our project. This information can be obtained using this command:

gcloud config get-value project

When creating a bucket, one has to decide about the storage class for that bucket. A description of the storage classes is available here: https://cloud.google.com/storage/docs/storage-classes. The bucket we are going to create will be a standard one.

We also need to select a location for the bucket among the ones listed here: https://cloud.google.com/storage/docs/locations. In our case, we are going to choose the default region associated with our project.

Finally, we need to choose a globally unique name for the bucket. If you are not sure about how to choose a name for your bucket, you can have a look here: https://cloud.google.com/storage/docs/naming

Run the following command for the bucket creation:

gsutil mb -p [PROJECT_ID] -c standard -l us-east1 gs://[ORIGINAL_BUCKET_NAME]/

After the bucket has been created, we can list the buckets associated with the project:

gsutil ls -p [PROJECT_ID]

In addition to the bucket that we just created, you will see other buckets that are created by default for each project.

We can also display the content of the newly created bucket (it should be empty):

gsutil ls gs://[ORIGINAL_BUCKET_NAME]/

Note that, by default, the created bucket is accessible with write permissions only to the users that have the IAM editor role for the project. It is accessible with read permissions to the users that have the IAM viewer role for the project.

The “IAM” acronym stands for “Identity and Access Management”. To learn more about IAM roles, please check: https://cloud.google.com/iam/docs/understanding-roles

You can check the access policy for your bucket:

gsutil iam get gs://[ORIGINAL_BUCKET_NAME]/

Copying files in a bucket

To have a file to copy to the bucket, we suggest you to download the freely available version of the book of Arthur Conan Doyle named The Adventures of Sherlock Holmes. To this end, run the following command:

wget https://www.gutenberg.org/files/1661/1661-0.txt

To check the content of the file, we can count the number of lines in the obtained file.

cat 1661-0.txt | wc -l

You should obtain the value 12310.

In the following, we are going to copy this file into the bucket and then access it from the VM instance that we have created earlier.

To copy the file in the bucket, run:

gsutil cp ./1661-0.txt gs://[ORIGINAL_BUCKET_NAME]/

You can then check that the file appears in the bucket.

Accessing a bucket from a VM instance

Now, it is time to connect again to the VM instance that we created earlier using SSH. To obtain all the information about the instance we created earlier, we can run:

gcloud compute instances list

Once connected to the instance, we can check whether we can read the content of the bucket by running:

gsutil ls gs://[ORIGINAL_BUCKET_NAME]/

We can even copy the content of the file locally:

gsutil cp gs://[ORIGINAL_BUCKET_NAME]/1661-0.txt ./

Finally we can count the number of lines in the file to check that it corresponds to the initial file.

Cleaning a bucket

Once we do not need the bucket anymore, we can delete the content of the bucket and delete the bucket.

gsutil rm gs://[ORIGINAL_BUCKET_NAME]/*
gsutil rb gs://[ORIGINAL_BUCKET_NAME]/

Note that the two steps are required since non-empty buckets cannot be deleted.