Google Container Engine

Container Engine is the third and last component of the Compute layer of Google Cloud Platform that I want to explore.

What is Container Engine?

• Google Container Engine is a Google-hosted version of Kubernetes running on Compute Engine. 
• Kubernetes is an open source orchestration system for Docker containers.
• Basically, Container Engine enables you to host Docker containers with Kubernetes very easily. For more details, on why containers makes sense, see: An introduction to containers, Kubernetes, and the trajectory of modern cloud computing.
• Most useful command lines tools for Container Engine:
  • gcloud: used to push Docker images, create cluster.
  • kubectl: used to send requests to Kubernetes cluster manager (eg. to create Kubernetes pods, services, get cluster info and scale up/down pods).

Container Engine App Lifecycle

These are roughly the steps you need to follow in order to host the app in a Kubernetes backed cluster in Container Engine for the first time:

1. Create your app.
2. Create a Docker image containing your app.
3. Build Docker image and run it locally to make sure it works as expected.
4. Push the Docker image to Google Container Registry.
5. Create a Kubernetes cluster: A cluster consists of a master API server hosted by Google and a set of worker nodes which are Compute Engine VMs.
6. Create a Kubernetes pod: A pod is a group of containers for Kubernetes to manage together, from the Docker image pushed earlier.
7. Expose the pod as a Kubernetes service to make it accessible to the outside world.

If you need to update an existing app:

1. Make the needed changes to your app.
2. Build and test a new docker image locally.
3. Push the new image to Google Container Registry.
4. Do a rolling-update using kubectl command.

Resources

• Main Container Engine Docs
• Kubernetes Docs
• Stackoverflow – Google Container Engine

 

Google App Engine

Intro

The next component I want to explore in the Compute layer of Google Cloud Platform (GCP) is App Engine. As usual, App Engine Docs is the best place to learn more but I want to provide shorter notes here. Being a Java guy myself, I also want to emphasize aspects of App Engine relevant to Java.

What is App Engine?

• Platform as a Service (PaaS) offering of Google Cloud.
• Provides managed runtimes for specific versions of Java, Go, PHP, Python.
• Mainly for web and mobile apps.
• App Engine provides:
  • Automatic scaling and load balancing.
  • Access to other App Engine services such as Datastore, Blobstore, Memcache, Endpoints, Task Queues, Scheduled Tasks, Search API, Logs API and more.
• Each language comes with an App Engine SDK for local development, testing and deployment.
• There are 2 flavors of App Engine:
  1. Sandbox: Provides a sandboxed (and limited) environment to run your web app.
  2. Managed VM (beta): Built on top of Compute Engine, provides more freedom on environment.

Sandbox

In the default flavor of App Engine, your Java app needs to conform these sandbox constraints in order to run on App Engine:

• App needs to run in Java 7 and a limited subset of Java standard library.
• App cannot write to file system (use Datastore instead). 
• No arbitrary network connections.
• No slow responses: A web request must be handled within reasonable amount of time (the exact time depends on scaling mode).
• Limited threads and background threads (depends on scaling mode).

App Engine (Sandbox) and Java

• Runs your Java web application using Java 7 in a sandboxed environment.
• Uses the Java Servlet 2.5 standard.
• App Engine Java SDK for development. It also has a plugin for supporting development with Apache Maven.
• Development server for local development and testing.
• IDE support: Google Plugin for Eclipse to manage App Engine directly from Eclipse. Additionally, Google App Engine Integration for IntelliJ and NetBeans plugin for Gaelyk Framework.
• Java apps are deployed as App Engine Modules where each module can act like a microservice.
• In terms of configuration:
  • Top level app is packaged as EAR (similar to JEE).
  • Individual modules are packaged as WAR within EAR (again similar to JEE).
• Each Module can independently choose its own scaling type (how each instance is created and scaled)
  • 3 scaling types: Manual, Basic and Automatic.
• Each scaling type has its own instance classes. The instance class determines compute resources and pricing:
  • Manual: B1, B2, B4, B4_1G, or B8.
  • Basic: B1, B2, B4, B4_1G, or B8.
  • Automatic: F1, F2, F4, or F4_1G.

Managed VM

• Managed VM is in beta.
• Based on Google Compute Engine, allows one to customize runtime and OS (beyond what Sandbox provides) using Docker while providing the benefits of App Engine (automatic scaling, load balancing etc.)
• Native support for Java 8 / Servlet 3.1, Java 7 / Jetty 9, Python 2.7 and Python 3.4, Node.js, and Go. Further customization with Docker.
• Google Cloud SDK is needed to administer Managed VMs.

Resources

• Main App Engine Docs
• App Engine Features List
• App Engine (Sandbox) Docs for Java
• App Engine (Managed VM) Docs for Java 
• Google App Engine API for Java
• JRE Class White List
• App Engine Modules in Java
• App Engine Java Runtime

Compute Engine Java API

The usual way of managing Compute Engine instances is from GCP Console.  However, if you need programmatic access to Compute Engine, there is a comprehensive REST API. There are also client libraries (that basically wrap the REST API) provided by Google and the community, for Java, .NET, Go, JavaScript, Ruby and more.

In this post, I want to take a look at the Java client libraries. There are 2 client libraries for Java:

1. Apache JClouds: This is an open source library that allows one to use Java across multiple Cloud providers, including Compute Engine on Google Cloud. This is a good option if you want to work with multiple cloud vendors from the same API.
2. Google API Client Library for Java: Provided by Google, this is a Java library for accessing all Google APIs (not just Google Cloud), including Compute Engine on Google Cloud.

A good starting point to explore Google API Client Library for Java is Compute Engine Cmdline Sample. This sample basically contains a single class ComputeEngineSample that lists all compute engine instances you have in your project from Java. Nothing earth-shattering but it shows how to authenticate with Google API and interact with Compute Engine.

There’s an instructions.html file that tells you how to setup and use the sample. One thing you’ll realize is that you need to grant API access to your Java app before you can run the sample but instructions.html is a little outdated on that. This is what you need to do to grant API access:

1. Either create or go to your project from Developer Console.
2. Open side-menu on top left and go to “API Manager”.
3.  Under “API Manager”, click on “Credentials” -> “Create Credentials” -> “OAuth Client ID”.
4. Now, you need to select the application type. In our case, select “Other” and give it a name.
5. At this point, you’ll have a pop-up with client ID and secret. Copy these down as you’ll need them later.

Once that’s done, you can import the project into Eclipse (or your favorite IDE), fill client_secrets.json in resources folder with client ID and secret from step 5, and fill the following info in ComputeEngineSample:

• APPLICATION_NAME: Optional but give it a name, so you don’t get warnings.

• projectId: Your project id from Developers Console.

• zoneName: Make sure to enter the right zone for your project. In my case, I chose europe-west1-c.

That’s it. Now, you can run ComputeEngineSample from Eclipse (or your favorite IDE) and you can get info about your Compute Engine instances right from Java:

================== Listing Compute Engine Instances ==================
{
"canIpForward" : false,
"creationTimestamp" : "2016-02-16T05:58:00.714-08:00",
"description" : "",
"disks" : [ {
"autoDelete" : true,
"boot" : true,
"deviceName" : "instance-1",
"index" : 0,
"kind" : "compute#attachedDisk",
"mode" : "READ_WRITE",
    ....

 

Google Compute Engine

Intro

As I’m learning about different parts of Google Cloud Platform (GCP), I thought it’d be a good idea to share my notes. GCP documentation is great and it contains a lot of detailed information but sometimes I prefer cheat sheet style notes and that’s what I intend to provide here.

GCP is nicely divided into different sections: Compute, Storage, Networking, Big Data, Services, Management. I started looking into Compute layer and more specifically, Compute Engine. Here are my notes on Compute Engine.

What is Compute Engine?

• Infrastructure as a Service (IaaS) offering of Google Cloud.
• At its core, it’s 3 things: Virtual Machines, Disks, and Networking.
• Additionally: Scaling, monitoring, regions & zones.
• Access via web-based GCP console, Restful API & client libraries, and gcloud compute command-line tool.

Virtual Machines

• Core component of Compute Engine.
• Supports different machine types: Standard, high-CPU, high-memory, shared core (full list). You can even have custom machine types, if none of these meet your needs.
• Supports different operating systems: Ubuntu, Debian, Windows and more. (full list).

Disks

• Persistent Disk and Local SSD storage (more detail).
• Persistent Disk has two flavors: standard (HDD) and solid-state (SSD).
• Automatic encryption and redundancy against data corruption.

Networking

• Private network to your project with firewall rules. Only IPv4 support (more detail)
• Includes HTTP(S) and network load balancer (more detail).
• Custom health checks for load balancer to use (more detail).
• Advanced features like Subnetworks, Protocol Forwarding, VPN and Cloud Router.

Scaling

• Managed instance groups for identical instances.
• Unmanaged instance groups for unique instances.
• Automatic autoscaling for managed instance groups.

Monitoring

• Usage Report about life-cycle of resources.
• Activity Logs for system events and API calls.

Regions & Zones

• Regions: Central US, Eastern US, East Asia, Western Europe (more detail).
• Zones are sub-regions basically.

Management

These are different management options for Compute Engine:

• GCP Console: Browser-based Google tool that to manage Compute Engine resources.
  • GCP Console
  • GCP Console docs
• gcloud compute: command-line tool to manage Compute Engine, instead of the API.
• Compute Engine API: HTTP/JSON based REST API for pretty much everything in Compute Engine.
• Client libraries: Many community and Google supported client libraries for  Compute Engine API.

Resources

• Main Docs
• Overview Video
• Stackoverflow-Compute Engine
• More Videos and Tutorials