Google Cloud Platform
******************
Google Cloud Platform
The Google Cloud Platform (GCP) is a portfolio of cloud computing services and solutions, orignally based around the initial Google App Engine framework for hosting web applications from Google’s data centers. (The Google App Engine was originally launched in 2008). GCP is now widely regarded as one of the top three premier cloud computing platforms available. However, it still trails Amazon Web Services (AWS) and Microsoft Azure in market share. GCP’s pricing models are very different from those of AWS or Azure.
Following the introduction of Google App Engine, Google later released a variety of complementary tools, including a data storage layer, and Google Compute Engine, which is Infrastructure as a Service (IaaS), and supports the use of virtual machines. Once establishing itself as an IaaS provider, Google added additional products including;
- a load balancer,
- DNS, monitoring tools, and
- data analysis services
This brought GCP closer to functional parity with AWS and Azure, making them much more competitive in the cloud market.
Even though it has drawn closer to the functionality offered by AWS, GCP is no ‘cookie-cutter’ version of AWS. GCP apparently seeks to differentiate itself, through a hybrid cloud and multi-cloud strategy. The critical functionality to make this happen, is based on a new service offering called ‘Anthos‘.
Google Anthos Service Mesh
Google Anthos is a collection of services and tools, which provides a consistent platform for all application deployments, both legacy in addition to cloud native. Cloud in addition to on-premises. Third party cloud, or existing on-premises solutions can be fully utilized, without requiring administrators and developers to learn different environments and APIs.
Anthos is built on open source technologies like Kubernetes. This makes GCP a flexible and open cloud solution for companies with hybrid or multi-cloud ambitions.
Cloud Run for Anthos
Anthos “Serverless” solutions are accomplished by utilizing Cloud Run for Anthos. Cloud Run brings the best of both serverless and containers together. It allows developers to write code in any language they choose, using any binary, without having to worry about managing the underlying infrastructure.
Anthos, along with Cloud Run for Anthos, allows the underlying infrastructure to be completely ‘abstracted’, and not serve as a design issue, for developers and architects.
Anthos, Cloud Run for Anthos, and related Kubernetes Clusters represent a family of tools and technologies that are critical in enabling the Google Cloud Platform to deliver on a hybrid cloud & multi-cloud strategy. The development philosophy here is ‘develop once, and deploy anywhere‘.
Google Cloud Compute Services
Google Cloud Compute Services consists of four components;
- Cloud Functions
- App Engine
- Kubernetes Engine
- Compute Engine
Each of these abstracts a different part of the solutions architecture, as follows;
- Cloud Functions abstracts the application layer, and provides a control surface for service invocations
- App Engine abstracts the infrastructure, and provides a control surface at the application layer
- Kubernetes Engine abstracts the Virtual Machines (VM’s), and provides a control surface for managing Kubernetes cluster and related hosted containers
- Compute Engine abstracts the underlying hardware and provides a control surface for infrastructure components
Google Cloud
Architectural Framework & Solution Scenarios
We will now look at possible architectural options and solution scenarios, for those deploying the Google Cloud Platform to deliver on organizational IT functional requirements. We will review four ‘families’ of architectural frameworks/solution scenarios, including;
- Infrastucture Updating
- Data Control
- Application Development
- Analytics Development
Options groups assigned to each of these solutions scenarios may be found next;
infrastructure updating
- Hybrid
- Networking
- Migrations
- Security & Compliance
- Windows
DATA CONTROL
- Databases
- Storage
- SAP
- Energy
- Healthcare
- Media
APPLICATION DEVELOPMENT
- Hybrid
- Networking
- Migrations
- Security & Compliance
- Windows
Analytics development
- Big Data
- Data Warehouse
- AI & ML
- Retail & eCommerce
- Financial Services
Before we expand on each of the four families of architectural frameworks & solutions, and the twenty one options they include, we want to provide a preview of the graphics we’ll be using, next;
We’ll now perform a detailed review of these four Architectural Frameworks, the twenty-one options they include, and the one-hundred-fourteen specific solutions they cover, next;
For GCP, the Hybrid option covers the following specific solutions;
- Hybrid with On-Premise Data and Applications
- Hybrid Kubernetes & on-prem svcs w/ Istio
- Cloud bursting
- Business continuity hybrid/multi-cloud
- Environment hybrid
- Edge hybrid
- Analytics hybrid/multi-cloud
- Tiered Hybrid
Of course, the GCP Hybrid option, by definition covers solutions which link on-premises solutions with the GCP.
The table below provides a summary of the most important use case for each of these options.
Hybrid Options
Use Case
Hybrid with On-Premise Data & Apps
Hybrid Kubernetes & on-prem svcs w/ Istio
Cloud bursting
Business continuity hybrid/multi-cloud
Environment hybrid
Edge hybrid
Analytics hybrid/multi-cloud
Tiered Hybrid
*Some systems run on GCP, private link to others at onprem datacenter
*Cloud Interconnect, onprem to GCP,
w/ Istio open-source service mesh
*Private onprem datacenter for baseline loads, burst to cloud for extra capacity
*GCP runs cold, warm, hot standby systems of mission-critical apps to minimize (RPO) & (RTO).
* Production workload exists at local datacenter,
with the testing (DevOps) workload at GCP.
*Run time/business critical workloads locally, use GCP for non-critical workloads
*Use GCP for analytical workloads w/ pronounced ‘demand peaks’
*New front-end apps deployed to GCP, while backend apps stay on-prem
For GCP, the Networking option covers the following specific solutions;
- Floating IP addresses
- Using Your Own Public IP addresses
- High Availability NAT
- Latency optimized Travel Sample Architecture
Note: NAT = Network Address Translation
The table below provides a summary of the most important use case for each of these options.
Networking Options
Use Case
Floating IP addresses
Using Your Own Public IP addresses
High Availability NAT
Latency optimized Travel Sample Architecture
*Floating/virtual IP addresses, used onprem, accomodate via Routes API on GCP
*Provider independent address space ,
*Network Address Translation(NAT) delivered via GCP’s Managed Cloud NAT Service
*Serve users from closest region to location, via Google’s Global Cloud Load Balancing
For GCP, the ____ option covers the following specific solutions;
- DB2 On GCP
- HA DB2 On GCP
- Postgres to GCP
- Oracle to Cloud Spanner
- DynamoDB to Cloud Spanner
The table below provides a summary of the most important use case for each of these options.
Migrations Options
Use Case
*IBM‘s DB2 migrated to GCP
*IBM‘s high availability DB2 migrated to GCP
*Postgres SQL Db Master and Replica, to GCP’s Replica & Chained Replica
*Oracle db to CSV files to GCP’s Cloud Dataflow ETL & GCP’s Cloud Spanner
*AWS Dynamo DB migrated to GCP’s Cloud Spanner
For GCP, the option covers the following specific solutions;
- PCI
- Tokenizing Cardholder data for PCI
- Binary K8S Auth
- Multiple Network Interfaces
The table below provides a summary of the most important use case/deployment option for each of these options.
Security & Compliance Options
Use Case
PCI
Tokenizing Cardholder data for PCI
Binary K8S Auth
Multiple Network Interfaces
for Cloud Monitoring, to track PCI DSS acitivity
*Tokenization, substituting a benign placeholder value, for sensitive information
*Ensures only trusted container images are deployed on Google Kubernetes
*Create configurations, so an instance connects directly to several VPC networks
For GCP, the option covers the following specific solutions;
- Federate with AD (Active Directory)
- Federate with Azure AD
The table below provides a summary of the most important use case for each of these options.
Windows Options
Use Case
Federate with AD (Active Directory)
Federate with Azure AD
Cloud Identity
*Configure Google Cloud Identity or Workspace to use Active Directory as a third party identify provider (IdP) and authoritative source .
*Configure Google Cloud Identity or Workspace to use Azure AD as a third party identify provider (IdP) and source for identities .
For GCP, the Databases option covers the following specific solutions;
- DB2 On GCP
- HA DB2 On GCP
- Postgres to GCP
- Oracle to Cloud Spanner
- Gaming Backend Database using Cloud Spanner
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Database Options
Use Case
*IBM‘s DB2 on GCP
*IBM‘s high availability DB2 on GCP
*Postgres SQL Db Master and Replica, on GCP’s Replica & Chained Replica
*Oracle db via CSV files to GCP’s Cloud Dataflow ETL & GCP’s Cloud Spanner
*Use Google Cloud Spanner for match history, Cloud Bigtable to log events
For GCP, the Storage option covers the following specific solutions;
- DR Cold standby server
- DR Warm static site
- DR with Application Replication (production on-prem)
- Hosting Avere vFXT filer on GCP
- Hot HA across GCP and onprem
- DR – Cold from On-Premises to GCP with App
Note; RPO = Recovery Point Object
RTO = Recover Time Objective
DR = Disaster Recovery
HA = High Availability
The table below provides a summary of the most important use case for each of these options.
Storage Options
Use Case
DR Cold standby server
DR Warm static site
DR w/App Replication (prod on-prem)
Hosting Avere vFXT (AvF) filer on GCP
DR-Hot HA across GCP and onprem
DR – Cold from On-Premises to GCP with App
*Cold failover. Use GCE PD Snapshots to meet RPO. Replacement activates in different zone. Persistent disk (PD) from most recent snapshot
*Failover via configuring DNS to point to static GCP site.
*Full HA solution not required. Db replicated from on-premises to GCP.
App constructed from snapshots. Db replica resized to process prod workload
*AvF is filer & read thru cache. Single storage pool for all cloud instances.
Failover by configuring DNS to point to static GC site.
*Run HA across production & GCP environs concurrently. Weighted A records & health checks to support traffic splitting & failover. Db replicated.
*Cold failover. Db snapshots in GC. Deployment via GCP. Restore most recent bu snapshot from GC. Test & reconfigure DNS to GCP app.
For GCP, the SAP option covers the following specific solutions;
- SAP Hybris on GKE
- SAP S/4 HANA
- SAP S/4 Distributed
- HA SAP HANA
- SAP HANA Dynamic Tiering
- SAP Business One
- SAP Netweaver 3 Tier
The table below provides a summary of the most important use case for each of these options.
SAP Options
Use Case / Deployment Configuration
SAP Hybris on GKE
SAP S/4 HANA
SAP S/4 Distributed
HA SAP HANA
SAP HANA Dynamic Tiering
SAP Business One
SAP Netweaver 3 Tier
SAP Business Suite
*eCommerce, CRM, customer experience software
*Intelligent ERP, cloud & onprem
*Every instance can run on a separate host
*High availability
*Db used for managing less frequently accessed warm data
*ERP design for small & medium sized enterprises
*Supports advanced business application programming(ABAP).
Supports platform independent coding
*Bundled business apps, processes, collaboration, industry specific functionality
For GCP, the option covers the following specific solutions;
- Oil and Gas
The table below provides a summary of the most important use case and/or deployment configurations for each of these options.
Energy Options
Use Case / Deployment Configuration
Oil and Gas
*SCADA based, deploying
ML Engine, Datalab
For GCP, the Healthcare option covers the following specific solutions;
- Genomics, Secondary Analysis
- Patient Monitoring
- Variant Analysis
- Healthcare API Analytics
- Healthcare API ML
- Radiological Image Extraction
- ML on EHR via Healthcare API
ML = Machine Learning
The table below provides a summary of the most important use case and/or deployment configuration for each of these options
Healthcare Options
Use Case / Deployment Configuration
Genomics, Secondary Analysis
Patient Monitoring
Variant Analysis
Healthcare API Analytics
Healthcare API ML
Radiological Image Extraction
ML on EHR via Healthcare API
*Sequencers data to Ingest Server; metadata to Cloud SQL, raw data to GCS
Sequence to BAM files. Accessed via Jupyter notebooks, BigQuery analysis
*Patient data via mobile device to Cloud Pub/Sub, to BigTable.
Adv analytics on stored data via Prediction API or Tensor Flow. Notifications.
*Genomics API using Big Data, to FASTQ or BAM. Private or shared datasets.
Batch analysis using Cloud Dataflow, interactive via Big Query & DataLab
*Cloud Healthcare API, Pub/Sub, Storage, to Cloud Dataflow , Dataproc to
*Machine Learning, to Cloud Pub/Sub, to ML models, to Enterprise Viewer
*DICOM API, to Imaging Analytics, to BigQuery, Cloud ML, Dataproc, DataLab
*Machine learning and analytics using Cloud Healthcare API on GCP
For GCP, the Media option covers the following specific solutions;
- Transcoding
- Live Streaming
- Rendering
- Hybrid Rendering
- Virtual Studio
The table below provides a summary of the most important use case for each of these options.
Media Options
Use Case / Deployment Configuration
Transcoding
Live Streaming
Rendering
Hybrid Rendering
Virtual Studio
*Opensource OpenCue tool to manage encoding. Uses ffmpeg opensource tool to encode video. Output optimized for OTT, streaming, Android & iOS, set top boxes, social media platforms. Supports different bandwidth – LTE, 4G, Wifi.
*Wowza Streaming Engine, or Nginx RTP on GCE. Front streaming server with
a Content Delivery Network (CDN).
Generate playback formats HLS, HDS, Smooth Streaming, MPEG-DASH.
*CGE VM’s w/ NFS cloud file system
*Common VFX Rendering pipeline. Cloud Directory Sync to manage users/perms
*Shared POSIX files system via Cloud Filestore. Uses OpenCue, and opensource render mgmt system . Rendor workers as VM’s on managed instance group. Connection Broker provides 2FA & OAuth authentication.
For GCP, the AppDev option covers the following specific solutions;
- Microservices with GKE
- Microservices with App Engine
- Serverless Web Scraping with Cloud Functions
- REST and gRPC APIs with Cloud Endpoints
- Mobile Site Hosting
- Firebase and Google App Engine
The table below provides a summary of the most important use case for each of these options.
AppDev Options
Use Case / Deployment Configuration
Serverless Web Scraping w/ Cloud Functions
REST and gRPC APIs with Cloud Endpoints
Mobile Site Hosting
Firebase and Google App Engine
*Containerized microservices. Auto-scaling, auto-upgrade, auto-repair, via Google SRE’s
*App Engine Standard(PaaS). Python, Java, Go, NodeJS, PHP runtimes
*Event-driven web scraping w/ Cloud Functions, Firestore & Scheduler. Built-in support for Headless Chrome, providing sophisticated UI testing & web scraping.
*Use gRPC API for more efficient internal communication.
*Firebase for mobile or web. Firebase linked to App Engine as a backend app. Firebase linked to Compute Engine as a backend app.
*Firebase syncs across iOS, Android, Web. Processes data via App Engine.
DevOps
For GCP, the option covers the following specific solutions;
- Jenkins on k8s
- Continuous Delivery with Spinnaker
- Scale Testing with Kubernetes and Locust
- UI Testing with Kubernetes and Selenium
The table below provides a summary of the most important use case/deployment configuration for each of these options.
DevOps Options
Use Case / Deployment Configuration
Jenkins on k8s
Continuous Delivery with Spinnaker
Scale Testing w/Kubernetes & Locust
UI Testing w/ Kubernetes & Selenium
*Jenkins Namespace, Container Registry, Google Load Balancer
*Opensource continuous delivery platform Spinnaker w/ Cloud Build.
*Locust is opensource tool to quickly scaling up/down load tests of web frontend as REST APIs
*Selenium Hub allows coordination many browsers to perform the test suite for a web based product
For GCP, the option covers the following specific solutions;
- Event Driven
- Mobile Site Hosting
- Platform Services on App Engine
- Firebase and Google App Engine
- Serverless Web Scraping with Cloud Functions
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Serverless Options
Use Case / Deployment Configuration
Event Driven
Mobile Site Hosting
Platform Services on App Engine
Firebase and Google App Engine
Serverless Web Scraping w/ Cloud Functions
*Event Source, to Cloud Pub/Sub,
*Firebase for mobile or web. Firebase linked to App Engine as a backend app. Firebase linked to
Compute Engine as a backend app. (See AppDev above)
*Gaming on GCP, via RESTful HTTP endpoints.
Cloud Datastore:Memcache front end,
provides NoSQL db. GKE:Agones:OpenMatch to autoscale server resources.
*Firebase syncs across iOS, Android, Web. Processes data via App Engine.
(See AppDev above)
*Serverless, scalable, event-driven web scraping w/ Cloud Functions, Firestore & Scheduler
For GCP, the option covers the following specific solutions;
- Cloud IoT Remote Monitoring
- Cloud IoT MQTT Bridge
- Smart Home Devices
- Cloud to Edge ML
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Cloud IoT Core Options
Use Case / Deployment Configuration
IoT Remote Monitoring
IoT MQTT Bridge
Smart Home Devices
Cloud to Edge ML
*Cloud IoT Core to connect IoT devices via MQTT or HTTP bridge to GCP.
*Devices of any size may connect thru secured, bidirectional MQTT bridge.
*Smart Home actions controls IoT devices thru Google Assistant. MQTT or HTTP bridge(s) connect IoT devices to GCP using per-device public/private key auth.
*Cloud IoT Edge extends GCP data processing and machine learning to gateways, cameras, and other connected devices.
For GCP, the Big Data option covers the following specific solutions;
- Data Warehouse Modernization
- Log Processing
- Time-series Analysis
- Data Lake
- Real-Time Inventory
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Big Data Options
Use Case / Deployment Configuration
Data Warehouse Modernization
Log Processing
Time-series Analysis
Data Lake
Real-Time Inventory
via Cloud Dataflow/Dataproc /Composer
*StackDriver to Dataflow to BigQuery
*OpenTSDB time series database engine on GKE to
For GCP, the Data Warehouse option covers the following specific solutions;
- Data Lake
- Shopping Cart Analysis
- DMP
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Data Warehouse Options
Use Case / Deployment Configuration
Data Lake
Shopping Cart Analysis
DMP
*Analyze customer behavior(heuristics) via Cloud Dataproc, Dataflow , detail analtyics via BigQuery
*Data Management Platform
For GCP, the AI & ML option covers the following specific solutions;
- Recommendation Engines
- Chatbot with Dialogflow
- TensorFlow on GPU
- Low Latency ML Serving
- Feature Embeddings
- Semantic Similarity
The table below provides a summary of the most important use case/deployment configuration for each of these options.
AI & ML Options
Use Case / Deployment Configuration
Recommendation Engines
Chatbot with Dialogflow
TensorFlow on GPU
Low Latency ML Serving
Feature Embeddings
Semantic Similarity
*GCP Prediction API to train regression /classification models & generate realtime predictions..OR Spark MLlib sourced custom machine learning algorithms, deployed to Cloud Dataproc
*Dialogflow is an end-to-end, create-once, and deploy-anywhere development suite for creating conversational interfaces for websites/mobile apps/messaging platforms, & IoT devices
*TensorFlow training application on Graphics Processing Units (GPU) to accelerate training process for deep learning models.
*Speeds availability of Machine Learning output
*An embedding is a translation of a high-dimensional vector into a low-dimensional space
*Explore similar articles via embeddings comparable to SQL queries.
For GCP, the Retail & eCommerce option covers the following specific solutions;
- Fraud Detection
- Real-Time Inventory
- Beacons and Targeted Marketing
- Shopping Cart Analysis
- Recommendation Engines
- PCI
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Retail & eCommerce Options
Use Case / Deployment Configuration
Fraud Detection
Real-Time Inventory
Beacons and Targeted Marketing
Shopping Cart Analysis
Recommendation Engines
PCI
*GCP Prediction API to train regression /classification models & generate realtime predictions..OR Spark MLlib sourced custom machine learning algorithms, deployed to
*Back Office Biz Apps to App Engine &
*Beacon is a proximity notification. Uses Dataflow,
*Analyze customer behavior(heuristics) via Cloud Dataproc,
*GCP Prediction API to train regression /classification models & generate realtime predictions..
OR Spark MLlib sourced custom machine learning algorithms, deployed to
PCI = Payment Card Industry
For GCP, the Financial Services option covers the following specific solutions;
- Monte Carlo Simulations
- Time Series Analysis
- Fraud Detection
- Modernized Risk Analytics
The table below provides a summary of the most important use case/deployment configuration for each of these options.
Financial Services Options
Use Case / Deployment Configuration
Monte Carlo Simulations
Time Series Analysis
Fraud Detection
Modernized Risk Analytics
Apache Spark provide infrastructure, capacity to run Monte Carlo simulations written in Java, Python, or Scala.
DataLab
*GCP Prediction API to train regression /classification models & generate realtime predictions..OR Spark MLlib sourced custom machine learning algorithms, deployed to Cloud Dataproc
For a continued review of the Google Cloud Platform, relating to Tools & Apps, go here:
We WILL deliver the solution that you need !
As a first step, we will be delighted to answer any and all of your questions !