Google Cloud Platform
Master GCP: Google Cloud services, data analytics, machine learning, and cloud architecture.
What is Google Cloud Platform?
Google Cloud Platform (GCP), now presented as Google Cloud, is a public cloud platform. It lets teams create compute, storage, database, networking, analytics, machine-learning, and operations resources through APIs. A team places resources in projects and locations, connects them through networks and service APIs, and pays for reserved or measured use.
GCP matters because managed services can remove work such as hardware procurement, control-plane maintenance, elastic placement, and some replication. They do not remove the need to choose ownership, identity, data, failure, capacity, and recovery contracts.
The core invariant is: Google operates the cloud platform; the workload team owns the application's user-visible outcome. A green resource status is not proof that requests are correct, authorized, affordable, or recoverable.
Start with the resource hierarchy and owner
The resource hierarchy is the management structure that answers where policy is inherited, who can change a workload, and which project carries its APIs, quotas, and billing. It is separate from the runtime request path.
Google Cloud governance flows from organization to service resource
Attach a control at the highest scope where it is true for every descendant. Keep exceptions narrow, owned, and reviewable.
Enterprise root
Organization
Represents the company. Organization Policy, IAM, security evidence, and administrative controls can apply across every descendant.
Delegation boundary
Folders
Group projects by environment, business unit, application family, or regulatory scope. Use them to delegate ownership without granting organization-wide access.
Workload boundary
Projects
Contain service resources and their enabled APIs, IAM bindings, quotas, labels, and billing association. Separate production when it needs an independent blast radius.
Runtime capability
Service resources
VMs, Cloud Run services, GKE clusters, buckets, topics, datasets, and databases carry their own location, access, capacity, lifecycle, and recovery behavior.
Make the hierarchy operational
- Give every project an accountable team, environment, data classification, cost center, and deletion policy.
- Separate shared networking and security projects only when the platform team can publish availability, capacity, change, and incident contracts for them.
- Apply broad policy in audit mode first; provide a supported remediation path before denying deployments.
- Track application dependencies across projects. Resource hierarchy alone does not show the complete service graph.
Review the current Google Cloud resource hierarchy before designing inheritance or project boundaries.
Trace a production request through the data plane
A public request crosses global and regional infrastructure, but management actions use a separate control plane. During an incident, ask whether users can complete the journey before asking whether a deployment API is healthy.
A governed request through Google Cloud
Carry one request identity, deadline, and trace context across the path. Put retries at the layer that understands whether the operation is safe to repeat.
Global edge
Cloud Load Balancing and Cloud CDN
Terminates public traffic, applies edge policy, serves eligible cached content, and selects a healthy backend. Health checks must represent the journey they protect.
Policy boundary
Cloud Armor and admission
Filters abusive traffic and enforces protocol, identity, quota, and request-size rules before expensive application work begins.
Regional compute
Cloud Run, GKE, or Compute Engine
Executes bounded business logic with a workload identity, explicit timeouts, controlled concurrency, and autoscaling that respects downstream capacity.
Authoritative state
Managed data and messaging
Cloud SQL, Spanner, Firestore, Bigtable, Cloud Storage, and Pub/Sub provide different transaction, consistency, location, replay, and recovery contracts.
Feedback plane
Cloud Monitoring and audit evidence
Metrics, logs, traces, audit events, deployment versions, and business outcomes reveal user impact and support a specific operator action.
The control plane creates and changes resources. The data plane serves requests and data. An existing workload can sometimes keep serving during a control-plane issue, while a successful deployment can still create a broken data plane.
Choose compute by the control the workload needs
Prefer the highest-level runtime that still satisfies the workload's isolation, networking, startup, scaling, portability, and operational requirements.
Serverless containers
Cloud Run
Fits stateless HTTP services, event handlers, and jobs that can use the platform's request, concurrency, startup, scaling, and execution model. Bound retries and side effects because instances can start and stop dynamically.
Managed Kubernetes
GKE
Fits teams that need Kubernetes APIs, scheduling policy, sidecars, operators, or multi-workload orchestration and can own workload, upgrade, network, and capacity policy.
Virtual machines
Compute Engine
Fits guest operating-system control, legacy software, specialized drivers, or custom machine layouts. The team owns images, guest patching, process health, and fleet rollout.
Queued execution
Batch and managed jobs
Fit bounded asynchronous work that can declare resources, checkpoint progress, and retry safely without keeping an always-on request-serving fleet.
Do not select a runtime from the happy path alone. Test cold starts, rollout overlap, connection limits, quota, accelerator availability, autoscaler delay, private egress, and the loss of one declared placement slice. The current compute selection guide documents the platform's supported options.
Turn demand into a capacity and cost envelope
Cloud cost follows architecture. Peak requests drive serving capacity, average requests drive monthly volume, cacheability changes origin work, response bytes drive network delivery, and telemetry policy drives ingestion and retention.
Peak x origin share
Origin demand
Cache hits reduce origin work, not user demand
Demand / tested unit
Capacity floor
Apply target utilization and deployment reserve
Requests x bytes
Variable volume
Separate edge, origin, logs, and retained data
Total / business unit
Unit economics
Track cost per request, tenant, job, or model result
The lab uses transparent teaching assumptions, not current Google Cloud prices. Replace its rates with measured telemetry, current regional pricing, discounts, taxes, support, and negotiated terms before making a budget or purchase decision.
Loading the workload model
The lesson is loading workload profiles and transparent planning assumptions.
Loading planning assumptions...
Read the model in the right order
- Measure peak and average demand separately; a monthly average cannot size a burst.
- Benchmark sustainable work per capacity unit at the target tail latency and error rate, including real dependencies.
- Reserve capacity for rollout, warm-up, retry bursts, and a failed unit before buying a commitment.
- Validate regional quota for both normal deployment and recovery, including CPUs, addresses, load balancers, accelerators, database connections, and API limits.
- Reconcile actual spend by project, workload, environment, owner, and business unit.
Design for failure domains, not product labels
A failure domain is a resource or group of resources that can fail together. A zone is an infrastructure failure domain inside a region. A second region is a separate recovery design, not a checkbox: it needs routing, serving capacity, data authority, dependencies, IAM, quota, and a tested promotion and failback procedure.
Keep four promises separate
- Availability: can a healthy target accept a new request now?
- Durability: which acknowledged writes remain recoverable after the incident?
- Recovery point objective (RPO): how much recent state may be lost?
- Recovery time objective (RTO): how long until routing, capacity, state, and dependencies restore the required journey?
A backup protects historical data; it is not live serving capacity. A replica can improve availability; it can also copy a valid deletion or corruption. Use isolated retention, restore testing, and application-level validation alongside replication. Google's reliability building blocks describe regions and zones as design boundaries.
Select data services from the contract
Start with the authoritative data model, transaction boundary, access path, consistency, latency, scale, location, retention, replay, RPO, and RTO. Then select a service.
Objects and artifacts
Cloud Storage
Use for objects addressed by key: media, exports, logs, model artifacts, and backups. Choose location, lifecycle, retention, versioning, access, and recovery intentionally.
Managed relational
Cloud SQL
Use when MySQL, PostgreSQL, or SQL Server semantics fit. Model connection limits, maintenance, high availability, read replicas, backups, and regional recovery.
Scalable relational
Spanner
Use when relational transactions and horizontal scale justify its schema, key, placement, consistency, and operational model. Do not choose it only for the word global.
Non-relational access
Firestore or Bigtable
Choose document or wide-column semantics from query shape, key distribution, transaction needs, hotspot risk, consistency, and retention rather than avoiding SQL.
Asynchronous delivery
Pub/Sub
Decouple producers and consumers with explicit acknowledgement, retry, ordering, dead-letter, replay, idempotency, and backlog-age policies.
Analytics and ML
BigQuery and Vertex AI
Build governed analytical and model workflows with partitioned data, lineage, quality checks, evaluation, versioned artifacts, endpoint objectives, monitoring, and rollback.
For asynchronous work, acknowledge only after the durable side effect commits. Redelivery must not apply that side effect twice.
Apply the platform to real workload shapes
Low-latency journey
Public transactional API
Use edge policy and caching, regional compute, an authoritative relational store, and Pub/Sub for non-blocking follow-up. Track tail latency, error rate, saturation, payment or order correctness, and retry amplification.
Event-time pipeline
Streaming analytics
Use Pub/Sub for ingestion, Dataflow for bounded or streaming transforms, Cloud Storage for durable landing, and BigQuery for governed analysis. Track backlog age, late data, duplicate handling, schema changes, and replay cost.
Model lifecycle
Production ML service
Use governed data in Cloud Storage or BigQuery, Vertex AI for training and registry workflows, and a versioned serving path. Gate releases on quality, safety, latency, cost, drift, fallback behavior, and rollback evidence.
In every shape, keep the system of record explicit. Caches, search indexes, analytical tables, features, and model outputs are derived state unless the business contract says otherwise.
Secure and operate the workload as one system
1 Authenticate
Issue short-lived identity
Federate people and external workloads. Attach a dedicated identity to each runtime and avoid downloadable service-account keys where a managed identity path exists.
2 Control
Authorize the exact path
Grant narrow roles on named resources, constrain network ingress and egress, protect secrets and keys, and review inherited IAM and Organization Policy together.
3 Observe
Measure user outcomes
Correlate service-level indicators, dependency saturation, quota, deployment version, audit events, and cost with one request or business operation.
4 Respond
Rehearse recovery
Inject instance, zone, dependency, credential, data, and regional failures; verify routing, capacity, authority, restore, reconciliation, and safe failback.
Production review checklist
- Name the project, region, zones, data locations, network paths, and accountable owner.
- Use workload identity and short-lived credentials; alert on service-account key use and privileged policy changes.
- Define SLOs from user journeys, then alert on actionable error-budget consumption and dependency pressure rather than resource health alone.
- Set budgets and anomaly alerts, export detailed billing data, label resources, and review unallocated spend.
- Treat quotas as capacity limits. Monitor consumption and approve increases before launches or recovery exercises.
- Provision through reviewed infrastructure code, detect drift, use progressive rollout, and retain a tested rollback path.
- Record data classification, encryption, retention, deletion, backup, restore, RPO, RTO, and residency decisions for every authoritative dataset.
See the current workload identity guidance when removing long-lived keys.
A credible GCP design states who owns it, where every request and data copy travels, what survives each failure, how identity is issued, which quota constrains recovery, how cost scales, and what evidence permits rollback or failover.