Solution Architecture Design - GCP Professional Cloud Architect¶
Overview¶
This document covers solution architecture design principles, patterns, and methodologies for the Google Cloud Professional Cloud Architect certification. It focuses on designing robust, scalable, and cost-effective cloud solutions that meet business and technical requirements.
The Professional Cloud Architect exam tests your ability to design, develop, and manage robust, secure, scalable, highly available, and dynamic solutions to drive business objectives. This requires deep understanding of architecture patterns, GCP services, trade-off analysis, and real-world design scenarios.
Exam Weight Distribution: - Solution Design: ~35% - Infrastructure Design: ~25% - Data Management: ~15% - Security & Compliance: ~15% - Migration & Modernization: ~10%
Google Cloud Architecture Framework¶
The GCP Architecture Framework provides five pillars for designing excellent cloud architectures:
1. Operational Excellence¶
Definition: Ability to run and monitor systems to deliver business value and continually improve processes.
Key Principles: - Infrastructure as Code (Terraform, Deployment Manager) - Automated CI/CD pipelines (Cloud Build, Cloud Deploy) - Centralized logging and monitoring (Cloud Logging, Cloud Monitoring) - Incident management and blameless postmortems - Regular performance reviews and optimization cycles
Best Practices: - Use version control for all infrastructure code - Implement comprehensive observability (logs, metrics, traces) - Automate operational tasks with Cloud Functions or Cloud Scheduler - Use Cloud Operations suite for unified monitoring - Implement SLOs, SLIs, and error budgets - Conduct regular disaster recovery drills
Exam Tip: Questions often present operational challenges requiring automation solutions. Always consider managed services and automation over manual processes.
2. Security, Privacy, and Compliance¶
Definition: Protect data, systems, and assets while delivering business value through risk management.
Key Principles: - Defense in depth with multiple security layers - Least privilege access (IAM, service accounts, workload identity) - Zero trust security model - Data encryption at rest and in transit - Continuous security monitoring and threat detection
Best Practices: - Use VPC Service Controls for data exfiltration protection - Implement Binary Authorization for container security - Use Cloud KMS for encryption key management - Enable Security Command Center for security posture management - Implement Cloud Armor for DDoS protection - Use Private Google Access and VPC peering - Audit with Cloud Audit Logs and Access Transparency
Exam Tip: Security questions often involve multiple layers. Look for answers that implement defense in depth rather than single security controls.
3. Reliability¶
Definition: Design systems that perform their intended function correctly and consistently when expected.
Key Principles: - Eliminate single points of failure - Design for failure with graceful degradation - Multi-region and multi-zone deployments - Automated health checks and self-healing - Circuit breakers and bulkhead patterns
Best Practices: - Deploy across multiple zones (99.95% SLA) or regions (99.99% SLA) - Use Global Load Balancing for automatic failover - Implement health checks at multiple levels - Use Cloud SQL HA configuration with automatic failover - Design for eventual consistency in distributed systems - Implement retry logic with exponential backoff - Use Cloud Spanner for globally consistent databases - Test failure scenarios with chaos engineering
Exam Tip: High availability questions require understanding of zonal vs regional vs multi-regional resources and their SLA implications.
4. Performance and Scalability¶
Definition: Systems efficiently use resources to meet requirements and scale to meet demand.
Key Principles: - Horizontal scaling over vertical scaling - Asynchronous processing for non-critical paths - Caching at multiple layers - Database optimization and sharding - Autoscaling based on metrics
Best Practices: - Use Cloud CDN for static content caching - Implement Memorystore (Redis/Memcached) for application caching - Use Cloud Load Balancing for traffic distribution - Enable autoscaling on GKE, GCE, and Cloud Run - Optimize database queries and use read replicas - Use Pub/Sub for asynchronous decoupling - Implement connection pooling (Cloud SQL Proxy) - Use appropriate storage classes (Standard, Nearline, Coldline, Archive)
Exam Tip: Performance questions often involve caching strategies and database optimization. Know when to use each caching layer.
5. Cost Optimization¶
Definition: Deliver business value at the lowest price point while meeting requirements.
Key Principles: - Right-sizing resources based on actual usage - Committed use discounts for predictable workloads - Preemptible/Spot VMs for fault-tolerant workloads - Automated resource scheduling - Storage lifecycle management
Best Practices: - Use Committed Use Discounts (1-year: 25%, 3-year: 52% savings) - Use Sustained Use Discounts (automatic 30% discount) - Use Preemptible VMs (up to 80% savings) for batch workloads - Implement Cloud Storage lifecycle policies - Use BigQuery flat-rate pricing for predictable costs - Enable Cloud Storage Autoclass for automatic tier optimization - Use custom machine types for precise resource allocation - Monitor costs with Cloud Billing reports and budgets - Use labels for cost allocation and chargeback
Exam Tip: Cost optimization questions often require balancing cost with reliability. Always consider the business requirements and acceptable trade-offs.
Key Topics¶
1. Architecture Design Patterns¶
- Microservices architecture with GKE and Cloud Run
- Event-driven architecture with Pub/Sub and Eventarc
- Serverless architecture with Cloud Functions and Cloud Run
- Hybrid and multi-cloud patterns with Anthos
- Data architecture patterns (data lakes, data warehouses, streaming)
2. Business Requirements Analysis¶
- Stakeholder requirement gathering and prioritization
- Business constraint identification (budget, timeline, compliance)
- Success criteria definition with measurable KPIs
- Cost-benefit analysis and TCO calculations
- Risk assessment and mitigation strategies
3. Technical Requirements Translation¶
- Performance requirements (latency, throughput, IOPS)
- Scalability needs (concurrent users, data volume growth)
- Security and compliance (PCI-DSS, HIPAA, GDPR, SOC 2)
- Integration requirements (APIs, legacy systems, third-party services)
- Operational requirements (monitoring, alerting, incident response)
4. High Availability and Disaster Recovery¶
- Multi-region design for global applications
- Failover strategies (active-active, active-passive, pilot light)
- Backup and recovery planning with Cloud Backup and DR
- RTO and RPO requirements and implementation strategies
- Business continuity planning and testing
5. Migration Planning¶
- Assessment and discovery with Cloud Discovery and Planning
- Migration strategies (6 Rs: Rehost, Replatform, Refactor, Repurchase, Retire, Retain)
- Workload prioritization and wave planning
- Phased migration approach with pilot projects
- Post-migration optimization and modernization
Architecture Patterns Deep Dive¶
Microservices Architecture¶
Characteristics: - Service decomposition by business capability - Independent deployment and scaling - Polyglot persistence and programming - Decentralized data management - API-first design - Failure isolation and resilience - Technology heterogeneity
GCP Services: - Container Orchestration: GKE (Google Kubernetes Engine) with autopilot or standard mode - Serverless Containers: Cloud Run for fully managed container execution - API Management: Apigee for full lifecycle API management, API Gateway for simple use cases - Service Mesh: Istio on GKE or Traffic Director for advanced traffic management - Asynchronous Communication: Cloud Pub/Sub for event-driven messaging - Service Discovery: Cloud Service Directory or Kubernetes native discovery - Observability: Cloud Trace, Cloud Profiler, Cloud Logging
Architecture Components:
┌──────────────────────────────────────────────────────────┐
│ Cloud Load Balancer (HTTPS) │
└───────────────────┬──────────────────────────────────────┘
│
┌───────────┴───────────┐
│ │
┌───▼────┐ ┌────▼────┐
│ Cloud │ │ Apigee │
│ CDN │ │ API Mgmt│
└───┬────┘ └────┬────┘
│ │
│ ┌─────────────────┴──────────────────┐
│ │ │
┌───▼────▼─────┐ ┌─────────────┐ ┌────▼─────┐
│ Frontend │ │ Auth │ │ Payment │
│ Service │◄───┤ Service │ │ Service │
│ (Cloud Run) │ │ (GKE) │ │ (Cloud │
└──────┬───────┘ └─────┬───────┘ │ Run) │
│ │ └────┬─────┘
│ │ │
┌──────▼──────────────────▼──────────────────▼─────┐
│ Cloud Pub/Sub (Event Bus) │
└──────┬───────────────────┬───────────────────┬───┘
│ │ │
┌──────▼───────┐ ┌─────▼────────┐ ┌─────▼──────┐
│ Order │ │ Inventory │ │ Notification│
│ Service │ │ Service │ │ Service │
│ (GKE) │ │ (GKE) │ │ (Cloud │
└──────┬───────┘ └──────┬───────┘ │ Functions)│
│ │ └─────┬──────┘
┌──────▼───────┐ ┌─────▼────────┐ │
│ Cloud SQL │ │ Firestore │ │
│ (Orders) │ │ (Inventory) │ │
└──────────────┘ └──────────────┘ │
┌───────▼───────┐
│ Firebase Cloud│
│ Messaging │
└───────────────┘
When to Use: - Complex applications requiring independent scaling of components - Large teams working on different business capabilities - Need for technology diversity (different languages, frameworks) - Frequent, independent deployments - Different scaling requirements per service - Need for failure isolation - Long-term system evolution and maintenance
When NOT to Use: - Simple applications with low complexity - Small teams without DevOps expertise - Tight latency requirements (network overhead) - Strong consistency requirements across services - Limited operational capabilities
Design Considerations: 1. Service Boundaries: Define clear boundaries based on business capabilities, not technical layers 2. Data Management: Each service owns its data; avoid shared databases 3. Communication: Use synchronous (REST/gRPC) for read operations, asynchronous (Pub/Sub) for writes 4. Resilience: Implement circuit breakers, timeouts, and retries 5. Observability: Centralized logging, distributed tracing (Cloud Trace) 6. API Versioning: Maintain backward compatibility with versioned APIs 7. Security: Implement service-to-service authentication (Workload Identity)
Exam Tip: For microservices questions, look for requirements around independent scaling, team autonomy, and polyglot development. Choose GKE for complex orchestration needs and Cloud Run for simpler serverless containers.
Event-Driven Architecture¶
Characteristics: - Asynchronous communication between components - Loose coupling between producers and consumers - Event producers publish without knowing consumers - Multiple consumers can process same events - Event streaming and processing capabilities - Real-time data processing and reactions - Scalability through parallel processing
GCP Services: - Messaging: Cloud Pub/Sub for reliable, scalable message queuing (at-least-once delivery) - Event Handling: Cloud Functions (2nd gen) for lightweight event processing - Stream Processing: Dataflow for complex stream processing with Apache Beam - Event Routing: Eventarc for event-driven architectures across 100+ event sources - Task Queuing: Cloud Tasks for explicit task scheduling and execution - Workflow Orchestration: Cloud Workflows for orchestrating multi-step processes - Event Storage: Cloud Storage for event archival, BigQuery for event analytics
Architecture Pattern:
┌────────────────────────────────────────────────────────────┐
│ Event Producers │
├──────────────┬──────────────┬──────────────┬───────────────┤
│ Web Apps │ Mobile Apps │ IoT Devices │ Third-party │
│ (Cloud Run) │ (Firebase) │ (IoT Core) │ APIs │
└──────┬───────┴──────┬───────┴──────┬───────┴──────┬────────┘
│ │ │ │
│ │ │ │
└──────────────┴──────────────┴──────────────┘
│
│ Events
▼
┌────────────────────────┐
│ Cloud Pub/Sub Topic │
│ (Decoupling Layer) │
└────────┬───────────────┘
│
┌──────────┼──────────┬────────────┐
│ │ │ │
▼ ▼ ▼ ▼
┌─────────┐ ┌────────┐ ┌────────┐ ┌─────────┐
│ Cloud │ │Dataflow│ │ Cloud │ │BigQuery │
│Functions│ │Stream │ │Storage │ │ (Event │
│(Process)│ │Process │ │(Archive)│ │Analytics)│
└────┬────┘ └───┬────┘ └────────┘ └──────────┘
│ │
▼ ▼
┌─────────┐ ┌────────┐
│Firestore│ │Cloud SQL│
└─────────┘ └────────┘
Event-Driven Patterns:
- Event Notification: Simple notification that something occurred
- Example: User registered -> Send welcome email
-
Use: Cloud Functions triggered by Pub/Sub
-
Event-Carried State Transfer: Events contain full state changes
- Example: Order placed with complete order details
-
Use: Multiple services update their local caches
-
Event Sourcing: Store all state changes as sequence of events
- Example: Bank account transactions as event log
-
Use: Dataflow to process events, Cloud Spanner for event store
-
CQRS (Command Query Responsibility Segregation): Separate read and write models
- Example: Write to Cloud SQL, replicate to Firestore for reads
- Use: Pub/Sub to synchronize between write and read models
When to Use: - Real-time data processing requirements (IoT, clickstream) - Need for loose coupling between systems - High scalability with variable workloads - Asynchronous workflows that don't need immediate response - Multiple consumers need same data - Building reactive systems - Integrating heterogeneous systems
When NOT to Use: - Need for immediate synchronous responses - Simple request-response patterns - Strong consistency requirements across operations - Limited number of events (overhead not justified)
Design Considerations: 1. Idempotency: Design event handlers to be idempotent (Pub/Sub has at-least-once delivery) 2. Ordering: Use message ordering in Pub/Sub when sequence matters 3. Dead Letter Queues: Configure for failed message handling 4. Event Schema: Define clear event schemas and versioning strategy 5. Monitoring: Track message age, undelivered messages, and processing latency 6. Replay: Design for event replay capability for recovery scenarios 7. Filtering: Use Pub/Sub filtering to reduce unnecessary processing
Pub/Sub Best Practices: - Use topics for event types, subscriptions for consumers - Enable message retention (up to 31 days) for replay - Set appropriate acknowledgment deadlines (default 10s, max 600s) - Use exponential backoff for retries - Configure dead letter topics for poisoned messages - Use push subscriptions for Cloud Functions/Cloud Run, pull for batch processing
Exam Tip: For event-driven questions, identify loose coupling requirements and asynchronous processing needs. Pub/Sub is the foundation for most event-driven architectures on GCP. Remember: Pub/Sub for messaging, Eventarc for event routing across GCP services.
Serverless Architecture¶
Characteristics: - No server management or infrastructure provisioning - Automatic scaling from zero to planetary scale - Pay-per-use pricing (no idle charges) - Event-driven execution model - Stateless functions (state externalized) - Built-in high availability - Fast deployment and iteration cycles
GCP Serverless Services: - Cloud Functions: Event-driven FaaS (Node.js, Python, Go, Java, .NET, Ruby, PHP) - 1st gen: HTTP and background functions - 2nd gen: Built on Cloud Run, more features, longer timeouts - Cloud Run: Containerized serverless (any language, any library) - Fully managed or on GKE - HTTP requests and Pub/Sub events - Up to 60 minutes execution time - App Engine: PaaS with automatic scaling - Standard: Serverless, fast scaling - Flexible: Container-based, slower scaling - Firestore: Serverless NoSQL database with real-time sync - BigQuery: Serverless data warehouse and analytics
Serverless Architecture Pattern:
┌────────────────────────────────────────────────┐
│ Users (Web/Mobile/API) │
└────────────────┬───────────────────────────────┘
│
▼
┌───────────────────────┐
│ Cloud Load Balancer │
│ + Cloud Armor │
└───────┬───────────────┘
│
┌────────┴────────┐
│ │
▼ ▼
┌─────────┐ ┌──────────┐
│ Cloud │ │ App │
│ CDN │ │ Engine │
└─────────┘ └────┬─────┘
│
┌───────────┼───────────┐
│ │ │
▼ ▼ ▼
┌────────┐ ┌────────┐ ┌────────┐
│ Cloud │ │ Cloud │ │Cloud │
│ Run │ │Function│ │Function│
│(API) │ │(Auth) │ │(Process)│
└───┬────┘ └───┬────┘ └───┬────┘
│ │ │
└───────────┼───────────┘
│
┌───────────┴───────────┬────────────┐
│ │ │
▼ ▼ ▼
┌─────────┐ ┌─────────┐ ┌─────────┐
│Firestore│ │ Cloud │ │BigQuery │
│ │ │Storage │ │ │
└─────────┘ └─────────┘ └─────────┘
Serverless Service Selection Matrix: | Requirement | Cloud Functions | Cloud Run | App Engine | |-------------|----------------|-----------|------------| | Max Request Duration | 9 min (1st gen), 60 min (2nd gen) | 60 minutes | 60 minutes | | Concurrency | 1 per instance (1st gen), 1000 (2nd gen) | Up to 1000 | Configurable | | Container Support | No (code only) | Yes (any container) | Flexible only | | Cold Start | Fast (~100ms) | Fast (~200ms) | Standard: Fast, Flexible: Slow | | Language Support | 7 languages | Any (containerized) | Standard: 6, Flexible: Any | | Custom Binary | No | Yes | Flexible only | | WebSockets | No | Yes | Yes | | gRPC | No | Yes | No | | VPC Connector | Yes | Yes | Yes | | Pricing Model | Per invocation + compute | Per request + compute | Per instance hour |
When to Use Serverless: - Variable or unpredictable workloads with spiky traffic - Rapid development and deployment requirements - Event-driven processing (file uploads, database changes, Pub/Sub messages) - Cost optimization for sporadic or low-traffic usage - Minimal operational overhead requirements - Quick prototyping and MVPs - Stateless applications - API backends for mobile/web applications
When NOT to Use Serverless: - Long-running batch jobs (use Compute Engine or Batch) - Applications requiring persistent connections - Workloads needing specific hardware (GPUs) - use GKE or GCE - Applications with very tight latency requirements (cold starts) - Stateful applications (externalize state first) - Legacy applications that can't be containerized
Serverless Best Practices: 1. Minimize Cold Starts: - Keep function code small and dependencies minimal - Use Cloud Run min instances for critical services - Implement lazy loading for heavy libraries - Consider 2nd gen Cloud Functions for better cold start performance
- Optimize Performance:
- Reuse connections (database, external APIs) across invocations
- Use global variables for initialization code
- Implement caching (Memorystore, in-memory cache)
-
Configure appropriate memory allocation
-
Cost Optimization:
- Right-size memory allocation (CPU scales with memory)
- Use Cloud Scheduler to keep warm if needed (vs. min instances)
- Set appropriate timeouts to avoid runaway costs
-
Monitor and optimize function execution time
-
Security:
- Use Workload Identity for GCP service access
- Implement least privilege IAM roles
- Use Secret Manager for sensitive data
-
Enable VPC Service Controls for data protection
-
Reliability:
- Implement idempotency for event-driven functions
- Use dead letter queues for failed processing
- Set appropriate retry policies
- Implement circuit breakers for external dependencies
Exam Tip: For serverless questions, consider the execution time limits, cold start impacts, and whether the workload is event-driven or request-driven. Cloud Functions for simple event handling, Cloud Run for containerized applications with HTTP/Pub/Sub triggers, App Engine for traditional web applications.
Hybrid and Multi-Cloud Architecture¶
Characteristics: - Workload portability across environments - Data sovereignty and compliance requirements - Risk mitigation through diversification - Vendor lock-in avoidance - Legacy system integration - Consistent operations across clouds - Unified security and policy management
GCP Services: - Anthos: Hybrid and multi-cloud application platform - Anthos GKE on-premises and multi-cloud - Anthos Config Management for policy enforcement - Anthos Service Mesh for service-to-service communication - Connectivity: Cloud VPN (up to 3 Gbps), Cloud Interconnect (10-200 Gbps) - Compute: Cloud Run for Anthos, GKE on-premises - Migration: Migrate for Anthos and GKE, Migrate for Compute Engine - Load Balancing: Traffic Director for global service mesh load balancing - Operations: Cloud Logging and Monitoring for unified observability - Security: Binary Authorization, Policy Controller, Cloud Armor
Hybrid Architecture Pattern:
┌─────────────────────────────────────────────────────┐
│ On-Premises Data Center │
│ ┌──────────────┐ ┌──────────────┐ │
│ │ Legacy │ │ Anthos │ │
│ │ Systems │◄───────┤ Clusters │ │
│ │ (DB/Apps) │ │ (GKE) │ │
│ └──────────────┘ └───────┬──────┘ │
│ │ │
└──────────────────────────────────┼──────────────────┘
│
┌──────────────┴──────────────┐
│ Cloud Interconnect or │
│ Cloud VPN (HA VPN) │
└──────────────┬──────────────┘
│
┌──────────────────────────────────┼──────────────────┐
│ Google Cloud │ │
│ ┌────────────┐ ┌────────▼──────┐ │
│ │ Anthos │◄────────┤ Anthos Hub │ │
│ │ Config │ │ (Management) │ │
│ │ Mgmt │ └────────┬──────┘ │
│ └────────────┘ │ │
│ ┌──────────────┼─────────┐ │
│ │ │ │ │
│ ┌───────▼─────┐ ┌────▼────┐ ┌─▼─────┐ │
│ │ GKE │ │ Cloud │ │Cloud │ │
│ │ Clusters │ │ Run │ │SQL │ │
│ └─────────────┘ └─────────┘ └───────┘ │
└─────────────────────────────────────────────────────┘
Multi-Cloud Connectivity Options: | Option | Bandwidth | Use Case | Cost | Latency | |--------|-----------|----------|------|---------| | VPN Gateway | Up to 3 Gbps per tunnel | Low bandwidth, test | $ | Higher | | Partner Interconnect | 50 Mbps - 10 Gbps | Moderate bandwidth | $$ | Medium | | Dedicated Interconnect | 10 Gbps or 100 Gbps | High bandwidth, production | $$$ | Lowest |
When to Use Hybrid/Multi-Cloud: - Regulatory requirements for data residency (GDPR, data sovereignty) - Existing significant on-premises investments - Enterprise multi-cloud strategy - Gradual cloud migration approach - Application dependencies on on-premises systems - Disaster recovery across providers - Avoiding vendor lock-in - Burst to cloud for peak demand
When NOT to Use: - Simple cloud-native applications - No existing on-premises infrastructure - Limited operational expertise for multi-cloud management - Cost-sensitive projects (multi-cloud adds complexity and cost)
Anthos Benefits: 1. Consistent Platform: Same Kubernetes API across all environments 2. Centralized Management: Single pane of glass for all clusters 3. Policy Enforcement: GitOps-based configuration management 4. Service Mesh: Unified traffic management and observability 5. Security: Consistent security policies across environments 6. Modernization: Migrate VMs to containers
Design Considerations: 1. Connectivity: Choose appropriate connection type based on bandwidth and latency needs 2. Data Residency: Understand regulatory requirements for data location 3. Latency: Design for network latency between on-premises and cloud 4. Security: Implement consistent security policies across environments 5. Disaster Recovery: Plan for failover between environments 6. Cost: Consider data egress charges and interconnect costs 7. Operations: Unified monitoring and logging across environments
Hybrid Cloud Best Practices: - Use Private Google Access to access GCP services without internet exposure - Implement VPC Service Controls for data exfiltration protection - Use Cloud Router for dynamic BGP routing - Configure HA VPN for redundancy (99.99% SLA) - Use Shared VPC for centralized network management - Implement consistent security policies with Anthos Config Management - Monitor connectivity with Cloud Monitoring - Use Traffic Director for global load balancing across environments
Exam Tip: For hybrid/multi-cloud questions, Anthos is almost always the answer. Understand the differences between VPN (lower cost, lower bandwidth) and Interconnect (higher cost, higher bandwidth, lower latency). HA VPN provides 99.99% SLA with redundant tunnels.
High Availability and Disaster Recovery Design¶
Availability Tiers and SLA Targets¶
GCP Service SLAs: | Resource Deployment | SLA | Downtime/Year | Use Case | |---------------------|-----|---------------|----------| | Single zone | No SLA | N/A | Dev/Test | | Multi-zone (Regional) | 99.95% | 4.4 hours | Production | | Multi-region | 99.99% | 52 minutes | Mission-critical | | Global (e.g., Cloud Load Balancing) | 99.99% | 52 minutes | Global services | | Cloud Spanner multi-region | 99.999% | 5.26 minutes | Financial systems |
RTO and RPO Strategies¶
Recovery Time Objective (RTO): Maximum acceptable downtime Recovery Point Objective (RPO): Maximum acceptable data loss
DR Strategy Comparison: | Strategy | RTO | RPO | Cost | Complexity | Use Case | |----------|-----|-----|------|------------|----------| | Backup & Restore | Hours-Days | Hours | $ | Low | Non-critical workloads | | Pilot Light | Hours | Minutes | $$ | Medium | Lower-priority production | | Warm Standby | Minutes | Seconds | $$$ | Medium | Business-critical | | Hot Standby (Active-Active) | Seconds | Near-zero | $$$$ | High | Mission-critical |
Disaster Recovery Patterns¶
1. Backup and Restore Pattern¶
RTO: 4-24 hours, RPO: 1-24 hours, Cost: Lowest
Primary Region (us-central1) Backup Region (us-east1)
┌──────────────────┐ ┌──────────────────┐
│ Production App │ │ │
│ (GKE/Cloud Run) │ │ Backup Storage │
│ │ │ (Cloud Storage │
│ ┌────────────┐ │ Scheduled │ or Snapshots) │
│ │ Cloud SQL │──┼────Backups────►│ │
│ │ (Primary) │ │ │ (Coldline/ │
│ └────────────┘ │ │ Archive) │
└──────────────────┘ └──────────────────┘
Implementation: - Automated Cloud SQL backups (daily) with point-in-time recovery - GCE persistent disk snapshots to multi-regional Cloud Storage - Cloud Storage lifecycle policies for retention management - Backup GKE cluster configurations to Cloud Storage - Test recovery procedures quarterly
GCP Services: - Cloud SQL automated backups (7-365 days retention) - Persistent disk snapshots (incremental) - Cloud Storage lifecycle management - Backup and DR service for hybrid workloads
2. Pilot Light Pattern¶
RTO: 1-4 hours, RPO: 5-30 minutes, Cost: Medium
Primary Region (us-central1) DR Region (us-east1)
┌──────────────────┐ ┌──────────────────┐
│ Production App │ │ Minimal Core │
│ (GKE Cluster) │ │ Infrastructure │
│ [ACTIVE] │ │ [STANDBY] │
│ │ │ │
│ ┌────────────┐ │ Continuous │ ┌────────────┐ │
│ │ Cloud SQL │──┼───Replication──┼─►│ Cloud SQL │ │
│ │ (Primary) │ │ │ │ (Replica) │ │
│ └────────────┘ │ │ └────────────┘ │
└──────────────────┘ └──────────────────┘
│ ▲
│ │
└───────────────┬────────────────────┘
Global Load Balancer
(Failover configured)
Implementation: - Cloud SQL with cross-region read replicas - GKE cluster configurations stored in Cloud Source Repositories - Container images in Artifact Registry (multi-regional) - DNS failover with Cloud DNS or external DNS - Infrastructure as Code (Terraform) for rapid deployment - Minimal compute resources in DR region (scaled down)
GCP Services: - Cloud SQL cross-region replicas (async replication, 5-10s lag) - Cloud Load Balancing with health checks - Cloud DNS with failover routing - GKE with node pools configured but scaled to zero
3. Warm Standby Pattern¶
RTO: 5-30 minutes, RPO: 1-5 minutes, Cost: High
Primary Region (us-central1) DR Region (us-east1)
┌──────────────────┐ ┌──────────────────┐
│ Production App │ │ Scaled-down App │
│ (100% capacity) │ │ (25% capacity) │
│ [ACTIVE] │ │ [WARM STANDBY] │
│ │ │ │
│ ┌────────────┐ │ Continuous │ ┌────────────┐ │
│ │ Cloud SQL │◄─┼───Replication──┼─►│ Cloud SQL │ │
│ │ (Primary) │ │ (Bi-direct) │ │ (Standby) │ │
│ └────────────┘ │ │ └────────────┘ │
└──────────────────┘ └──────────────────┘
│ │
└───────────────┬────────────────────┘
Global Load Balancer
(Active health checks, auto-failover)
Implementation: - Cloud SQL HA configuration in primary, read replica in DR - GKE clusters in both regions with reduced capacity in DR - Cloud Load Balancing with traffic distribution (90% primary, 10% DR) - Cloud Spanner for globally consistent database (if needed) - Continuous data replication with minimal lag - Automated failover with health checks - Active monitoring and alerting
GCP Services: - Cloud SQL with HA and cross-region replicas - Global HTTP(S) Load Balancing with backend services - Cloud Monitoring with uptime checks and alerting - Cloud Spanner for multi-region strong consistency
4. Hot Standby (Active-Active) Pattern¶
RTO: <1 minute, RPO: Near-zero, Cost: Highest
Region 1 (us-central1) Region 2 (us-east1)
┌──────────────────┐ ┌──────────────────┐
│ Production App │ │ Production App │
│ (100% capacity) │ │ (100% capacity) │
│ [ACTIVE] │ │ [ACTIVE] │
│ │ │ │
│ ┌────────────┐ │ Synchronous │ ┌────────────┐ │
│ │Cloud Spanner◄┼───Replication───┼─►│Cloud Spanner│ │
│ │ (Multi-reg) │ │ │ │ (Multi-reg) │ │
│ └────────────┘ │ │ └────────────┘ │
└──────────────────┘ └──────────────────┘
│ │
└───────────────┬────────────────────┘
Global Load Balancer
(Traffic distributed to both regions)
Implementation: - Cloud Spanner multi-region configuration for strong consistency - GKE clusters in multiple regions with full capacity - Global HTTP(S) Load Balancing distributing traffic to all regions - Firestore with multi-region replication - Cloud CDN for static content caching globally - Active-active data replication with conflict resolution - No failover required (automatic handling)
GCP Services: - Cloud Spanner multi-region (nam-eur-asia3 configurations) - Global HTTP(S) Load Balancing with proximity-based routing - Cloud CDN for global content delivery - Firestore multi-region mode
Failover Strategies¶
DNS Failover: - Use Cloud DNS health checks - Configure failover routing policies - Update DNS records automatically on failure - Consider DNS TTL (lower TTL = faster failover, more DNS queries)
Load Balancer Failover: - Global HTTP(S) Load Balancing with health checks - Automatic backend service failover - Configure appropriate health check intervals (default 5s) - Set unhealthy threshold (default 2 consecutive failures)
Database Failover: - Cloud SQL: Automatic failover with HA configuration (60s typical) - Cloud Spanner: Automatic, transparent failover - Manual promotion of read replica to primary (if needed)
Design Principles for High Availability¶
- Eliminate Single Points of Failure: Use multi-zone/multi-region deployment
- Implement Redundancy: N+1 or N+2 redundancy for critical components
- Use Managed Services: Leverage GCP managed services with built-in HA
- Design for Failure: Assume components will fail and design accordingly
- Implement Health Checks: Multiple levels (service, application, infrastructure)
- Graceful Degradation: Maintain partial functionality during outages
- Circuit Breakers: Prevent cascade failures in microservices
- Timeouts and Retries: Implement with exponential backoff
- Automated Recovery: Self-healing systems with automatic restart
- Regular Testing: Conduct DR drills and chaos engineering exercises
Scalability Design Principles¶
- Horizontal Scaling: Design for scale-out rather than scale-up
- Use managed instance groups with autoscaling
- Deploy stateless applications
-
Use load balancers for traffic distribution
-
Stateless Components: Separate state from compute
- Externalize session state (Memorystore, Firestore)
- Use shared storage (Cloud Storage, Filestore)
-
Design for ephemeral compute instances
-
Caching Layers: Implement multi-level caching
- CDN Layer: Cloud CDN for static content
- Application Cache: Memorystore (Redis/Memcached)
- Database Cache: Query result caching in BigQuery
-
Browser Cache: HTTP cache headers
-
Asynchronous Processing: Use queues for non-real-time work
- Pub/Sub for event-driven processing
- Cloud Tasks for scheduled/deferred tasks
-
Dataflow for batch/stream processing
-
Database Optimization:
- Read replicas for read-heavy workloads
- Database sharding for write-heavy workloads
- Cloud Spanner for horizontal scalability with ACID
-
Connection pooling (Cloud SQL Proxy)
-
Auto-scaling: Use managed auto-scaling services
- GKE Cluster Autoscaler and Horizontal Pod Autoscaler
- Cloud Run automatic scaling (0 to 1000s)
-
Compute Engine autoscaling based on CPU, load balancing, custom metrics
-
Load Distribution: Implement effective load balancing
- Global Load Balancing for multi-region distribution
- Regional Load Balancing for regional resources
- Internal Load Balancing for internal services
Security Design Principles¶
- Defense in Depth: Multiple layers of security controls
- Network security (VPC, firewall rules, Cloud Armor)
- Application security (IAM, service accounts, Workload Identity)
-
Data security (encryption, VPC Service Controls, DLP API)
-
Least Privilege: Minimum necessary permissions
- Use predefined roles over primitive roles
- Create custom roles for specific needs
-
Service accounts for service-to-service communication
-
Zero Trust: Never trust, always verify
- BeyondCorp Enterprise for context-aware access
- Certificate-based authentication
-
Verify every request regardless of network location
-
Encryption Everywhere: Data at rest and in transit
- Default encryption at rest for all GCP storage
- Customer-managed encryption keys (CMEK) with Cloud KMS
-
TLS/SSL for data in transit
-
Security by Design: Build security into architecture
- Security review in architecture phase
- Threat modeling exercises
-
Security Command Center for posture management
-
Segregation of Duties: Separate critical roles
- Separate development, staging, production projects
- Use organization policies for guardrails
-
Implement approval workflows for sensitive operations
-
Audit Everything: Comprehensive logging and monitoring
- Cloud Audit Logs for all API calls
- VPC Flow Logs for network traffic
- Export logs to BigQuery for analysis
- Set up alerting for anomalous activities
Cost Optimization Principles¶
- Right-Sizing: Match resources to actual needs
- Committed Use: Purchase commitments for predictable workloads
- Spot/Preemptible: Use for fault-tolerant workloads
- Lifecycle Management: Archive or delete unused data
- Serverless First: Leverage serverless for variable workloads
- Monitoring and Alerting: Track costs continuously
- Resource Scheduling: Shut down non-production during off-hours
Best Practices¶
Architecture Documentation Best Practices¶
- Architecture Diagrams: Use standard notation (C4, UML)
- Decision Records: Document key architectural decisions
- Component Descriptions: Detail each component's purpose
- Data Flow Diagrams: Show data movement through system
- Security Architecture: Document security controls
- Deployment Architecture: Show deployment topology
- Cost Estimates: Include TCO calculations
- Non-Functional Requirements: Document performance, security, etc.
Requirement Gathering Best Practices¶
- Stakeholder Interviews: Understand business needs
- Current State Analysis: Document existing architecture
- Constraint Identification: Identify technical and business constraints
- Success Metrics: Define measurable success criteria
- Prioritization: Rank requirements by importance
- Trade-off Analysis: Document design trade-offs
- Assumption Documentation: Record all assumptions
Migration Planning Best Practices¶
- Assessment Phase: Comprehensive inventory and analysis
- Migration Strategy: Choose appropriate strategy per workload
- Pilot Projects: Start with low-risk applications
- Phased Approach: Migrate in manageable phases
- Testing Strategy: Validate each migration phase
- Rollback Planning: Prepare for potential rollback
- Training and Documentation: Prepare team for new environment
- Post-Migration Optimization: Optimize after migration
Data Architecture Patterns¶
Data Lake Architecture¶
Purpose: Centralized repository for structured and unstructured data at any scale
Data Sources Ingestion Storage & Processing
┌─────────────┐ ┌──────────┐ ┌──────────────────┐
│ IoT │──────────────►│ Pub/Sub │───────────►│ Cloud Storage │
│ Devices │ └──────────┘ │ (Data Lake) │
└─────────────┘ │ - Raw Zone │
│ - Curated Zone │
┌─────────────┐ ┌──────────┐ │ - Enriched Zone│
│ Applications│──────────────►│ Transfer │───────────►└─────────┬────────┘
│ Logs │ │ Service │ │
└─────────────┘ └──────────┘ │
│
┌─────────────┐ ┌──────────┐ ┌────────▼────────┐
│ Databases │──────────────►│ Dataflow │───────────►│ BigQuery │
│ (On-prem) │ │ ETL │ │ (Data Warehouse)│
└─────────────┘ └──────────┘ └────────┬────────┘
│
┌─────────▼────────┐
│ Looker/Data │
│ Studio (BI) │
└──────────────────┘
Key Components: - Ingestion: Pub/Sub, Transfer Service, Dataflow, Datastream - Storage: Cloud Storage (multi-regional for HA, regional for cost) - Processing: Dataflow (Apache Beam), Dataproc (Hadoop/Spark) - Analytics: BigQuery, Vertex AI for ML - Governance: Data Catalog, DLP API, Cloud Data Loss Prevention
Storage Class Strategy: | Data Zone | Access Pattern | Storage Class | Use Case | |-----------|----------------|---------------|----------| | Raw | Infrequent | Nearline | Original data, compliance | | Curated | Regular | Standard | Cleaned, validated data | | Enriched | Frequent | Standard | Analytics-ready data | | Archive | Rare | Archive | Long-term retention |
Data Warehouse with BigQuery¶
Purpose: Serverless, highly scalable data warehouse for analytics
Best Practices: 1. Table Design: - Use partitioning (date, timestamp, ingestion time) for large tables - Use clustering (up to 4 columns) for frequently filtered columns - Denormalize data for better query performance - Use nested and repeated fields for hierarchical data
- Query Optimization:
- Use
_PARTITIONTIMEor partition column in WHERE clause - Avoid SELECT * (specify columns explicitly)
- Use approximate aggregation functions (APPROX_COUNT_DISTINCT)
-
Materialize frequently accessed query results
-
Cost Optimization:
- Use partitioned tables to reduce data scanned
- Set table expiration for temporary data
- Use clustering for filter and aggregate queries
- Consider flat-rate pricing for predictable costs (>$40K/month)
-
Use BI Engine for caching frequently accessed data
-
Performance:
- Use batch loading over streaming for large data volumes
- Enable result caching (24 hours free)
- Avoid self-joins (use window functions instead)
- Use INFORMATION_SCHEMA for metadata queries
Streaming Analytics Architecture¶
Real-time Sources Stream Processing Storage & Analytics
┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ IoT Devices │─────────►│ Pub/Sub │────────►│ Dataflow │
│ (millions) │ │ (Buffer) │ │ (Streaming) │
└──────────────┘ └──────────────┘ └──────┬───────┘
│
┌──────────────┐ ┌──────────────┐ ┌─────▼───────┐
│ Web/Mobile │─────────►│ Eventarc/ │────────►│ BigQuery │
│ Clickstream │ │ Functions │ │ (Streaming │
└──────────────┐ └──────────────┘ │ Insert) │
└──────┬──────┘
┌──────────────┐ │
│ App Logs │─────────────────────────────────────────►│
│ (Cloud Log) │ ┌─────▼──────┐
└──────────────┘ │ Real-time │
│ Dashboard │
│ (Looker) │
└────────────┘
Streaming Considerations: - Pub/Sub: Message buffer, at-least-once delivery, up to 10 GB/s ingestion - Dataflow: Stream processing with exactly-once semantics, windowing, state management - BigQuery Streaming: Up to 100K rows/s per table, limited DML operations - Bigtable: High-throughput streaming writes, sub-10ms reads
Data Migration Patterns¶
Database Migration Service (DMS): - Minimal downtime migration for MySQL, PostgreSQL, SQL Server - Continuous replication with CDC (Change Data Capture) - Support for homogeneous and heterogeneous migrations - Automatic schema conversion
Transfer Appliance: - Physical data transfer for petabyte-scale migrations - 100 TB or 480 TB capacity - Ship to Google data center - Use when network transfer would take >1 week
Storage Transfer Service: - Online data transfer from AWS S3, Azure Blob, HTTP/HTTPS sources - Scheduled, incremental transfers - Bandwidth throttling and filtering - Free data transfer (only pay for GCP storage)
Comprehensive Real-World Case Studies¶
Case Study 1: Global E-Commerce Platform¶
Company Profile: - Online retailer with 50M active users - $5B annual revenue - Operations in 30 countries - Black Friday traffic 20x normal load
Business Requirements: - 99.99% availability (52 min downtime/year) - Page load time <2 seconds globally - Handle 100K concurrent checkouts - PCI DSS Level 1 compliance - Real-time inventory across 500 warehouses - Support for 10 million SKUs - Personalized product recommendations - Multi-currency and multi-language support
Technical Constraints: - Budget: $500K/month infrastructure - Must integrate with existing SAP ERP - Data residency requirements (EU GDPR) - Existing Oracle database with 10TB data - Team of 50 developers across 5 time zones
Non-Functional Requirements: - RTO: 15 minutes - RPO: 5 minutes - Peak load: 50K requests/second - Data retention: 7 years for compliance - Disaster recovery in different region
Architecture Solution:
┌─────────────────────────┐
│ Cloud CDN (Global) │
│ + Cloud Armor (DDoS) │
└────────────┬────────────┘
│
┌────────────▼────────────┐
│ Global HTTP(S) Load │
│ Balancer (Multi-region) │
└─────┬──────────────┬────┘
│ │
┌──────────────────┴────┐ ┌────┴──────────────────┐
│ Region: us-central1 │ │ Region: europe-west1 │
│ ┌─────────────────┐ │ │ ┌─────────────────┐ │
│ │ GKE Autopilot │ │ │ │ GKE Autopilot │ │
│ │ (Microservices)│ │ │ │ (Microservices)│ │
│ ├─────────────────┤ │ │ ├─────────────────┤ │
│ │ - Product API │ │ │ │ - Product API │ │
│ │ - Cart Service │ │ │ │ - Cart Service │ │
│ │ - Checkout API │ │ │ │ - Checkout API │ │
│ │ - User Service │ │ │ │ - User Service │ │
│ └────────┬────────┘ │ │ └────────┬────────┘ │
│ │ │ │ │ │
│ ┌────────▼────────┐ │ │ ┌────────▼────────┐│
│ │ Cloud Spanner │◄─┼───┼─►│ Cloud Spanner ││
│ │ (Multi-region) │ │ │ │ (Multi-region) ││
│ │ - Users │ │ │ │ - Products ││
│ │ - Orders │ │ │ │ - Inventory ││
│ └─────────────────┘ │ │ └─────────────────┘│
│ │ │ │
│ ┌─────────────────┐ │ │ ┌─────────────────┐│
│ │ Memorystore │ │ │ │ Memorystore ││
│ │ Redis (Session) │ │ │ │ Redis (Session) ││
│ └─────────────────┘ │ │ └─────────────────┘│
└───────────────────────┘ └──────────────────────┘
│ │
┌──────────┴──────────────┴──────────┐
│ Cloud Pub/Sub (Global) │
│ - Order Events │
│ - Inventory Updates │
└────────┬──────────────┬────────────┘
│ │
┌──────────▼─────┐ ┌────▼──────────┐
│ Cloud │ │ BigQuery │
│ Functions │ │ (Analytics) │
│ (Processing) │ │ │
└────────────────┘ └───────────────┘
│
┌───────▼────────┐
│ Vertex AI │
│ (Recommender) │
└────────────────┘
Service Selection Rationale:
| Requirement | Service Choice | Rationale | Alternative Considered |
|---|---|---|---|
| Database | Cloud Spanner | Global consistency, horizontal scalability, 99.999% SLA | Cloud SQL (lacks global scale) |
| Compute | GKE Autopilot | Managed Kubernetes, autoscaling, security | Cloud Run (stateful services need K8s) |
| Session Store | Memorystore Redis | Sub-millisecond latency, session affinity | Firestore (higher latency) |
| CDN | Cloud CDN | Integrated with LB, cache invalidation | External CDN (integration complexity) |
| Load Balancer | Global HTTP(S) LB | Multi-region, auto-failover, Cloud Armor integration | Regional LB (no global routing) |
| Analytics | BigQuery | Serverless, petabyte scale, ML integration | Dataproc (operational overhead) |
| ML | Vertex AI | Managed ML, AutoML, feature store | Self-hosted (complexity, cost) |
| Message Queue | Pub/Sub | Global availability, at-least-once delivery | Cloud Tasks (not pub-sub model) |
Security Implementation: 1. PCI DSS Compliance: - VPC Service Controls for payment data perimeter - Separate GCP project for payment processing - Binary Authorization for container image validation - Customer-managed encryption keys (CMEK) for payment data - Audit logging exported to BigQuery for compliance reporting
- GDPR Compliance:
- Data residency in europe-west1 for EU customers
- DLP API for PII detection and redaction
- Data retention policies (7 years for orders, GDPR right to deletion)
-
Encryption at rest and in transit
-
Application Security:
- Cloud Armor WAF rules (OWASP Top 10 protection)
- Identity-Aware Proxy for admin access
- Workload Identity for pod-to-service authentication
- Secret Manager for API keys and credentials
Cost Optimization: - Committed use discounts for GKE nodes (40% savings: $200K → $120K/month) - Sustained use discounts for Compute Engine (automatic 30%) - Cloud Spanner processing units right-sized (monitoring-based) - BigQuery flat-rate pricing for predictable analytics costs - Cloud CDN cache hit ratio >85% (reduced origin load) - Preemptible VMs for batch processing (ML training, analytics jobs)
Estimated Monthly Cost Breakdown: - GKE clusters (2 regions, 100 nodes): $180K - Cloud Spanner (1000 processing units): $100K - Memorystore Redis (2 instances, 100GB each): $20K - Cloud Load Balancing + CDN: $40K - Cloud Storage + Firestore: $30K - BigQuery + Vertex AI: $60K - Networking (Interconnect, egress): $50K - Total: $480K/month (within $500K budget)
High Availability Design: - Multi-region deployment (us-central1 + europe-west1) - Global Load Balancing with health checks (5s interval) - Cloud Spanner multi-region configuration (99.999% SLA) - GKE cluster autoscaling (10-100 nodes per region) - Memorystore Redis with HA configuration - RTO: 15 minutes (automatic failover) - RPO: Near-zero (Spanner synchronous replication)
Performance Optimization: - Cloud CDN for static assets (images, CSS, JS) - 95% cache hit ratio - Memorystore Redis for session management (sub-ms latency) - Read replicas for read-heavy queries (product catalog) - Database query optimization (indexes, denormalization) - API response caching (Cloud CDN, API Gateway cache) - Vertex AI recommendations precomputed and cached
Monitoring and Observability: - Cloud Monitoring dashboards for key metrics (latency, errors, traffic) - SLOs defined: 99.9% availability, p95 latency <500ms, p99 latency <2s - Error budget: 0.1% (43 minutes/month) - Cloud Trace for distributed tracing across microservices - Cloud Logging centralized with log sinks to BigQuery - Alerting policies for critical metrics (uptime checks, error rates)
Integration with Existing Systems: - SAP ERP integration via Cloud Interconnect (10 Gbps) - Oracle database replication to Cloud SQL using Database Migration Service - Shared VPC for secure connectivity - Private Google Access for GCP API access without internet exposure
Exam Tips for Similar Scenarios: 1. Multi-region requirements → Cloud Spanner or Firestore multi-region 2. PCI DSS compliance → VPC Service Controls + separate project 3. Global low latency → Cloud CDN + Global Load Balancing 4. Extreme scalability → GKE or Cloud Run with autoscaling 5. Real-time inventory → Pub/Sub for event-driven updates 6. Analytics at scale → BigQuery with partitioning and clustering 7. Cost optimization → Committed use discounts + right-sizing
Case Study 2: Real-Time Financial Trading Platform¶
Company Profile: - High-frequency trading firm - Processing 1M trades per second during peak - Sub-10ms latency requirement - $50B daily trading volume - Regulatory compliance (SEC, FINRA)
Business Requirements: - Ultra-low latency (<10ms p99) - 99.999% availability (5 minutes downtime/year) - Strong consistency for trade execution - Real-time risk management - Audit trail for all transactions - Market data processing (1TB/hour) - Fraud detection in real-time
Technical Constraints: - Legacy trading engine (C++) must be migrated - Connection to stock exchanges via dedicated lines - Data retention: 7 years - Disaster recovery in <60 seconds - No data loss acceptable (RPO = 0)
Architecture Solution:
┌─────────────────────────────────────────────────────┐
│ Stock Exchanges (Low-latency links) │
└────────────────────┬────────────────────────────────┘
│ Dedicated Interconnect (10 Gbps)
│
┌────────────────────▼────────────────────────────────┐
│ Region: us-east4 (Virginia) │
│ │
│ ┌─────────────────────────────────────────────┐ │
│ │ GKE (Standard mode, high-perf networking) │ │
│ │ ┌─────────────┐ ┌──────────────────┐ │ │
│ │ │ Trading │ │ Risk Management │ │ │
│ │ │ Engine │ │ Service │ │ │
│ │ │ (Low-lat) │ │ │ │ │
│ │ └──────┬──────┘ └────────┬─────────┘ │ │
│ └─────────┼───────────────────┼───────────────┘ │
│ │ │ │
│ ┌─────────▼───────────────────▼─────────────┐ │
│ │ Cloud Spanner (Regional) │ │
│ │ - Trade Orders │ │
│ │ - Positions │ │
│ │ - 99.999% SLA │ │
│ └─────────┬───────────────────┬─────────────┘ │
│ │ │ │
│ ┌─────────▼─────────┐ ┌──────▼──────────┐ │
│ │ Memorystore Redis │ │ Cloud Bigtable │ │
│ │ (Order book cache)│ │ (Market data) │ │
│ │ Sub-ms latency │ │ 1M writes/sec │ │
│ └───────────────────┘ └────────┬────────┘ │
│ │ │
└──────────────────────────────────┼──────────────────┘
│
┌───────────▼────────────┐
│ Cloud Pub/Sub │
│ (Trade Events) │
└───────┬────────────────┘
│
┌──────────────┼──────────────┐
│ │ │
┌────────▼───────┐ ┌───▼─────────┐ ┌─▼──────────┐
│ Dataflow │ │ BigQuery │ │ Vertex AI │
│ (Streaming) │ │ (Analytics) │ │ (Fraud │
│ (Fraud Det) │ │ │ │ Detection) │
└────────────────┘ └─────────────┘ └────────────┘
Service Selection Rationale:
| Component | Service | Rationale |
|---|---|---|
| Trading Engine | GKE Standard + n2-highmem | Ultra-low latency, pod-to-pod optimization, tier-1 networking |
| Database | Cloud Spanner | Strong consistency, horizontal scale, 99.999% SLA |
| Order Book Cache | Memorystore Redis | Sub-millisecond reads, in-memory performance |
| Market Data | Cloud Bigtable | High-throughput writes (1M/s), time-series data |
| Event Stream | Pub/Sub | Reliable event delivery, integrates with Dataflow |
| Fraud Detection | Vertex AI + Dataflow | Real-time ML inference, streaming processing |
Performance Optimization: 1. Network Latency: - Deploy in us-east4 (closest to NYSE, NASDAQ) - Use Premium Network Tier for lowest latency - Dedicated Interconnect to exchanges (bypasses internet) - Pod anti-affinity for separate nodes
- Compute Optimization:
- n2-highmem machine types (high memory for order books)
- 64 vCPU per node for trading engine
- Local SSDs for ephemeral high-IOPS storage
-
Disable CPU throttling
-
Database Optimization:
- Cloud Spanner regional configuration (single region for lowest latency)
- Hotspot avoidance with UUID primary keys
- Memorystore Redis for read-heavy order book queries
-
Cloud Bigtable for time-series market data (row key: symbol#timestamp)
-
Caching Strategy:
- L1: In-memory cache in trading engine pods
- L2: Memorystore Redis for shared cache
- L3: Cloud Spanner with stale reads (10s staleness acceptable for some queries)
Security and Compliance: - VPC Service Controls for data exfiltration prevention - Audit Logs exported to immutable Cloud Storage (WORM) - Binary Authorization with attestation for deployed containers - Customer-managed encryption keys for all data - Access Transparency for Google access visibility - Security Command Center Premium for threat detection
Disaster Recovery: - Pilot light in us-central1 region - Cloud Spanner cross-region replica (async, 5-10s lag) - Continuous backup of market data to Cloud Storage - RTO: 60 seconds (manual failover) - RPO: Near-zero (Spanner replication)
Cost: ~$300K/month - Compute (GKE high-perf): $150K - Cloud Spanner (high processing units): $80K - Memorystore Redis: $30K - Cloud Bigtable: $20K - Networking (Interconnect): $20K
Exam Tips: - Ultra-low latency → Regional deployment, Premium Network Tier, Memorystore - Strong consistency → Cloud Spanner - High-throughput writes → Cloud Bigtable - Financial compliance → Audit Logs, VPC Service Controls, CMEK
Case Study 3: Healthcare Data Platform (HIPAA Compliant)¶
Company Profile: - Healthcare analytics company - Processing medical records for 500 hospitals - 50M patient records - Research and clinical analytics
Business Requirements: - HIPAA compliance - PHI (Protected Health Information) encryption - Audit trail for all data access - Data sharing with research institutions - De-identification for research - 99.95% availability - Multi-tenancy (one tenant per hospital)
Technical Constraints: - Data residency (US only) - Integration with HL7 FHIR standards - Legacy EMR systems (HL7v2, DICOM imaging) - Budget: $200K/month
Architecture Solution:
┌──────────────────────────────────────────────────┐
│ Hospital EMR Systems (On-prem) │
│ HL7v2, FHIR, DICOM │
└─────────────────────┬────────────────────────────┘
│
│ Cloud VPN (HA VPN, encrypted)
│
┌─────────────────────▼────────────────────────────┐
│ VPC (us-central1, HIPAA controls) │
│ ┌────────────────────────────────────────────┐ │
│ │ Healthcare API (FHIR stores) │ │
│ │ - Patient data │ │
│ │ - Clinical observations │ │
│ │ - DICOM imaging (DICOMweb) │ │
│ └──────────────┬─────────────────────────────┘ │
│ │ │
│ ┌──────────────▼──────────────────────────────┐ │
│ │ Cloud Storage │ │
│ │ - CMEK encryption │ │
│ │ - Object lifecycle management │ │
│ │ - Medical imaging (DICOM) │ │
│ └──────────────┬──────────────────────────────┘ │
│ │ │
│ ┌──────────────▼──────────────────────────────┐ │
│ │ DLP API (De-identification) │ │
│ │ - Remove PHI for research │ │
│ │ - Redact sensitive fields │ │
│ └──────────────┬──────────────────────────────┘ │
│ │ │
│ ┌──────────────▼──────────────────────────────┐ │
│ │ BigQuery (De-identified data) │ │
│ │ - Research analytics │ │
│ │ - Column-level encryption │ │
│ │ - Authorized views per institution │ │
│ └──────────────┬──────────────────────────────┘ │
│ │ │
└─────────────────┼──────────────────────────────────┘
│
┌────────▼────────┐
│ Looker Studio │
│ (Research BI) │
└─────────────────┘
Service Selection:
| Component | Service | HIPAA Justification |
|---|---|---|
| PHI Storage | Healthcare API | FHIR-native, HIPAA compliant, BAA available |
| Medical Imaging | Cloud Storage + Healthcare API | DICOM support, CMEK encryption |
| De-identification | DLP API | Automated PHI detection and redaction |
| Analytics | BigQuery | Column-level security, authorized views, audit logs |
| Access Control | VPC Service Controls | Data exfiltration prevention, IP allow-listing |
| Encryption | Cloud KMS (CMEK) | Customer-controlled encryption keys |
| Connectivity | HA VPN | Encrypted connectivity, 99.99% SLA |
HIPAA Compliance Implementation: 1. BAA (Business Associate Agreement): Sign with Google Cloud 2. Encryption: CMEK for all PHI data at rest, TLS 1.2+ for data in transit 3. Access Controls: - Least privilege IAM roles - Separate service accounts per hospital (tenant) - Context-aware access policies 4. Audit Logging: - All data access logged to Cloud Audit Logs - Exported to immutable Cloud Storage - Retained for 7 years 5. Data Segmentation: - Separate Healthcare API FHIR stores per hospital - BigQuery authorized views for multi-tenant data access - VPC Service Controls perimeter
Multi-Tenancy Design: - One FHIR store per hospital (logical isolation) - Separate Cloud Storage buckets with per-tenant CMEK keys - BigQuery authorized views for cross-tenant research queries - Resource labels for cost allocation per hospital - Organization policies for governance
Data Flow: 1. Hospital EMR → HA VPN → Healthcare API (FHIR ingestion) 2. Healthcare API → Cloud Storage (DICOM imaging) 3. Cloud Storage → DLP API → De-identified data → BigQuery 4. Researchers access via Looker Studio with OAuth authentication
Cost: ~$150K/month - Healthcare API (50M records): $50K - Cloud Storage (medical imaging, 1PB): $20K - BigQuery (analytics, 10TB processed/day): $30K - Cloud KMS (CMEK, 500 keys): $5K - HA VPN: $5K - Networking and other services: $40K
Exam Tips: - HIPAA compliance → BAA, CMEK, VPC Service Controls, Audit Logs - Healthcare data → Healthcare API (FHIR), DLP API for de-identification - Multi-tenancy → Separate FHIR stores, authorized views, resource labels - Data residency → Regional resources, organization policy constraints
Case Study 4: IoT Fleet Management Platform¶
Company Profile: - Smart logistics company - 100K connected vehicles and sensors - Real-time GPS tracking and telemetry - Predictive maintenance for fleet
Business Requirements: - Ingest 1 billion events per day (10K events/second) - Real-time location tracking with <5s latency - Predictive maintenance alerts - Historical analytics for route optimization - 99.9% availability - Secure device communication - Scale to 500K devices in next 2 years
Technical Constraints: - Existing devices use MQTT protocol - Budget: $100K/month - Legacy fleet management system (PostgreSQL) - Need mobile app for drivers - Data retention: 3 years hot, 7 years archive
Architecture Solution:
┌──────────────────────────────────────────────────┐
│ 100K IoT Devices (Vehicles, Sensors) │
│ MQTT Protocol │
└─────────────────────┬────────────────────────────┘
│ Secure MQTT
▼
┌─────────────────────────────────────────────────┐
│ IoT Core (Device Management) │
│ - Device Registry │
│ - Authentication (JWT tokens) │
│ - Protocol Gateway (MQTT → Pub/Sub) │
└─────────────────────┬───────────────────────────┘
│ Pub/Sub Topics
┌─────────────┼─────────────┬────────────┐
│ │ │ │
▼ ▼ ▼ ▼
┌────────────┐ ┌─────────┐ ┌──────────┐ ┌────────┐
│ Dataflow │ │ Cloud │ │ Bigtable │ │ Cloud │
│ Streaming │ │Functions│ │ (Real- │ │Storage │
│ (Alerts) │ │(Process)│ │ time) │ │(Archive)│
└──────┬─────┘ └────┬────┘ └────┬─────┘ └────────┘
│ │ │
▼ ▼ ▼
┌──────────────┐ ┌────────────────────────────┐
│ Firebase │ │ BigQuery │
│ Realtime DB │ │ (Data Warehouse) │
│ (Mobile App) │ │ - Historical Analytics│
└──────────────┘ │ - ML Training Data │
└────────┬───────────────────┘
│
┌──────▼──────┐
│ Vertex AI │
│ (Predictive│
│ Maintenance)
└─────────────┘
Service Selection: | Component | Service | Rationale | |-----------|---------|-----------| | Device Connectivity | IoT Core | MQTT protocol support, device authentication, auto Pub/Sub integration | | Message Ingestion | Pub/Sub | 10K/s throughput, reliable delivery, fanout to multiple consumers | | Real-time Processing | Dataflow | Stream processing, windowing, exactly-once semantics | | Real-time Storage | Bigtable | High write throughput, sub-10ms reads, time-series data | | Historical Analytics | BigQuery | Serverless, partitioned tables, ML integration | | Archival | Cloud Storage Coldline | Cost-effective long-term storage | | Mobile App | Firebase | Real-time sync, offline capability, authentication | | Predictive ML | Vertex AI | AutoML, model training and deployment |
Data Flow: 1. Devices send telemetry → IoT Core (MQTT) 2. IoT Core publishes → Pub/Sub topics (by event type) 3. Real-time path: Pub/Sub → Dataflow → Bigtable (for dashboards) 4. Analytics path: Pub/Sub → BigQuery (streaming insert) 5. Alert processing: Dataflow detects anomalies → Firebase → Mobile app notifications 6. Archive: BigQuery → Cloud Storage (Object Lifecycle Management)
Security: - IoT Core device authentication with JWT tokens - TLS 1.2 for device-to-cloud communication - Per-device credentials with automatic rotation - VPC Service Controls for data exfiltration prevention - IAM roles for principle of least privilege
Cost Optimization: - Preemptible workers for Dataflow (60% savings on batch jobs) - BigQuery partitioning by date (reduce scan costs) - Cloud Storage lifecycle: Standard (30 days) → Nearline (1 year) → Coldline (3 years) → Archive (7 years) - Bigtable autoscaling based on CPU utilization - Committed use discount for Bigtable nodes
Cost: ~$85K/month - IoT Core (100K devices, 1B messages/day): $20K - Pub/Sub (1B messages/day): $15K - Dataflow (streaming jobs): $25K - Bigtable (3 nodes with autoscaling): $10K - BigQuery (10TB/month data, 50TB scanned): $10K - Cloud Storage (100TB Coldline): $3K - Vertex AI (model training/inference): $2K
Exam Tips: - IoT at scale → IoT Core + Pub/Sub + Dataflow - Time-series data → Bigtable with row key design (device_id#timestamp) - Real-time + Historical → Bigtable for real-time, BigQuery for analytics - Cost optimization → Storage lifecycle, preemptible VMs, partitioning
Case Study 5: Media Streaming Platform¶
Company Profile: - Video streaming service - 10 million subscribers - 4K/HD video content - Global audience across 50 countries
Business Requirements: - Low latency video streaming (<2s) - Support 100K concurrent streams during peak - Content transcoding (multiple bitrates) - Personalized recommendations - DRM and content protection - 99.95% availability - Minimize CDN costs
Technical Constraints: - 500TB video library (growing 50TB/month) - Budget: $300K/month - Existing Oracle database for user data - Mobile apps (iOS/Android), web, smart TVs - Need to comply with regional content licensing
Architecture Solution:
┌────────────────────┐
│ Content Upload │
│ (Cloud Storage) │
└─────────┬──────────┘
│
┌─────────▼──────────┐
│ Transcoder API │
│ - Multiple formats│
│ - Adaptive bitrate│
└─────────┬──────────┘
│
┌─────────▼──────────┐
│ Cloud Storage │
│ (Multi-regional) │
│ - Standard tier │
└─────────┬──────────┘
│
┌─────────▼──────────┐
│ Cloud CDN │
│ (Global) │
│ Cache hit >90% │
└─────────┬──────────┘
│
┌────────────────────┼────────────────────┐
│ │ │
┌───────▼───────┐ ┌────────▼──────┐ ┌────────▼──────┐
│ Web App │ │ Mobile Apps │ │ Smart TV │
│ (Cloud Run) │ │ (Firebase) │ │ (Cloud Run) │
└───────┬───────┘ └────────┬──────┘ └────────┬──────┘
│ │ │
└────────────────────┼────────────────────┘
│
┌──────────────┴──────────────┐
│ │
┌───────▼────────┐ ┌────────▼────────┐
│ Cloud Spanner │ │ Memorystore │
│ - User data │ │ - Session mgmt │
│ - Subscriptions│ │ - Cache │
└───────┬────────┘ └─────────────────┘
│
┌───────▼────────┐ ┌─────────────────┐
│ BigQuery │ │ Vertex AI │
│ - View analytics│ │ - Recommendations│
│ - User behavior│ │ - Content discovery│
└────────────────┘ └─────────────────┘
Service Selection: | Component | Service | Rationale | |-----------|---------|-----------| | Video Storage | Cloud Storage Multi-regional | High durability, regional replication, CDN integration | | Transcoding | Transcoder API | Multiple formats, adaptive bitrate, parallel processing | | CDN | Cloud CDN | Global edge locations, high cache hit ratio, reduces origin load | | Streaming App | Cloud Run | Autoscaling, container-based, low latency | | User Database | Cloud Spanner | Global consistency, horizontal scale, multi-region | | Caching | Memorystore Redis | Sub-ms latency, session management, API caching | | DRM | Widevine/PlayReady | Content protection, license management | | Recommendations | Vertex AI | ML-based personalization, real-time inference | | Analytics | BigQuery | View tracking, user behavior analysis, real-time dashboards |
Content Delivery Optimization: 1. Multi-CDN Strategy: - Primary: Cloud CDN (90% of traffic) - Fallback: Third-party CDN for regions with poor coverage - Cache hit ratio target: >95%
- Adaptive Bitrate Streaming:
- Multiple bitrates: 4K (25 Mbps), 1080p (8 Mbps), 720p (5 Mbps), 480p (2.5 Mbps)
- HLS or DASH protocols
-
Client-side quality adaptation
-
Storage Optimization:
- Hot content (last 30 days): Standard storage class
- Cold content (30-365 days): Nearline storage class
- Archive (>365 days): Coldline storage class
- Lifecycle policies for automatic tier transition
Security and DRM: - Widevine DRM for Android, ChromeOS - FairPlay DRM for iOS, tvOS - PlayReady DRM for Windows, Xbox - Signed URLs with expiration for content access - Content encryption at rest (default GCS encryption) - TLS 1.3 for all client connections
Global Content Distribution: - Multi-regional Cloud Storage buckets for low-latency access - Regional content restrictions using Cloud Armor geo-filtering - Separate content catalogs per region (licensing compliance)
Cost Optimization: - Cloud CDN cache hit ratio >95% (reduced origin egress) - Storage lifecycle management (Standard → Nearline → Coldline) - Committed use discounts for Cloud Spanner (40% savings) - Sustained use discounts for Cloud Run (automatic) - Preemptible VMs for batch transcoding jobs (80% savings)
Cost: ~$280K/month - Cloud Storage (500TB Standard, growing): $10K - Cloud CDN (1PB egress with 95% cache hit): $150K - Transcoder API (50TB/month): $30K - Cloud Spanner (100 processing units): $10K - Cloud Run (API backends): $20K - Memorystore Redis (100GB): $10K - BigQuery + Vertex AI: $20K - DRM licensing and other: $30K
Scalability: - Cloud Run autoscaling: 0 to 1000 instances - Cloud Spanner: Horizontal scaling with processing units - Cloud CDN: Automatic global scaling - Memorystore: Vertical scaling with minimal downtime
Monitoring and Analytics: - Cloud Monitoring for infrastructure metrics - BigQuery for view analytics and user behavior - Cloud Logging for application logs - SLO: 99.95% availability, p95 latency <2s
Exam Tips: - Media streaming → Cloud CDN, Cloud Storage, Transcoder API - Global scale → Multi-regional resources, Cloud CDN - Cost optimization for CDN → High cache hit ratio, lifecycle management - DRM → Widevine, FairPlay, PlayReady - Transcoding → Transcoder API for managed service, or Compute Engine with FFmpeg for custom needs
Case Study 6: Gaming Backend (Real-time Multiplayer)¶
Company Profile: - Mobile game developer - Real-time multiplayer battles (100 players per match) - 5 million daily active users - Global player base
Business Requirements: - Ultra-low latency (<50ms p95) - Support 50K concurrent matches - Real-time matchmaking - Leaderboards and player stats - In-game chat - Anti-cheat detection - 99.9% availability
Architecture Solution:
┌──────────────────────────────────────────────────┐
│ Mobile Clients (iOS/Android) │
│ Unity/Unreal Engine │
└─────────────────────┬────────────────────────────┘
│ WebSocket connections
▼
┌─────────────────────────────┐
│ Global HTTP(S) LB │
│ (Proximity-based routing) │
└──────┬──────────────┬───────┘
│ │
┌──────────▼────┐ ┌────▼──────────┐
│ Region: │ │ Region: │
│ us-central1 │ │ asia-east1 │
│ │ │ │
│ ┌─────────┐ │ │ ┌─────────┐ │
│ │ GKE │ │ │ │ GKE │ │
│ │ Game │ │ │ │ Game │ │
│ │ Servers│ │ │ │ Servers│ │
│ └────┬────┘ │ │ └────┬────┘ │
│ │ │ │ │ │
│ ┌────▼────┐ │ │ ┌────▼────┐│
│ │Memorystore│ │ │ │Memorystore││
│ │ (State) │ │ │ │ (State) ││
│ └─────────┘ │ │ └─────────┘ │
└───────────────┘ └───────────────┘
│ │
└─────────┬─────────┘
│
┌──────────▼──────────┐
│ Cloud Spanner │
│ - Player profiles │
│ - Match history │
│ - Leaderboards │
└──────────┬──────────┘
│
┌──────────▼──────────┐
│ Cloud Pub/Sub │
│ - Match events │
│ - Chat messages │
└──────────┬──────────┘
│
┌─────────────┼─────────────┐
│ │ │
┌────▼────┐ ┌────▼────┐ ┌─────▼────┐
│ Dataflow│ │BigQuery │ │ Vertex AI│
│(Analytics)│ │(Analytics)│ │(Anti-cheat)│
└─────────┘ └──────────┘ └──────────┘
Service Selection: | Component | Service | Rationale | |-----------|---------|-----------| | Game Servers | GKE with Agones | Game server orchestration, autoscaling, stateful workloads | | Session State | Memorystore Redis | Sub-ms latency, in-memory state, persistence | | Player Data | Cloud Spanner | Strong consistency, global scale, low latency | | Matchmaking | Cloud Run | Serverless, autoscaling, REST APIs | | Load Balancing | Global HTTP(S) LB | Proximity routing, WebSocket support | | Real-time Events | Pub/Sub | Event streaming, fanout to multiple consumers | | Anti-cheat | Vertex AI | ML-based anomaly detection, real-time inference | | Leaderboards | Bigtable | High read/write throughput, time-series data |
Game Server Design: - Agones (open-source K8s controller for game servers): - Dedicated game server per match - Automatic scaling based on player demand - Health checks and server lifecycle management - Fleet autoscaling (10-1000 servers)
Low-Latency Optimizations: 1. Regional Deployment: Deploy game servers in 10+ regions globally 2. Proximity Routing: Global Load Balancer routes to nearest region 3. In-Memory State: Memorystore Redis for match state (no disk I/O) 4. WebSocket: Persistent connections for real-time communication 5. Premium Network Tier: Lower latency between regions 6. Custom Machine Types: High CPU, low memory for game logic
Matchmaking System: - Cloud Run service with autoscaling - Player skill rating stored in Cloud Spanner - Queue system with Pub/Sub - Match formation algorithm (skill-based, latency-based) - Average matchmaking time: <30 seconds
Leaderboards: - Global leaderboards: Bigtable (row key: score#player_id) - Regional leaderboards: Bigtable with separate table per region - Real-time updates via Pub/Sub → Dataflow → Bigtable - Caching top 100 players in Memorystore
Anti-Cheat: - Vertex AI model trained on player behavior patterns - Real-time inference on match events - Anomaly detection (impossible movements, aim bot detection) - Automatic flagging for review
Cost: ~$150K/month - GKE game servers (10 regions, 500 nodes total): $80K - Memorystore Redis (10 instances): $20K - Cloud Spanner (500 processing units): $50K - Bigtable (leaderboards): $5K - Cloud Load Balancing: $10K - Other services (Pub/Sub, BigQuery, Vertex AI): $15K
Exam Tips: - Real-time gaming → GKE with Agones, Memorystore for state - Global low latency → Multi-region deployment, Premium Network Tier, proximity routing - Leaderboards → Bigtable for high throughput, clever row key design - WebSocket → Global HTTP(S) Load Balancer supports WebSocket - Stateful workloads → GKE StatefulSets or Agones for game servers
Architecture Decision Records (ADRs)¶
Purpose: Document significant architectural decisions, context, and trade-offs.
ADR Template:
## ADR-### : [Title]
Date: YYYY-MM-DD
Status: [Proposed | Accepted | Superseded | Deprecated]
### Context
What is the issue we're facing that motivates this decision?
### Decision
What is the change we're proposing or have agreed to implement?
### Consequences
What becomes easier or more difficult to do because of this change?
### Alternatives Considered
What other options were evaluated?
### Trade-offs
Cost vs. Performance vs. Complexity vs. Security
Example ADR:
ADR-001: Use Cloud Spanner for Global E-Commerce Database¶
Date: 2024-10-11 Status: Accepted
Context: - E-commerce platform needs globally consistent database - Support 50K requests/second with <100ms latency - Multi-region deployment (US, EU, Asia) - Strong consistency for inventory and orders - 99.99% availability requirement
Decision: Use Cloud Spanner with multi-region configuration (nam-eur-asia3) for transactional database.
Consequences: Positive: - Strong consistency across all regions - Automatic horizontal scaling - 99.999% SLA - No need for complex replication logic - Global SQL queries
Negative: - Higher cost than Cloud SQL (~10x for equivalent workload) - Limited to 10,000 processing units per instance - More complex schema design (avoid hotspots) - Learning curve for interleaved tables
Alternatives Considered: 1. Cloud SQL with cross-region replicas: - Pro: Lower cost, familiar PostgreSQL/MySQL - Con: Eventual consistency, manual failover, no global transactions
- Firestore multi-region:
- Pro: Lower cost, good for document model
-
Con: Limited query capabilities, eventual consistency option only
-
Self-managed MySQL with Vitess:
- Pro: More control, potentially lower cost
- Con: Operational overhead, need to manage sharding
Trade-offs: - Cost (+50%): Spanner is expensive but eliminates complex application logic for consistency - Performance (Excellent): Sub-100ms globally, horizontal scaling - Complexity (Lower): Managed service reduces operational burden - Security (Excellent): Built-in encryption, IAM integration, audit logging
Cost Analysis: - Cloud Spanner (1000 PUs): $100K/month - Alternative (Cloud SQL HA + replicas): $40K/month + application complexity - Decision: Accept higher cost for simplified architecture and better SLAs
Integration Patterns¶
API Management with Apigee¶
Use Cases: - API monetization and developer portals - Rate limiting and quota management - API analytics and monitoring - Legacy system facade - Multi-cloud API gateway
Architecture:
External Clients → Apigee Edge → Backend Services (GKE/Cloud Run)
↓
Analytics, Policies, Security
Key Features: - API proxies with transformation - OAuth 2.0, API keys, JWT validation - Rate limiting, quota management - Developer portal for API documentation - Hybrid deployment (cloud + on-premises)
When to Use: - Complex API management requirements - Need for API monetization - Legacy system integration - Multi-cloud deployments
Alternative: API Gateway for simpler use cases (serverless, lower cost)
Pub/Sub Messaging Patterns¶
1. Fan-out Pattern: - One message to multiple subscribers - Use case: Order placed → notify inventory, shipping, analytics - Implementation: One topic, multiple subscriptions
2. Work Queue Pattern: - Distribute work across multiple workers - Use case: Image processing, batch jobs - Implementation: Pull subscriptions with load balancing
3. Dead Letter Queue Pattern: - Handle failed messages - Use case: Retry poisoned messages, alert on failures - Implementation: Configure dead letter topic on subscription
Event Routing with Eventarc¶
Purpose: Route events from 100+ GCP sources to Cloud Run, Cloud Functions, or GKE
Sources: - Cloud Storage (object finalized, deleted) - Pub/Sub messages - Cloud Audit Logs (any API call) - Custom events via Pub/Sub
Example:
Cloud Storage (new file) → Eventarc → Cloud Run (process file)
BigQuery (job complete) → Eventarc → Cloud Function (send notification)
Benefits: - Declarative event routing - No code to poll or listen - Automatic retries - Built-in filtering
Exam Preparation Strategy¶
How to Approach Case Study Questions¶
Step-by-Step Approach:
- Read Carefully (5 minutes):
- Identify business requirements (availability, scale, cost)
- Note technical constraints (existing systems, budget, team skills)
- Highlight compliance requirements (HIPAA, PCI-DSS, GDPR)
-
Understand non-functional requirements (RTO, RPO, latency)
-
Categorize Requirements:
- Must-Have: Core business requirements
- Should-Have: Important but not critical
-
Nice-to-Have: Future enhancements
-
Identify Patterns:
- High availability → Multi-region, redundancy
- Low latency → CDN, caching, regional deployment
- Global scale → Cloud Spanner, Global Load Balancing
- Real-time processing → Pub/Sub, Dataflow
- Analytics → BigQuery
-
Cost-sensitive → Serverless, preemptible VMs, lifecycle policies
-
Service Selection:
- Start with compute layer (GKE, Cloud Run, Compute Engine)
- Choose database based on consistency needs (Spanner, Cloud SQL, Firestore, Bigtable)
- Add caching where appropriate (CDN, Memorystore)
- Include messaging for async processing (Pub/Sub)
-
Consider security and compliance services
-
Validate Design:
- Does it meet availability SLA?
- Can it scale to required load?
- Is it within budget?
- Does it satisfy compliance requirements?
- Are there single points of failure?
Common Exam Mistakes to Avoid¶
- Over-Engineering:
- Don't choose complex solutions when simple ones suffice
-
Example: Using Cloud Spanner for a simple web app with regional users → Use Cloud SQL instead
-
Ignoring Costs:
- Always consider cost implications
-
Look for cost optimization opportunities (CUDs, preemptible VMs, lifecycle policies)
-
Missing Compliance Requirements:
- HIPAA → BAA, CMEK, VPC Service Controls, Healthcare API
- PCI-DSS → Separate project, VPC Service Controls, audit logging
-
GDPR → Data residency, DLP API, right to deletion
-
Forgetting High Availability:
- Single-zone deployments have no SLA
- Always use multi-zone for production
-
Consider multi-region for mission-critical systems
-
Wrong Database Choice:
- Strong consistency globally → Cloud Spanner
- Relational, regional → Cloud SQL
- Document model → Firestore
- High-throughput, low-latency → Bigtable
-
Analytics → BigQuery
-
Ignoring Existing Constraints:
- Legacy systems that must be integrated
- Team skills and operational capabilities
- Budget limitations
- Timeline constraints
Service Selection Decision Matrix¶
| Requirement | Recommended Services | Why |
|---|---|---|
| Global consistency | Cloud Spanner | Only option for global ACID transactions |
| Regional relational DB | Cloud SQL | Cost-effective, familiar PostgreSQL/MySQL |
| Document store | Firestore | Real-time sync, offline support, mobile-friendly |
| High-throughput NoSQL | Bigtable | Petabyte scale, sub-10ms latency, time-series |
| Data warehouse | BigQuery | Serverless, petabyte scale, ML integration |
| Object storage | Cloud Storage | Durable, lifecycle management, CDN integration |
| In-memory cache | Memorystore | Sub-ms latency, Redis/Memcached compatibility |
| Container orchestration | GKE | Full Kubernetes, complex workloads |
| Serverless containers | Cloud Run | Simple deployment, autoscaling, cost-effective |
| FaaS | Cloud Functions | Event-driven, lightweight processing |
| Message queue | Pub/Sub | Reliable, scalable, at-least-once delivery |
| Stream processing | Dataflow | Apache Beam, exactly-once, complex transformations |
| Batch processing | Dataproc or Batch | Hadoop/Spark or custom batch jobs |
| API management | Apigee or API Gateway | Apigee for complex, API Gateway for simple |
| CDN | Cloud CDN | Global, integrated with Load Balancing |
| Load balancing | Cloud Load Balancing | Global, regional, internal options |
| Hybrid/Multi-cloud | Anthos | Kubernetes everywhere, consistent management |
| IoT | IoT Core | Device management, MQTT, auto Pub/Sub integration |
| ML/AI | Vertex AI | Managed ML, AutoML, model deployment |
| DLP | DLP API | PII detection, de-identification, redaction |
Practice Resources¶
- Official GCP Case Studies:
- Mountkirk Games (gaming)
- Dress4Win (e-commerce/fashion)
-
TerramEarth (IoT/heavy equipment)
-
Hands-On Practice:
- Create sample architectures in draw.io or Lucidchart
- Estimate costs using GCP Pricing Calculator
-
Build proof-of-concepts for key patterns
-
Mock Exams:
- Official practice exam
- Third-party practice tests
-
Review incorrect answers and understand why
-
Documentation:
- Architecture Center (cloud.google.com/architecture)
- Solution guides for common patterns
- Best practices for each service
Study Tips¶
Architecture Design Practice¶
- Case Study Analysis: Work through official GCP case studies (Mountkirk Games, Dress4Win, TerramEarth)
- Draw Architectures: Practice creating architecture diagrams using standard notation
- Cost Calculations: Estimate costs for different designs using GCP Pricing Calculator
- Trade-off Analysis: Compare multiple approaches for same problem (cost vs. performance vs. complexity)
- Security Review: Identify security gaps in architectures and propose remediations
- Scalability Analysis: Determine scaling bottlenecks and design for horizontal scalability
- Migration Planning: Create detailed migration plans for lift-and-shift, replatform, and refactor scenarios
Key Concepts to Master¶
- Service Selection: Know when to use each GCP service and trade-offs between alternatives
- Architecture Patterns: Understand common patterns (microservices, event-driven, serverless) and anti-patterns
- Trade-offs: Balance performance, cost, complexity, and security in every design decision
- NFRs: Design for non-functional requirements (availability, latency, throughput, durability)
- Migration Strategies: 6 Rs (Rehost, Replatform, Refactor, Repurchase, Retire, Retain) and when to use each
- Cost Modeling: Calculate and optimize TCO including compute, storage, networking, and licensing
- Compliance: Design for regulatory requirements (HIPAA, PCI-DSS, GDPR, SOC 2)
Common Exam Topics¶
- Scenario-Based Design: Designing complete architectures for given business scenarios with constraints
- Service Selection: Choosing appropriate GCP services based on requirements
- Migration Planning: Planning and executing cloud migrations with minimal downtime
- High Availability and DR: Designing for multi-region failover, RTO/RPO targets
- Cost Optimization: Right-sizing, committed use discounts, preemptible VMs, storage lifecycle
- Security and Compliance: VPC Service Controls, CMEK, IAM, audit logging
- Performance Optimization: Caching strategies, database optimization, CDN usage
- Hybrid and Multi-Cloud: Anthos, VPN vs. Interconnect, consistent management
Exam Day Tips¶
- Time Management: 2 hours for 50 questions = ~2.4 minutes per question
- Read Carefully: Pay attention to constraints (budget, timeline, existing systems, compliance)
- Eliminate Wrong Answers: Often easier to eliminate 2-3 obviously wrong answers
- Look for Keywords: "Global", "low latency", "strong consistency", "cost-effective", "serverless"
- Consider Trade-offs: Best answer often balances multiple concerns
- Trust Your Knowledge: Don't second-guess yourself too much
- Flag and Review: Flag uncertain questions and review if time permits
Migration Strategies (6 Rs)¶
1. Rehost (Lift-and-Shift)¶
- Move as-is to cloud
- Minimal changes
- Fast migration
- Limited cloud benefits
- GCP Tools: Migrate for Compute Engine
2. Replatform (Lift-and-Reshape)¶
- Minor optimizations
- Use managed services
- Moderate effort
- Some cloud benefits
- Example: Move to Cloud SQL instead of self-managed DB
3. Refactor (Re-architect)¶
- Redesign for cloud-native
- Maximum cloud benefits
- Highest effort
- Best long-term approach
- Example: Monolith to microservices
4. Repurchase (Replace)¶
- Move to SaaS
- Abandon custom software
- Fast transition
- Recurring costs
- Example: Move to Google Workspace
5. Retire¶
- Decommission unnecessary applications
- Reduce maintenance
- Cost savings
- Simplify landscape
6. Retain¶
- Keep on-premises
- Not ready for migration
- Regulatory constraints
- Migrate later