Monthly Archives: January 2021

GCP Cloud Architect Study Guide – Databases

FeaturesSizeLatencyComparable
Cloud SQLManaged MySQL, PostgreSQL and SQL ServerUp to 30TB
Cloud SpannerUnlimited scale relational databasePBRegional Low Latency R/W
Multiregional Low Latency Read
Oracle
DynamoDB
Cloud BigtableNo-SQL, Key Value, Low Latency PB< 10msHBase
Cassandra
Firestore (Datastore)No-SQL, document based, key:value pairsPB>10ms
Firebase
Memory StoreIn-memory, managed Redis and Memcached compatibleReids:300GB
Memcached:5TB
<10ms

Cloud SQL Notes and Best Practices

  • Encrypted and have automatic backup available.
  • Data replication and automatic failover between zones

Cloud Spanner Notes and Best Practices

  • Globally replicated ensures very high availability (99.999%)
  • Strong consistency across all replicas as well as external consistency (commit order is preserved)
  • Local availability for better response times
  • Regional instances replicate across 3 zones.
  • Multi-Regional instances replicate across multiple regions. This allows for read latency to be maintained but write latency is impacted due to the mechanisms to ensure consistency.
  • Place read-write workloads with read-write replicas, read-only workloads with read-only replicas

Bigtable Notes and Best Practices

  • Single key based index.
  • Scale linearly with additional nodes
  • Highly durable using Colossus magic

FireStore (Datastore) Notes and Best Practices

  • Do not write to a document more than 1x per second
  • Regional and multi-regional options are available, select based on costs, availability and latency
  • Multi-regional availability is 99.999%
  • Datastore mode allows for backwards compatibility to Datastore and is required for App Engine integration
  • Supports offline data persistence

MemoryStore Notes and Best Practices

  • Replication across zones for standard tier to provide HA
  • Must use Private Service Access to connect.
  • Memcached is in BETA.

Summary of use cases

Use Cases
Cloud SQLSQL Support for OLTP
ERP, CRM, E-Commerce, Web
Cloud SpannerSQL Support for OLTP
Global Financial Ledger
Supply Chain/Inventory Management
BigTableStreaming time series data (IoT)
Fraud detection
Personalization
Ad tech
Financial Analysis
FirestoreUser Profiles
User Sessions
State Management
MemorystoreLow latency in memory data storage

GCP Cloud Architect Study Guide – Resource Management and IAM

Organizations form the root of the resource hierarchy. Folders, Projects and ultimately resources flow up into an Organization.

  • Cloud Identity is an Identity Service which manages users and groups. Cloud Identity can be configured to support federation with AD or AAD for users and groups.
  • IAM manages the role based access to resources. IAM policies let you control who, has what type of access to which resource.
  • The resource hierarchy provides an inheritance model to allow for structured application of IAM policies to resources.
  • Organization Policy manages the actions which are allowed on a resource regardless of who is performing them. It provides for a centralized and programatic control over cloud resources within an Organization.

Policies are inherited from higher level resources.

  • You cannot revoke a policy which has been inherited.
  • Policies are not retroactive. Resources which are already in place but violating a policy must be manually remediated.
  • Moving a project from one folder to another inherits all of the policies from the target hierarchy

GCP Cloud Architect Study Guide – Data Services

ETL solutions

Underlying TechnologyUse Case
DataflowApache BeamBatch and Streaming Data at Scale
Log ingestion/transformation
AI/ML
Data FusionCDAP
DataProc
Streaming Data at Scale from various sources into a DWH
Regular ingestion into DWH
DataProcGCESpark/Hadoop Cluster
Distributed processing of large datasets
DataprepApache BeamData visualization, to explore, clean and prep for analysis and ML.

Not being a data scientist and not having enough knowledge on the individual products, use cases or technology makes this one hard to get my head around.