b20c:c011:5:b109::/64

AI services on GCP have been divided into two main areas.

AI Building Blocks are a set of pre-trained models which can be used as is or customized to easily develop AI enhanced applications. By providing appropriate input to the building block (video, document, speech, etc.) the AI model can provide specific responses (annotations, text, etc.).

AI Platforms and Accelerators are a set of managed services to assist in the building of AI solutions. This includes

• VMs or Kubeflow clusters with AI tools (Tensorflow Enterprise, Jupyter Notebooks, etc.) pre installed. These services include additional infrastructure components, GPUs and TPUs, to support model training.
• Tensorflow Enterprise is a managed TF solution which provides additional support over the opensource version.

AI Solutions are vertically focused AI platforms and APIs meant to solve a specific business problem within a vertical.

The architectural focus is on a 3 tier architecture consisting of

• Presentation Layer – transfers data from the user and between services
• Business Logic Layer – performs tasks useful for the user, retrieval, processing, transformation and management of data, business logic and policies
• Data Access Layer – stores and retrieves data for the BLL which is then transformed and passed to the user

For each layer you must consider the following technical aspects

• Resiliency – ability to recover well
• Reliability – trustworthyness of the service and the data
• Scalability – ability to grow and shrink as load changes over time
• Availability – reachability of the services
• Disaster Recovery – ability to recover after a catastrophic event

As well as the security aspects

• Privacy
• Denial of Service

Management of the environment includes

• Service Provisioning – Launching the service
• Monitoring/Logging
• Alerting
• Responding
• Capacity Planning
• Cost Management

Twelve Factor Methodology

App design should follow the Twelve Factor App Development methodology.

1. Codebase – Track all code in a source code repository
2. Dependencies – Use containers and package all dependencies into the container.
3. Configuration – Separate configuration from code base. Store configuration in environment variables.
4. Backing Services – Treat storage and messaging queues as external services.
5. Build, Release Run – Separate development into the three stages
6. Processes – Apps run as processes. These processes should be stateless and not share data with other processes. Data should be persisted in an external backing service.
7. Port Binding – Do not use app containers (Tomcat, etc.) instead package the webserver library in the container and expose the port as an evnironment variable.
8. Concurrency – Decompose your app into independent processes which can scale on demand.
9. Disposability – Apps should be able to be removed and restarted easily and quickly.
10. Environment Parity – Dev, Test, Prod, etc. should be kept as similar as possible.
11. Logs – Decouple log collection, processing and analysis from apps
12. Admin Processes – Decouple one-off admin tasks (reporting, backups, batch jobs, etc.) from the app itself

Depolyments and Testing

• Blue/Green Deployment – Is a way of performing a controlled depolyment of a new version of an application. You have two identical environments, blue and green. If blue is the production environment, you implement the new version to the green staging environment. In the green environment you can perform any final tests and when ready, gracefully cut over all users. If needed failback to blue.
• A/B Testing – Is for testing new software features. It is not used for deploying the feature.
• Canary Deployments – Deploys new features/version to a subset of all users allowing for user feedback and bug identification before making available to all users.
• Rolling Deployments / Upgrades – Deploys a new version of the OS or application to a subset of the servers as load on those servers is removed. Generally best used with stateless servers managed through a load balancer.

Migrating to the cloud is centered around a pre-defined framework

• Assess
• Plan – in this phase the things to consider are
  • From an Enterprise perspective, you should consider changes to the Operational model when moving to a shared responsibility mode, compliance and security requirements and Data Residency restrictions
  • The resource hierarchy needs to also be considered in terms of how the organization structure maps onto folders and projects. Quota management should also be a consideration to control how much of a resource can be consumed.
  • IAM considerations such as mappings for the administrative roles, identity integration/SSO, and service accounts for applications.
  • Billing considerations include understanding how cloud billing maps to internal billing processes, how to allocate costs to appropriate cost center and defining who has access to cloud billing.
  • As networking relates to migrations, understand the data transfer models including egress billing and zonal/regional/global resources.
• Deploy – Automatic deployment is preferred over manual deployment.
  • Configuration management tools are useful for configuring or deploying the environment or to implement a deployment process for your workloads.
  • Configuration automation will help to structure the deployment into an automated and auditable process
  • Infrastructure as Code tools allow you to programmatically deploy your infrastructure entirely through the API
  • Container orchestration tools like K8S when you have already containerized your applications.
• Optimize – set realistic goals to optimize your environment and teams. Measure success against these goals. Optimization can be planned for service management or cost management and can be implemented through moving to more cloud native services, upskilling a team, codifying or automating the environment, etc.

Data Migration Options

3 options are available for data migration (outside of application specific replication, or using Cloud Migrate)

• gsutil should be used for smaller migration volumes from on-prem
  • a few TB with normal enterprise network connectivity
  • a small number of files
• Storage Transfer Service should be used to migrate data from other cloud providers or from GCS to GCS. If bandwidth is sufficient and available, STS for On-Prem can be used for larger volumes of on-prem data located on NFS drives.
• Transfer Appliance should be used for large transfers where bandwidth is limited. In general volumes above 10TB should be considered as candidates for a physical transfer depending on bandwidth.

Cloud Migration for Compute Engine (formerly Velostrata) can be used to migrate already virtualized VMs to GCP.

• Source environments AWS, Azure, VMware
• Min Bandwidth larger of 20Mbps or 0.5Mbps per concurrent VM to be migrated
• Minimizes bandwidth by 75% using dedup, etc.