From Zero to ML

Make AI easy,
fast and useful
for enterprises
and developers
Serverless ML on Google Cloud Platform
Cloud Cloud ML
AutoML Engine

Data
Application Scientists
Developers & ML
Practitioners

Machine BQML
Learning APIs
AI building blocks for
all developers
Fully trained ML models from Google Cloud Platform

Cloud Cloud Cloud Natural Cloud Cloud Video Cloud

Translation Vision Language Speech-to-Text Intelligence Text-to-Speech
Stay tuned
AI / ML API
Build AI Solution by Simply Calling the API 1
Cloud Vision API

Label & web detection OCR Logo detection

Landmark detection Crop hints Explicit content detection

Cloud Natural Language API

Extract Detect Analyze Classify

entities sentiment syntax content
Cloud AutoML
Train custom machine learning models for
vision, natural language, and translation
without writing model code
2
How it works
AutoML
Generate predictions
Your training data with a REST API

Train Deploy Serve

 Cloud AutoML - Best-in-class Research
Learning to learn Transfer Learning Hyperparameter Tuning

Inception-v4, Inception-ResNet and the Progressive Neural Architecture Search Neural Architecture Search with Reinforcement
Impact of Residual Connections on Learning Chenxi Liu, Barret Zoph, Jonathon Shlens, Wei Learning
Christian Szegedy, Sergey Ioffe, Vincent Hua, Li-Jia Li, Li Fei-Fei, Alan Yuille, Jonathan Barret Zoph, Quoc V. Le. ICLR 2017.
Learning Transferable Architectures for Scalable Vanhoucke, and Alex Alemi. AAAI, 2017. Huang, Kevin Murphy, Arxiv, 2017
Image Recognition, Barret Zoph, Vijay Vasudevan,
Jonathon Shlens, and Quoc V. Le. Arxiv, 2017.
Large-Scale Evolution of Image Classifiers Bayesian Optimization for a Better Dessert
Esteban Real, Sherry Moore, Andrew Selle, Saurabh Saxena, Yutaka Leon Suematsu, Quoc Le, Alex Kurakin. Benjamin Solnik, Daniel Golovin, Greg Kochanski, John Elliot Karr
International Conference on Machine Learning, 2017.
 Neural Architecture Search
Idea: model-generating model trained via reinforcement learning

(1) Generate M neural networks

(2) Train them for a few hours
(3) Use loss of the generated models as reinforcement learning signal

Neural Architecture Search with Reinforcement Learning, Zoph & Le, ICLR 2016 arxiv.org/abs/1611.01578
Neural Architecture Search to find a model

Controller: proposes ML models Train & evaluate models

20K
times

Iterate to
find the
most
accurate
model
 AutoML outperforms handcrafted models

Inception-ResNet-v2

Accuracy (precision @1)

accuracy

computational cost
Learning Transferable Architectures for Scalable Image Recognition, Zoph et al. 2017, https://arxiv.org/abs/1707.07012

Machine Learning Day

 AutoML outperforms handcrafted models

Accuracy (precision @1)

accuracy

computational cost
Learning Transferable Architectures for Scalable Image Recognition, Zoph et al. 2017, https://arxiv.org/abs/1707.07012

Machine Learning Day

 AutoML outperforms handcrafted models

Accuracy (precision @1)

accuracy

computational cost
Learning Transferable Architectures for Scalable Image Recognition, Zoph et al. 2017, https://arxiv.org/abs/1707.07012

Machine Learning Day

 AutoML outperforms handcrafted models

Accuracy (precision @1)

accuracy

computational cost
Learning Transferable Architectures for Scalable Image Recognition, Zoph et al. 2017, https://arxiv.org/abs/1707.07012

Machine Learning Day

Let’s say I’m a
meteorologist
...

© 2018 Google LLC. All rights reserved.

I want to predict
weather trends and
flight plans from
images

© 2018 Google LLC. All rights reserved.

Can we use the
cloud to analyze
clouds?

© 2018 Google LLC. All rights reserved.

There are 10+ different types of clouds

Machine Learning Day

They all indicate different weather patterns

Machine Learning Day

Let’s try the Vision API
AutoML Vision:
Let’s see a demo
 AutoML Vision vs Vision API
Attribute AutoML Vision Vision API

Enabling developers with no ML expertise to Enabling ML practitioners to harness

Objective build state of the art ML models for Images power of Google’s ML for Images

Primary use Face detection, OCR,

Classification
case Object detection etc.

Data Just Images (may or may

Images with labelled data
requirements not required labelled data)

Output format Labels with probability As per the problem

Custom Can be used for any

Can’t be customized
requirements custom made solutions

Efforts Low for solution designing High for end to end model development

Status Beta phase; Limited processing Publically available

Machine Learning Day

 AutoML outperforms handcrafted models

Inception-ResNet-v2

Accuracy (precision @1)

accuracy

Years of effort by top ML

researchers in the world

computational cost
Learning Transferable Architectures for Scalable Image Recognition, Zoph et al. 2017, https://arxiv.org/abs/1707.07012

Machine Learning Day

 3
BigQuery
ML
Machine learning using
SQL in BigQuery
BigQuery
Managed big data analytics warehouse
Fast: terabytes in seconds

Simple: SQL

Interoperable: Java, Python, Tableau, Data Studio…

Instant data sharing

Free monthly quota - 1TB

Want to run your BQ data
through an ML model? Meet BQML

ML APIs BQML Training & Prediction

Google-trained Easy to use, End to End Platform for
models for your app Familiar SQL your data & models

App Data Engineers /

Developers Data Scientists
 Write SQL query to extract
1
training data from BigQuery

Working with Create a model,

BigQuery ML
2
specifying model type

Predict using model on data

3
extracted from BigQuery
Can we predict how much someone will
spend given some demographic data?

Occupation ID Zip Marital Years in Primary Dollar spend

Code Status current city purchase 2018
category

2 02142 Single <1 Electronics 980

5 60640 Single 4 Clothing 345

7 94103 Married 10 Kitchen 1200

Creating a model in BQML

CREATE OR REPLACE MODEL sales.predict_2019_spend

OPTIONS(model_type='linear_reg',input_label_cols=['Dollar_spend_2018']) AS
SELECT
Occupation_ID,
Zip_code,
Marital_status,
Years_current_city,
Primary_purchase_category,
Dollar_spend_2018
FROM `sara-bigquery.sales.purhcase_trends_2018`
 Getting a prediction

SELECT * FROM
ML.PREDICT(MODEL `sales.predict_2019_spend`, (
SELECT
4 Occupation_ID,
10014 Zip_code,
"Married" Marital_status,
5 Years_current_city,
"Toys" Primary_purhcase_category
FROM `sara-bigquery.sales.test_consumer_behavior` ))
 4
Cloud ML
Engine
Build, train, and serve custom
models with your own data
Cloud Machine Learning Engine

cloud.google.com/ml-engine

Fully managed Distributed Fast and scalable

platform for Training with online/batch
machine learning GPUs and TPUs prediction
 Proprietary + Confidential

Cloud Machine Learning—repeatable,

scalable, tuned
Hyper-parameter
tuning

Pre Feature Train

Inputs processing creation model
Model

Same Deploy

REST API call with

input variables Prediction
Clients Cloud MLE

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification 36
 Proprietary + Confidential

Cloud Machine Learning—repeatable,

scalable, tuned
Hyper-parameter
tuning

Pre Feature Train

Inputs processing creation model
Model

Same Deploy

REST API call with

input variables Prediction
Clients Cloud/Onprem

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification 37
 Proprietary + Confidential

Training your model with Cloud ML Engine

Use TensorFlow to create Configure

Package your and start a
computation graph and
trainer application Cloud ML
training application Engine job

Step 1 Step 2 Step 3

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
Create task.py to parse command-line parameters and Proprietary + Confidential

send along to train_and_evaluate task.py

parser.add_argument(
model.py '--train_data_paths', required=True)
parser.add_argument(
def train_and_evaluate(args): '--train_steps', ...
estimator = tf.estimator.DNNRegressor(
model_dir=args['output_dir'],
feature_columns=feature_cols,
hidden_units=args['hidden_units'])
train_spec=tf.estimator.TrainSpec(
input_fn=read_dataset(args['train_data_paths'],
batch_size=args['train_batch_size'],
mode=tf.contrib.learn.ModeKeys.TRAIN),
max_steps=args['train_steps'])
exporter = tf.estimator.LatestExporter('exporter', serving_input_fn)
eval_spec=tf.estimator.EvalSpec(...)
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
The model.py contains the ML model in Proprietary + Confidential

TensorFlow (Estimator API)

Example of the code in model.py (see previous module)

Training and CSV_COLUMNS = ...

evaluation input def read_dataset(filename, mode, batch_size=512):
functions ...

Feature columns INPUT_COLUMNS = [

tf.feature_column.numeric_column('pickuplon'),

Feature def add_more_features(feats):

engineering # will be covered in next chapter; for now, just a no-op
return feats

Serving input def serving_input_fn():

function ...
return tf.estimator.export.ServingInputReceiver(features,
feature_pholders)

Train and evaluate def train_and_evaluate(args):

loop ...
tf.estimator.train_and_evaluate(estimator, train_spec, eval_spec)
Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
 Proprietary + Confidential

Package up TensorFlow model as Python package

taxifare/
taxifare/PKG-INFO
taxifare/setup.cfg
Python packages need to
taxifare/setup.py
taxifare/trainer/ contain an __init__.py
taxifare/trainer/__init__.py in every folder
taxifare/trainer/task.py
taxifare/trainer/model.py

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
 Proprietary + Confidential

Verify that the model works as a Python package

export PYTHONPATH=${PYTHONPATH}:/somedir/taxifare
python -m trainer.task \
--train_data_paths="/somedir/datasets/*train*" \
--eval_data_paths=/somedir/datasets/*valid* \
--output_dir=/somedir/output \
--train_steps=100 --job-dir=/tmp

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
Hyperparameter tuning with HyperTune
Want to find this

Not these
Automatic hyperparameter tuning

Objective
Runs multiple trials in a training with
specified HP metrics

Gaussian process to optimize the trials

m #1
rPa ra
Hype
Hyperparameter tuning with HyperTune
hyperparameters:
goal: MAXIMIZE
maxTrials: 30
Hyperparameter types:
maxParallelTrials: 1 DOUBLE, INTEGER,
params: CATEGORICAL, DISCRETE
- parameterName: hidden1
type: INTEGER
Hyperparameter scaling:
minValue: 40 UNIT_LINEAR_SCALE,
maxValue: 400 UNIT_LOG_SCALE,
scaleType: UNIT_LINEAR_SCALE UNIT_REVERSE_LOG_SCALE
Then use the gcloud command to submit
Proprietary + Confidential

the training job, either locally or to cloud

gcloud ml-engine local train \

--module-name=trainer.task \
--package-path=/somedir/taxifare/trainer \
-- \
--train_data_paths etc.
REST as before
gcloud ml-engine jobs submit training $JOBNAME \
--region=$REGION \
--module-name=trainer.task \
--job-dir=$OUTDIR --staging-bucket=gs://$BUCKET \
--scale-tier=BASIC \
REST as before

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
 Proprietary + Confidential

Cloud ML makes deploying models and

scaling the prediction infrastructure easy

Model

Deploy

Cloud ML Engine Prediction pipeline

REST API call with
input variables
Serving Pre
Clients Model
Input Fn processing

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
We cannot reuse the training input function for serving
Proprietary + Confidential

Features Features
Data format Data format
Model Model
might be CSV might be JSON
Training Serving
input_fn input_fn

Labels Labels

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
 Proprietary + Confidential

The serving_input_fn specifies what the caller

of the predict() method will have to provide

def serving_input_fn():
feature_placeholders = {
'pickuplon' : tf.placeholder(tf.float32, [None]),
'pickuplat' : tf.placeholder(tf.float32, [None]),
'dropofflat' : tf.placeholder(tf.float32, [None]),
'dropofflon' : tf.placeholder(tf.float32, [None]),
'passengers' : tf.placeholder(tf.float32, [None]),
}
features = {
key: tf.expand_dims(tensor, -1)
for key, tensor in feature_placeholders.items()
}
return tf.estimator.export.ServingInputReceiver(features,
feature_placeholders)

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification
 Proprietary + Confidential

2. Deploy trained model to GCP

MODEL_NAME="taxifare" Could also be a locally-trained model

MODEL_VERSION="v1"
MODEL_LOCATION="gs://${BUCKET}/taxifare/smallinput/taxi_trained/export/Servo/..
./"

gcloud ml-engine models create ${MODEL_NAME} --regions $REGION

gcloud ml-engine versions create ${MODEL_VERSION} --model ${MODEL_NAME}
--origin ${MODEL_LOCATION}

Source: Lorem ipsum dolor sit amet, consectetur adipiscing elit. Duis non erat sem
Training and Certification 50
 AutoML vs CMLE (Cloud ML Engine)
Attribute AutoML CMLE

Fully deployed model with transfer Custom made models to be manually

Objective
learning/in build model feature deploy on engine

Primary use case Selected problems at scale Any problem at scale

Data requirements Specific format (based on solution) Any kind of data (can be transformed)

Output format Batch + Stream Batch + Stream

Custom
No Yes, fully
requirements

High efforts; Everything need to be

Efforts Very low efforts; No need to write any code
developed using Estimators

Status Few solutions are available (in Beta) Full available with auto scale feature

Machine Learning Day

 For machine learning, you need to build a streaming
pipeline in addition to a batch pipeline

Events, metrics, Data

etc.
Message Stream Data
scientists

queue warehouse
Raw logs, Stream Is this the Machine
files, assets, Visualization
Google processing same? Learning and BI
Analytics
data, etc.
Storage NoSQL
database
Co-workers

Batch
As this?
processing
Batch BAC
Applications
and Reports

Machine Learning Day

 Sophistication around real-time data is key

Events, metrics, Cloud

etc. Datalab
Cloud Stream
BigQuery
Pub/Sub

Raw logs, Cloud

Cloud Machine
files, assets, Visualization
Dataflow Learning and BI
Google
Analytics
data, etc. Cloud
Storage Bigtable
Batch
Co-workers

BAC
Applications
and Reports

Machine Learning Day