Introduction to

watsonx.ai

Content by:
Ahmad Muzaffar Baharudin
APAC Ecosystem Technical Enablement Specialist | Data & AI
ahmad.muzaffar@ibm.com

Instructor:
Farah Auni Hisham
Technical Enablement Specialist | Data & AI
farah.hisham@ibm.com

APAC Ecosystem Technical Enablement | Data & AI 1

Seller Slides in this presentation marked as
"IBM and Business Partner Internal Use
Only" are for IBM and Business Partner
References in this presentation to IBM
products, programs, or services do not
imply that they will be available in all

guidance use and should not be shared with clients

or anyone else outside of IBM or the
Business Partners’ company.
countries in which IBM operates. Product
release dates and/or capabilities
referenced in this presentation may

and legal
change at any time at IBM’s sole
© IBM Corporation 2024. discretion based on market opportunities
All Rights Reserved. or other factors and are not intended to be

disclaimer
a commitment to future product or feature
The information contained in this availability in any way. Nothing contained
publication is provided for informational in these materials is intended to, nor shall
purposes only. While efforts were made have the effect of, stating or implying that
to verify the completeness and accuracy any activities undertaken by you will result
of the information contained in this in any specific sales, revenue growth, or
IBM and Business Partner publication, it is provided AS IS without other results.
warranty of any kind, express or implied.
Internal Use Only In addition, this information is based on All client examples described are
IBM’s current product plans and strategy, presented as illustrations of how those
which are subject to change by IBM clients have used IBM products and the
without notice. IBM shall not be results, they
responsible for may have achieved. Actual environmental
any damages arising out of the use of, or costs and performance characteristics
otherwise related to, this publication or may vary by client.
any other materials. Nothing contained in
this publication is intended to, nor shall
have the effect of, creating any warranties
or representations from IBM or its
suppliers or licensors, or altering the
terms and conditions of the applicable
license agreement governing the use of
IBM software.

APAC Ecosystem Technical Enablement | Data & AI 2

Agenda 1 Generative AI

2 Foundation Model

3 Large Language Models (LLM)

4 Introduction to watsonx.ai

5 Hands-on: Exploring watsonx.ai UI

APAC Ecosystem Technical Enablement | Data & AI 3

1 Generative AI

▪ Artificial Intelligence (AI) at a Glance

▪ What is Generative AI

▪ Traditional AI vs. Generative AI

▪ Enterprise Use Cases of Generative AI

▪ Industry-Specific Enterprise Use Case of

Generative AI

APAC Ecosystem Technical Enablement | Data & AI 4

Artificial Intelligence (AI) AI can be defined as a technique that enables machines to mimic
cognitive functions associated with human minds – cognitive functions
Human intelligence exhibited by machines include all aspects of learning, reasoning, perceiving, and problem solving.

Machine Learning (ML) ML-based systems are trained on historical data to uncover patterns.
Users provide inputs to the ML system, which then applies these inputs
Systems that learn from historical data
to the discovered patterns and generates corresponding outputs.

Deep Learning (DL) DL is a subset of ML, using multiple layers of neural networks, which are
interconnected nodes, which work together to process information. DL is
ML technique that mimics
well suited to complex applications, like image and speech recognition.
human brain function

Foundation Model
Generative AI systems

AI model built using a specific kind of neural network architecture,

called a transformer, which is designed to generate sequences
of related data elements (for example, like a sentence).

5
1950’s 1980’s 2010’s 2020’s
What is Generative AI

▪ Generative AI refers to AI model that can generate high-quality text, images, audio, video, codes and other content
based on the data they were trained on.

Input (Prompt) Generated Output (Response)

Prompting Response
Generative AI Model

APAC Ecosystem Technical Enablement | Data & AI 6

What is Generative AI

▪ In training phase, Generative AI models are trained on large amount datasets to understand patterns and relationships
between words, pixels, or notes, resulting in the creation of a statistical model.

Training Phase

Pre-training Data

Pre-training
Generative AI Model

Massive unlabeled external data

APAC Ecosystem Technical Enablement | Data & AI 7

What is Generative AI

▪ During usage phase, when the model is given a prompt, it will predict what the expected response might be - and this
will create new content.

Usage Phase

Input (Prompt) Generated Output (Response)

Prompting Response
Pre-trained
Generative AI Model

APAC Ecosystem Technical Enablement | Data & AI 8

Traditional AI vs. Generative AI

Traditional AI Generative AI
Training Data Task Training Data Task

Model A Classification Summarization

Model B Classification Extraction

Model X
Model C Classification Generation

Model D Classification Coding

Labeled Data Predictive Tasks Massive unlabeled Data Generative Tasks

▪ A Traditional AI model is trained using Machine Learning (ML) • A Foundation Model can address multiple tasks.
for specific task. • Self-Supervised Learning with very large datasets.
▪ Supervised Learning with proper & large datasets. • Rapid adaptation to multiple tasks with additional small amounts
▪ Very efficient for specific task it was trained for. of task-specific data.

APAC Ecosystem Technical Enablement | Data & AI 9

Enterprise Use Cases of Generative AI

Summarization Extraction Generation Classification

▪ Transform text with domain- ▪ Identify and extract essential ▪ Generate content for a ▪ Read and classify written
specific content into information from text. specific purpose. input.
personalized overviews with
captured key points. ▪ Example: Audit support, SEC ▪ Content creation, Email ▪ Example: Sorting customer
10K fact extraction, Name drafting support, Product complaints, Sentiment
▪ Example: Financial reports, Entity Extraction descriptions analysis, Threat &
Meeting transcript vulnerability classification
summarization

Question answering Coding Conversation Language Translation

▪ Provide answers to specific ▪ Math reasoning ▪ Engage in natural-sounding ▪ Translate text from one
questions based on the conversations with users. language to another.
content of the text. ▪ Example: Code generation,
code translation ▪ Example: Chatbot ▪ Example: Translate a website
▪ Example: FAQs, Finance Q&A conversation from English to Spanish

APAC Ecosystem Technical Enablement | Data & AI 10

Industry-Specific Enterprise Use Case of Generative AI
Customer Care Digital Labor IT Operations Cybersecurity
Summarization • Call center transcripts • Summarize documents, contracts, • Summarize alerts, technical logs, • Summarize security event logs
• Omnichannel journey summary technical manuals, reports, etc. tickets, incident reports, etc. • Summarize steps to
Summarizing large documents, • Summarizing search snippets • Transcribe videos to • Summarize policy, procedure, recap security incident
conversations, and recordings to augment chatbots text and summarize meeting notes, etc. • Summarize security specs
to key takeaways • Summarize events, analyst reports, • Summarizing reports on Form 10K • Vendor report QBR summarization
financial info etc. for advisor
• Sentiment analysis
Extraction • Extracting interaction • Extract answers and data from • Extract key information from various • Extract information from incidents,
history with clients complex unstructured documents sources for report automation content for security awareness
Extract structured insights • Extract information from specific • Extract information from media • Extract relevant system/network • Extract key security markers and
from unstructured data types/categories of incidents files such as meeting records, information for administration, attributes from new threat reports.
audio, and video maintenance, and support purpose

Generation • User stories, personas • Automate the creation of marketing • Create technical document from code • Automate report generation
• Create personalized UX code material and language translation • Automate scripts to configure, • Social engineering simulation
Generate AI to create text from experience design • Automate image, text, and video deploy, and manage hybrid cloud • Security documentation creation
• Training, and testing data for chatbots creation for articles, blogs, etc. • Co-pilot to create code across • Automate threat detection by
• Automate responses to • Create automation scripts for various multiple programming languages looking for anomaly patterns
emails and reviews workflows across applications
Classification • Classify customer sentiments from • Classify documents by different • Classify incident reports • Classify flagged items properly as
feedback or chatbot interaction criteria – types, contents, keywords • Automate workflow based on threats or other categories
For sentiment or topics • Classify typical issues raised by • Sort digital contents in storage analysis of items/status/reports • Classify the type of security risks
clients for focused improvements into pre-defined categories and find the best response
• Classify log and other monitoring
output to determine the next action
Question answering • Knowledgebase articles • Analyze emails, attachments, • Knowledge search for IT helpdesk • Knowledge search across
• Augment chatbot w/search documents, invoices, reports, etc. • Ticket resolution by suggesting security spec documents
Knowledge base search across • Agent assist • Knowledge search for company solutions from resolved tickets • External threat intelligence
the company’s proprietary data. • Contract intelligence information to provide in-house • Error log and root cause analysis • Error log and root cause analysis
• Mart search in technical manuals, day-to-day assistance • Compliance monitoring • Security incident search @ forensics
HR documents, ethics codes, and automation
product documentation, etc.
APAC Ecosystem Technical Enablement | Data & AI 11
2 Foundation
Models

▪ What are Foundation Models

▪ Foundation Models are a paradigm shift for AI

APAC Ecosystem Technical Enablement | Data & AI 12

What are Foundation Models

▪ Foundation Models (FM) are general purpose AI models that are pre-trained on large amounts of text or other data and
have billions of parameters.
▪ FM are trained on a broad set of unlabeled data that can be used for different tasks, with minimal fine-tuning.
▪ FM are capable to do various tasks, including text, code, or image generation, classification, conversation, etc.
▪ FM can be further fine-tuned to complete different types of tasks or domains.

Pre-training Data Prompting Task

Summarization

Extraction

Pre-training Response Generation

Foundation Models
Classification

Question Answering

Coding

Conversation
Prompt Tuning
Tuning
Fine Tuning
Fine-tuned Question Answering
Model’s
Enterprise …
Response
Proprietary Data
APAC Ecosystem Technical Enablement | Data & AI … 13
Foundation
Models are a Physical Asset Business
Management Automation
paradigm shift Sensor/IoT
Data
Business
Process Data

for AI
IT Ops & Threat
IT Automation IT Cybersecurity Management
Unlock business advantage
Data Data
with Foundation Models
trained across the breadth of Foundation
enterprise data.
Models

Sustainability
Molecular Geospatial
Data Data

Natural Code
Language

Customer Care App Modernization

Digital Labor IT Automation
APAC Ecosystem Technical Enablement | Data & AI 14
Geospatial
Foundation Physical Asset Business
Management Automation
Models Sensor/IoT
Data
Business
Process Data

IT Ops & Threat

IT Automation IT Cybersecurity Management
NASA and IBM have teamed up
Data Data
to create an AI Foundation
Model for Earth Observations, Foundation
using large-scale satellite and
remote sensing data, including
Models
the Harmonized Landsat and
Sentinel-2 (HLS) data. Sustainability
Molecular Geospatial
Data Data

Natural Code
Language

Customer Care App Modernization

Digital Labor IT Automation
APAC Ecosystem Technical Enablement | Data & AI 15
IBM Partnership
with
NASA

Sriram Raghavan, VP of IBM

Research AI and
NASA IMPACT project lead,
Rahul Ramachandran
presented about IBM
Geospatial Foundation Model
during IBM THINK 2023.

Read more here

APAC Ecosystem Technical Enablement | Data & AI 16

IBM-NASA
Geospatial
Foundation
Models
Prithvi is a first-of-its-kind
temporal Vision transformer
pre-trained by the IBM and
NASA team on contiguous US
Harmonised Landsat Sentinel 2
(HLS) data.

The Geospatial Foundation

Model is called Prithvi-100M.
Learn more about Prithvi Model
here.
Watch the demo here.

APAC Ecosystem Technical Enablement | Data & AI 17

Large Language
Model (LLM) Physical Asset Business
Management Sensor/IoT Business Automation
Data Process Data

IT Ops & Threat

IT Automation IT Cybersecurity Management
In the present landscape,
Data Data
numerous LLM Foundation
Models are developed, Foundation
including those from IBM,
third-party sources, and open-
Models
source models.
Sustainability
Molecular Geospatial
Data Data

Natural Code
Language

Customer Care App Modernization

Digital Labor IT Automation
APAC Ecosystem Technical Enablement | Data & AI 18
IBM’s Granite
Foundation
Models

IBM Research has introduced

the Granite series of decoder-
only foundation models for
generative AI tasks that are
ready for enterprise use.

Learn more about Granite

Model here.
Read paper about Granite
Foundation Model by IBM
Research here.

APAC Ecosystem Technical Enablement | Data & AI 19

IBM’s Granite
Foundation
Models

IBM Research has introduced

the Granite series of decoder-
only foundation models for
generative AI tasks that are
ready for enterprise use.

Learn more about Granite

Model here.
Read paper about Granite
Foundation Model by IBM
Research here.

APAC Ecosystem Technical Enablement | Data & AI 20

3 Large
Language
Model (LLM)
▪ What is Large Language Models (LLM)

▪ Spectrum: LLM Deployment Strategies for

Enterprises

APAC Ecosystem Technical Enablement | Data & AI 21

What are Large Language Models (LLM)

▪ Large language models (LLM) is a category of Foundation Models pre-trained on massive amounts of data making them
capable of understanding and generating natural language such as text and other types of content.
▪ LLM can perform a wide range of tasks such as summarization, extraction, text generation, classification, question
answering, code generation, translation, etc.

LLM = Data

+
Architecture

+
Training

APAC Ecosystem Technical Enablement | Data & AI 22

LLM: Data

▪ LLM is trained on very large amount of data in term of data (file) size and token size.
▪ During training, an LLM will have billions of parameters.
– If the LLM has 70B parameters, it means that the model has 70 Billion adjustable parameters.
– These parameters are used to learn the relationship between tokens in the training data.
– The more parameters a model has, the more complex it can be and the more data it can process.

• Data size: ~Terabyte

• Token size: Trained on ~Trillion

LLM = Data
tokens of text data

+ • Parameter size: ~Billion

Architecture

+
Training

APAC Ecosystem Technical Enablement | Data & AI 23

LLM: Architecture

▪ The LLM architecture is based on the transformer, Encoder-only Decoder-only Encoder-Decoder

which allows models to process sequences of data Output Output Output
Probabilities Probabilities Probabilities
and understand the context of each word in a sentence
in relation to others.
▪ 3 types of transformer architectures used in LLM:
– Encoder-only (Autoencoding)
– Decoder-only (Autoregressive)
– Encoder-Decoder (Sequence-to-Sequence)

Decoder

Decoder

Decoder
=

Encoder
Encoder

Encoder
LLM Data

+
Architecture

+ Input Output Input Output Input Output

(Shifted Right) (Shifted Right) (Shifted Right)
Training Example: BERT, RoBERTa Example: GPT, BLOOM Example: T5, BART
Use: Sentiment analysis, NER, Use: Translation, Text Use: Translation, Text
Word classification Summarization Summarization

APAC Ecosystem Technical Enablement | Data & AI 24

LLM: Training

▪ The LLM architecture are trained using the data.

▪ During Pre-training, the model is trained on a large corpus of text using self-supervised learning objectives, such as
predicting the next word in a sequence (language modeling) or predicting masked tokens in a sequence (masked
language modeling).
▪ During Fine Tuning, the pre-trained model is further fine-tuned on a smaller dataset related to a specific downstream
task (e.g., text classification, question answering, language translation).

Pre-Training Fine Tuning

LLM = Data Large

Unlabeled
Small
Labeled
Corpus Corpus
+ Unlabeled Pre-training

Architecture Pre-trained LLM Pre-trained LLM

+ Fine Tuning

Training Fine-Tuned LLM

APAC Ecosystem Technical Enablement | Data & AI 25

Spectrum: LLM Deployment Strategies for Enterprises

▪ When deploying LLM for enterprise, there is a spectrum of strategies that enterprises can consider based on metrics
customized to their specific needs or conditions such as:
▪ Accuracy: How accurate are the responses?
▪ Deployment Complexity: How intricate is the deployment process?
▪ Effort: How much effort is needed for implementation?
▪ Total Cost of Ownership (TCO): What is the overall cost of owning the solution?
▪ Flexibility of updating: How flexible is the architecture for updates and changes? How straightforward is it to replace or upgrade
components?

1 2 3 4 5

Retrieval Augmented Parameter Efficient Fine Tuning

Prompt Engineering Model Fine Tuning Model Creation
Generation (RAG) (PEFT)

• Simple Prompting (Zero-shot • Prompting with data • Prompt tune a base model to perform • Fine tune a base model for certain non- • Create a domain or use case specific
prompt), Prompting with Contexts retrieved from Vector specific tasks generative tasks model
(n-shot prompt). Embeddings. – Prompt Tuning
• Model parameter adjustment. – LoRA

Simple Complex Specialized Highly Specialized

APAC Ecosystem Technical Enablement | Data & AI 26

4 Introduction
to watsonx.ai

▪ What is watsonx.ai

▪ watsonx.ai Key Capabilities

– Generative AI Capabilities
– MLOps Capabilities

APAC Ecosystem Technical Enablement | Data & AI 27

 watsonx
watsonx.ai watsonx.data watsonx.governance
Build, train, validate, tune and Scale AI workloads, for all Accelerate responsible,
deploy AI models your data, anywhere transparent and explainable AI
workflows

The platform
for AI and data A next generation enterprise
studio for AI builders to build,
Fit-for-purpose data store, built on
an open lakehouse architecture,
End-to-end toolkit for AI
governance across the entire model
train, validate, tune, and deploy supported by querying, governance lifecycle to accelerate responsible,
both traditional machine and open data formats to access transparent, and explainable AI
learning and new generative AI and share data. workflows
Scale and capabilities powered by
accelerate the foundation models. It enables
you to build AI applications in a
impact of AI with fraction of the time with a
fraction of the data.
trusted data

APAC Ecosystem Technical Enablement | Data & AI 28

What IBM offers
Enable fine-tuned models to be
managed through market leading
watsonx.ai: An enterprise governance and lifecycle
management capabilities
studio for AI builders Leverage foundation
models to automate data
search, discovery, and
linking in watsonx.data
watsonx.governance

watsonx.ai

watsonx.data

Leverage governed enterprise

data in watsonx.data to
seamlessly train or fine-tune
foundation models

Train

watsonx.data watsonx.ai watsonx.governance

Validate
Scale a workloads, Train, validate, tune Enable responsible,
for all your data, and deploy AI transparent and
Tune anywhere models explainable AI workflows

Deploy

APAC Ecosystem Technical Enablement | Data & AI 29

watsonx.ai Key Capabilities

Generative AI ML Ops
Capabilities Capabilities

Prompt Lab Tuning Studio Foundation Model ModelOps Automated Decision

Libraries Development Optimization

APAC Ecosystem Technical Enablement | Data & AI 30

watsonx.ai Capabilities

Generative AI ML Ops
Capabilities Capabilities

Prompt Lab Tuning Studio Foundation Model ModelOps Automated Decision

Libraries Development Optimization

APAC Ecosystem Technical Enablement | Data & AI 31

Prompt Lab
Experiment with Foundation Models and build prompts

Interactive prompt Experiment with prompt

builder engineering

▪ Includes prompt ▪ Choice of Foundation

examples for various use Models to use based on task
cases and tasks requirements

▪ Experiment with different ▪ Prevent the model from

prompts, save and reuse generating repeating
older prompts, use phrases
different models and vary
different parameters
▪ Number of min and max new
tokens in the response
▪ Experiment with zero-
shot, one-shot, or few-
▪ Stop sequences – specifies
shot prompting to get the
sequences whose
best results
appearances should stop
the model

APAC Ecosystem Technical Enablement | Data & AI 32

Tuning Studio
Tune your Foundation Models with labeled data

Prompt tuning Task support in the

Tuning Studio

▪ Efficient, low-cost way of ▪ Models support a range of

adapting an AI foundation Language Tasks:
model to new Summarize, Classify,
downstream tasks Generate, Extract, Q&A

▪ Tune the prompts with no ▪ Requires a small set of

changes to the underlying labelled data to perform
base model or weights specialized tasks

▪ Unlike prompt ▪ Can achieve close to fine-

engineering, prompt tuning results without model
tuning allows clients to
modification, at a lower cost
further train the model
to run
with focused, business
data

APAC Ecosystem Technical Enablement | Data & AI 33

Foundation Models Libraries
Choose your preferred Foundation Model

▪ Foundation Models libraries includes IBM’s proprietary models, Third-Party models and Open-Source models
from Hugging Face.
▪ You can find the latest list of supported FM in watsonx.ai here. See the deprecated FM updates here.

*Foundation Models list as of March 2024

APAC Ecosystem Technical Enablement | Data & AI 34
Foundation Models Variety
Cover various enterprise use cases & compliance requirements

▪ IBM watsonx.ai is a multi-model platform

– One model doesn’t fit all use cases. We offer IBM-developed, open-source, third-party, and Bring Your Own
Models (BYOM).
– Bigger model is not always better; specialized models can outperform general-purpose models with lower
infrastructure requirements.

Model generating code from a natural language

Language Q&A Model responds to a question in natural language Coding Code Gen
prompt
Tasks Tasks
Generate Model generates content in natural language

Extract Model extracts entities, facts, and info. from text

Summarize Model creates summaries of natural language Model can be prompt tuned
Prompt Prompt tune
Tuning *Available in Tuning Studio
Classify Model classifies text (e.g., sentiment, group)
Tasks

APAC Ecosystem Technical Enablement | Data & AI 35

Foundation Models in watsonx.ai

Foundation Models

IBM Open Source &

Foundation Models Third-Party Models
Ensure model trust and efficiency for Experiment with open-source models
specialized use cases with lower cost. and let clients define the best models
for their business needs.

Prompting

Bring Your Own Models

(BYOM)
Give more flexibility to pre-train
client’s own models by harnessing
their own proprietary data.

*BYOM capability availability:

1H 2024

APAC Ecosystem Technical Enablement | Data & AI 36

 Open Source &
IBM Foundation Models
Third-Party Foundation Models
Pro Pro
▪ Large collection of innovative models.
Data & Model ▪ Various vendors provide different models with many built for special
▪ Models trained on the largest trusted, enterprise-grade Data Lake - use cases (Image, video, extraction, code, etc.).
ready to solve enterprise use case.
▪ Built on highly curated, filtered data, removing Duplication, Copyright,
licensed material, HAP content.
▪ Compute-optimal model training and architectures.
Cons
▪ Smaller in size (so far) - and is less costly to run.
▪ May have issues with quality of training data (Copyright, licensed, HAP
contents).
Use cases
▪ May have issues with data privacy and security.
▪ Efficient Domain and Task Specialization (Finance, Cybersecurity,
Legal, Language, Geospatial, IT automation, Code, etc.).
▪ Focus on specific use cases (Ansible, code translation, and more).

Trustworthy
▪ The same IP protections for IBM software are applied to this LLM.
▪ Easier to govern.
▪ Full, auditable data lineage available for any IBM Model.

APAC Ecosystem Technical Enablement | Data & AI 37

IBM Foundation Models
Granite Slate Sandstone Obsidian
▪ Decoder-only LLM. ▪ Encoder-only LLM. ▪ Encoder-Decoder LLM. ▪ New modular architecture by IBM.
▪ Generative model, GPT-like ▪ Non-generative model, but fast & ▪ Well-suited for fine-tuning on specific ▪ High inference efficiency and levels of
architecture for generative tasks. effective for enterprise NLP tasks. tasks. performance across a variety of tasks.

granite-13b-instruct slate-153m
13 billion params 153 million params
Extract Extract

Summarize Classify

Classify
*Fine-tuned for entity extraction,
relationship detection, sentiment Coming Soon Coming Soon
analysis
granite-13b-chat
13 billion params

Q&A

Generate

Read more about IBM Foundation Model here

APAC Ecosystem Technical Enablement | Data & AI 38

Why IBM Granite ? Trusted, Performant, Cost-effective AI foundation models built for enterprises.

– Granite models were built for

enterprise-use, that is governed
by rules and safeguards, 27.19TB of extracted data
e.g., rooting out HAP and
copyright content.

– The models are trained on 7.36TB of deduplicated data

datasets that meet
rigorous data governance and
GRC criteria e.g., due diligence
and document quality. 7.35TB of usable data

– Chat fine-tuning techniques

were designed to prevent
hallucinations and 7.48TB of data for tokenization
misalignments in the model
outputs for Granite.13b.chat.

– IBM stands behind its Granite

models with industry leading IP IBM Granite 13b model was trained IBM Granite models support leading industry use cases and
indemnification. on 2.5 Trillion tokens of enterprise compatible across major languages.
relevant content devoid
of objectionable content.

APAC Ecosystem Technical Enablement | Data & AI 39

IBM Granite Trusted Performant Cost-effective

Financial acumen Lower Cost of inferencing

Trusted, Performant, Average of 11 financial tasks
Cost-effective AI foundation $0.0018/1K tokens
0.59
models built for enterprises.
0.57
Lower Infrastructure hosting

Can run on just 1 GPU

2.5T Tokens Response time

Lower carbon footprint

3x faster
Meet sustainability goals

Internet Code 70—chat model

Enterprise Academic Granite.13b.chat

Trained in accordance with IBM's AI ethics Performs on par with much larger 70b-chat Granite models are competitively priced in
code and principles with transparency in model especially in financial tasks, at faster the market with an almost 3X price-cut.
data curation and processing. response rates.

APAC Ecosystem Technical Enablement | Data & AI 40

Granite models are targeted at specific Model Performance on Finance Tasks
(average of 11 Financial tasks)
industry domains like finance and legal
Based on IBM's internal development tests Granite
demonstrates superior acumen across 11 financial tasks

IBM Research Finance Panel Entity Extraction

(Financial acumen measured – Edgar SEC Filings (x2)
across 11 tasks) – Financial Credit Risk
Assessment
Sentiment analysis Financial

Financial Acumen
Phrase Bank Question and Answering
– Stock and Earnings Call – Financial Question
Transcripts Answering (x2)
– Financial Question – Insurance
Answering
Summarization
Classification – Corporate Event Detection
– News Headline
Classification

Size of model (billions of parameters)

Source: Granite Foundation Model Report, IBM Research

APAC Ecosystem Technical Enablement | Data & AI 41

IBM Partnership
with
Hugging Face

Darío Gil, SVP and Director of

IBM Research, and
Hugging Face CEO Clem
Delangue, announced
collaboration during IBM
THINK 2023.

Watch more here

Read more here

APAC Ecosystem Technical Enablement | Data & AI 42

Open Source & Third-Party Foundation Models
Open-Source LLM Third-Party LLM
▪ Encoder-Decoder & Decoder-only Large Language Models available in Prompt lab. ▪ All third-party models, except gpt-neox-20b and
starcoder-15.5b are instruction-tuned.
▪ Open-source models are sourced from Hugging Face.
▪ IBM indemnification does not apply to any third-
▪ Fine tuning NOT required for most tasks. party models.

flan-ul2 gpt-neox mt0-xxl flan-t5-xxl mpt-instruct2 llama2-chat elyza.japanese

20 billion 20 billion 13 billion 11 billion 7 billion 70/13 billion 7 billion
Encoder-Decoder Decoder-only Encoder-Decoder Encoder-Decoder Decoder-only Decoder-only Decoder-only

Q&A Q&A Q&A Q&A Q&A Q&A Translation

Generate Generate Generate Generate Generate Generate Q&A

Extract Extract Summarize Extract Generate

Summarize Summarize Classify Summarize Extract

Classify Classify Classify Summarize

flan-t5-xl-3b starcoder-15.5b Classify43

3 billion 15.5 billion
Encoder-Decoder Decoder-only

Prompt tune Code Gen

APAC Ecosystem Technical Enablement | Data & AI

watsonx.ai Capabilities

Generative AI MLOps
Capabilities Capabilities

Prompt Lab Tuning Studio Foundation Model ModelOps Automated Decision

Libraries Development Optimization

APAC Ecosystem Technical Enablement | Data & AI 44

ModelOps

▪ ModelOps covers the end-to-end lifecycles for optimizing the use of models and applications, targeting Machine
Learning models, optimization models and other operational models, from development to deployment stage.
▪ Essential for effectively managing and operationalizing AI models at scale, ensuring compliance, fostering
collaboration, and driving continuous improvement in AI initiatives within enterprises.

APAC Ecosystem Technical Enablement | Data & AI 45

Automated Development

▪ AutoAI automates data preparation, model development, feature engineering and hyperparameter optimization.
▪ With AutoAI, beginners can quickly get started and expert data scientists can speed experimentation in AI
development.

APAC Ecosystem Technical Enablement | Data & AI 46

Decision Optimization

▪ Decision optimization streamlines the selection and deployment of optimization models and enables the creation of
dashboards to share results and enhance collaboration.

APAC Ecosystem Technical Enablement | Data & AI 47

5 Hands-on:
Exploring
watsonx.ai
▪ Lab Environment Reservation

▪ Lab Environment Access

▪ Exploring Generative AI Capabilities

– Prompt Lab
– Tuning Studio
– Foundation Model Libraries

APAC Ecosystem Technical Enablement | Data & AI 48

Lab Environment Reservation

▪ You can reserve your own watsonx.ai TechZone Lab anytime and use it for self-practice or client demo.
▪ Go to TechZone Certified Base Images (watsonx.ai/.governance SaaS) and click Reserve.

APAC Ecosystem Technical Enablement | Data & AI 49

Lab Environment Reservation

▪ Select Schedule for later for the reservation option

APAC Ecosystem Technical Enablement | Data & AI 50

Lab Environment Reservation

Creating Reservation
▪ Select Purpose as Practice / Self-Education, describe your purpose, select Preferred Geography and End date and time.

APAC Ecosystem Technical Enablement | Data & AI 51

Lab Environment Reservation

Creating Reservation
▪ Select Preferred Geography as AMERICAS Dallas Region (any ITZ account is fine). Select Install DB2? as Yes.
▪ Click agree and hit Submit, wait for the system auto-approval, which usually takes about 10 minutes (Up to 2 hours).

APAC Ecosystem Technical Enablement | Data & AI 52

Lab Environment Reservation

Extending Reservation
▪ A reservation allows lab access up to 3 days. However, you may extend the reservation a few times.
▪ To extend your lab access, go to My reservations and click 3 dots icon, then click Extend.

APAC Ecosystem Technical Enablement | Data & AI 53

Lab Environment Access

▪ You will receive an email notifying that your reservation is Ready.

▪ You can also go to TechZone, under My reservations and confirm that your reservation Status has changed to Ready.

APAC Ecosystem Technical Enablement | Data & AI 54

Lab Environment Access

▪ You will receive an account invitation email from IBM Cloud. Click the link Join now.
▪ Fill up the account details and accept the product Terms and Conditions.

APAC Ecosystem Technical Enablement | Data & AI 55

Lab Environment Access

▪ After successfully joining the invitation, you will have access to the invited IBM Cloud account created by TechZone.
▪ Note that the account name contains the itz-watsonx keyword.

APAC Ecosystem Technical Enablement | Data & AI 56

Lab Environment Access

▪ From IBM Cloud dashboard, we will navigate to watsonx product page.

▪ In the search bar, search the keyword watsonx and select watsonx.

APAC Ecosystem Technical Enablement | Data & AI 57

Lab Environment Access

▪ Navigate to watsonx.ai by selecting watsonx.ai and proceed by clicking Get started button.

APAC Ecosystem Technical Enablement | Data & AI 58

Lab Environment Access

▪ Select Dallas as the region and click Create account or log in button.

APAC Ecosystem Technical Enablement | Data & AI 59

Lab Environment Access

▪ If you are prompted this notification page, click on here.

APAC Ecosystem Technical Enablement | Data & AI 60

Lab Environment Access

▪ Provide the required information and click Continue.

APAC Ecosystem Technical Enablement | Data & AI 61

Lab Environment Access

▪ Congratulation! Now, you are already inside watsonx.ai dashboard.

▪ To create a new project, click on Create a sandbox project button.

APAC Ecosystem Technical Enablement | Data & AI 62

Lab Environment Access

▪ The project creation may take a few seconds to complete.

▪ Refer to the progress bar below the project’s tile.

APAC Ecosystem Technical Enablement | Data & AI 63

Lab Environment Access

▪ Once the project successfully created, you will see your watsonx.ai homepage as follow.
▪ The key features in the homepage are: Prompt Lab, Tuning Studio, Jupyter Notebook Editor, Projects and Resource hub

APAC Ecosystem Technical Enablement | Data & AI 64

Prompt Lab

Structured Mode
▪ Structured mode helps users create effective prompts. Text from the fields is sent to the model in a template format.

Instruction
Add an instruction if it makes sense for your use
case. An instruction is an imperative statement,
such as: Summarize the following article.

Examples
Add one or more pairs of examples that contain
the input and the corresponding output that you
want.

Test your input

In the Try area, enter the final input of your
prompt.

APAC Ecosystem Technical Enablement | Data & AI 65

Prompt Lab

Freeform Mode
▪ In freeform mode you can add your prompt in plain text. Your prompt text is sent to the model exactly as you typed it.

APAC Ecosystem Technical Enablement | Data & AI 66

Prompt Lab

Chat Mode
▪ In chat mode, you can input your prompt in chat style UI.

APAC Ecosystem Technical Enablement | Data & AI 67

Prompt Lab

Sample prompts
▪ You can try the sample prompts provided for Generative AI use cases such as:
▪ Summarization, Classification, Generation, Extraction, Question answering, and Code.

APAC Ecosystem Technical Enablement | Data & AI 68

Prompt Lab

Saving prompts
▪ You can save your prompts for more actions in your workflow.
▪ Prompts can be saved in 3 forms: Prompt template, prompt session or Notebook.

APAC Ecosystem Technical Enablement | Data & AI 69

 Prompt Lab

Saved prompts
▪ You can check your saved prompt template and prompt session from the left panel.

Saved Prompt Session

Saved Prompt Template

APAC Ecosystem Technical Enablement | Data & AI 70

Prompt Lab

Saved prompts
▪ For prompt saved as a Notebook, you can find the notebook as an asset under your Project.

APAC Ecosystem Technical Enablement | Data & AI 71

Prompt Lab

History
▪ You can view the summaries of your recent prompt entries in History tab.
▪ If you save your work as a prompt session, you retain the history. Otherwise, the history will be discarded.
▪ You can restore any of the past session as your current session by simply clicking Restore.

APAC Ecosystem Technical Enablement | Data & AI 72

Prompt Lab

Model Parameters
▪ You can control some settings and techniques to influence the output during model inferencing in LLMs.
▪ These settings are specific to the model inferencing phase and do not affect the model’s training.
Decoding Method
Choose greedy or sampling

Temperature
Range from 0.0 to 2.0

Top P
Range from 0.0 to 1.0

Top K
Range from 1 to 100

Random Seed
Range from 1 to 4,294,967,295

Repetition penalty
Value between 1 or 2

Stopping criteria
Add condition to stopping criteria

Min and Max Token

Any integer number

APAC Ecosystem Technical Enablement | Data & AI 73

Foundation Model Libraries

▪ You may find the list of all available Foundation Models under Resource hub.
▪ By clicking the tile, you can find the details of each model.

APAC Ecosystem Technical Enablement | Data & AI 74

 © 2024 International Business Machines Corporation

Thank you
IBM and the IBM logo are trademarks of IBM
Corporation, registered in many jurisdictions
worldwide. Other product and service names might be
trademarks of IBM or other companies. A current list
of IBM trademarks is available on ibm.com/trademark.

THIS DOCUMENT IS DISTRIBUTED “AS IS” WITHOUT

ANY WARRANTY, EITHER EXPRESS OR IMPLIED. IN
NO EVENT, SHALL IBM BE LIABLE FOR ANY DAMAGE
ARISING FROM THE USE OF THIS INFORMATION,
INCLUDING BUT NOT LIMITED TO, LOSS OF DATA,
BUSINESS INTERRUPTION, LOSS OF PROFIT OR
LOSS OF OPPORTUNITY.

Client examples are presented as illustrations of how

those clients have used IBM products and the results
they may have achieved. Actual performance, cost,
savings or other results in other operating
environments may vary.

Not all offerings are available in every country in which

IBM operates.

IBM’s statements regarding its plans, directions, and

intent are subject to change or withdrawal without
notice at IBM’s sole discretion. Information regarding
potential future products is intended to outline our
general product direction and it should not be relied
on in making a purchasing decision. The information
mentioned regarding potential future products is not
a commitment, promise, or legal obligation to deliver
any material, code or functionality. Information about
potential future products may not be incorporated into
any contract. The development, release, and timing of
any future features or functionality described for our
products remains at our sole discretion.

Red Hat and OpenShift are registered trademarks of

Red Hat, Inc. or its subsidiaries in the United States
and other countries.

APAC Ecosystem Technical Enablement | Data & AI 75

Accessing Pre-reserved Lab

▪ Access the pre-reserved watsonx.ai/watsonx.governance lab:

▪ https://techzone.ibm.com/my/workshops/student/661c039b8d5827001e24afdb
▪ Log in TechZone using your IBM ID.
▪ Enter the following Password & click Submit:
Password: watsonx

APAC Ecosystem Technical Enablement | Data & AI 77