Table of Contents
About the Authors
About the Technical Reviewer
Acknowledgments
Introduction
Chapter 1: Introducing Cognitive Services
Understanding Cognitive Services
What Is a Cognitive Service?
Reference Architecture
Cognitive Services Suite
Examples of Cognitive Services at Work
Flood Recognition Using the Vision API
Analysis of Social Media Posts to Alert for Aggressive Behavior Using the Language API
Social Media Marketing Success Analysis Using Cognitive Services Language
Translating Written Customer Communication to Multiple Languages Using Cognitive Services Language
Translating Speech in a Public Health Call Center
Cognitive Services Pricing
Azure Cognitive Services Architecture
Lambda Architecture Overview
Embed AI Speech + Text + Vision in an Application
API References
Getting Started
Azure Notebooks
VS Code
Cognitive Services SDKs
Learning Resources
Microsoft Learn
Summary
Chapter 2: Prerequisites and Tools
Cognitive Services and Machine Learning
The AI Engineer Toolkit
Azure Data Studio
Machine Learning or Deep Learning Frameworks
Pandas
Scikit-Learn
Spark MLlib
TensorFlow
CNTK
Azure AI Gallery
Programming Languages
C#
Python
SQL
API Code Samples
Integrating Code with GitHub
GitHub Repository
Using Markdown in GitHub README.md Files
Defining Data and Configurations in YAML Files
Downloading Sample Files from GitHub
Creating APIs via Azure Portal
Calling APIs with Postman
The Postman API Client
Calling the Cognitive Services API with the Postman API Client
Find the Subscription Key and URL for Your Cognitive Services API
Pass the Message to the API Using the Postman API Client
Docker Containers
Calling the Cognitive Services API on Docker
Summary
Chapter 3: Vision
The Vision APIs
What Is Included in the Vision APIs
Reference Architecture
Computer Vision
Custom Vision
Face
Form Recognizer
Azure Video Analyzer (formerly Video Indexer)
Computer Vision Service
Capabilities
Create a Cognitive Services Resource
API Reference: Computer Vision
Custom Vision Service
Capabilities
Best Practices: Images Used for Training
Leveraging Custom Vision for Puppy Detection
API Reference: Custom Vision Training API
API Reference: Custom Vision Prediction API
Predictive Weather Monitoring: Image Classification
Leveraging Custom Vision for Weather Alerting
Best Practices: Improving Precision
Face
Capabilities
Speaker Analysis: Using Face API
Microsoft’s Data Privacy and Security
API Reference: Face
Face
FaceList and LargeFaceList
PersonGroup Person and LargePersonGroup Person
PersonGroup and LargePersonGroup
Snapshot
Form Recognizer
Capabilities
Form Recognizer in Action
API Reference: Form Recognizer
Video Analyzer
Capabilities
Demo the Video Analyzer API
Media Compliance, Privacy, and Security
Summary
Chapter 4: Language in Cognitive Services
Introducing Language APIs
Immersive Reader
Language Understanding (LUIS)
QnA Maker
Text Analytics
Translator
Acquiring and Analyzing Social Media Sentiment
What Azure Components Are Involved?
Create a Text Analytics Resource
Create an Azure Logic App
Driving Customer/User Interaction Using Azure Functions and the Language API
Azure Function Used by an Azure Logic App
Diversifying Communication with the Translator Text API
Using the Translator Text API to Translate Text
Leveraging LUIS to Parse Text for Entities and Intents
Creating a Basic LUIS App
Summary
Sentiment Analysis
Azure Function Integration
Translator Text API JSON Deserialization
LUIS Training and Components
Chapter 5: Speech Services
Speech API Overview
Applied Cases in Real Industries
Navigation
Education
Accessibility
Interacting with the Speech APIs
Text-to-Speech and Speech-to-Text
Create a Speech Service
What Is the Speech-to-Text API?
What Is the Text-to-Speech API?
Samples
Swagger
Speech SDK – Python
Register Your Voice
Speaker Verification
Speaker Identification
Security and Privacy
How to Use the Speaker Recognition API
Speech Translation SDK
Translate Speech to Text
Translate Speech to Multiple–Target Language Text
Translate Speech to Speech
Advanced Speech Knowledge
How Do Speech-to-Text and Speech Recognition Work?
The Speech FSDD Dataset (Advanced Speech Case)
Summary
References
Chapter 6: Power Platform and Cognitive Services
AI in Your BI Overview
Power BI and Cognitive Services
Sentiment Analysis
Getting Started
AI Insights in Power BI Desktop
Start with AI Insights in Power Query Online
Start with Azure Cognitive Services in Power BI
Visualizing the result
AI Visuals
Explore Data with the Q&A Visual
Identify Key Influencers
Analyze Root Cause with the Decomposition Tree
AI Builder
Start with a Prebuilt AI Model
Customize Your AI Model
Create a Prediction Model
Use the Model
Detect Objects
Summary
Chapter 7: Chatbots
Chatbot Overview
Notification Assistants
FAQ Assistants
Contextual Assistants
Personal Assistants
Autonomously Organized Assistants
Levels of Understanding Progression
Chatbot Application Use Cases
Chatbot Terms and Definitions
Domain Words
Chatbot Scope and Definition
Development Tips
Guidelines for Building a Chatbot
QnA Maker
Developing a Knowledge-Based Bot
Test and Update
Publish
Analyze
Estimated Costs for QnA Maker
Building a Simple QnA Service
QnA Maker Summary
Introduction to LUIS
Authoring Resource
Prediction Resource
Cognitive Services Multiservice Resource
LUIS Conversation Processes
LUIS Creation Tutorial
Using Chatbots with QnA Maker and LUIS
Choosing Between QnA Maker and LUIS
Summary
References
Chapter 8: Ethics in AI
Why Is Ethics in AI an Issue?
Ethical AI Issues
Governmental Regulation and AI Analysis
Bias in AI
Fairness vs. Accuracy
Skewed Samples Compound Issues
Analyzing the Data for Ethical Issues
Evaluating Training Data
Omitted Variable Bias
Correlation and Causation
Cognitive Bias
Tools for Avoiding Data Bias
Fairlearn
Fairlearn Dashboards
Mitigation
InterpretML
Interpret-Community
Differential Privacy
Deploying Differential Privacy
Differential Privacy Platform
Summary
Index