Learn Grafana 10.x: A beginner's guide to practical data analytics, interactive dashboards, and observability, 2nd Edition

✍ Scribed by Eric Salituro

• Publisher: Packt
• Year: 2023
• Tongue: English
• Leaves: 542
• Edition: 2
• Category: Library

✦ Synopsis

Get up and running with building data pipelines and creating interactive dashboards to visualize, monitor, and present a wide variety of time-series data with this comprehensive introductory guide
Key Features

Install, set up, and configure Grafana for real-time data analysis, visualization, and alerting
Visualize and monitor data using data sources such as InfluxDB, Telegraf, Prometheus, and Elasticsearch
Explore Grafana's cloud support with Microsoft Azure, Amazon CloudWatch, and Google Cloud Monitoring
Purchase of the print or Kindle book includes a free PDF eBook

Book Description

Get ready to unlock the full potential of the open-source Grafana observability platform, ideal for analyzing and monitoring time-series data with this updated second edition. This beginners guide will help you get up to speed with Grafana’s latest features for querying, visualizing, and exploring logs and metrics, no matter where they are stored.

Starting with the basics, this book demonstrates how to quickly install and set up a Grafana server using Docker. You’ll then be introduced to the main components of the Grafana interface before learning how to analyze and visualize data from sources such as InfluxDB, Telegraf, Prometheus, Logstash, and Elasticsearch. The book extensively covers key panel visualizations in Grafana, including Time Series, Stat, Table, Bar Gauge, and Text, and guides you in using Python to pipeline data, transformations to facilitate analytics, and templating to build dynamic dashboards. Exploring real-time data streaming with Telegraf, Promtail, and Loki, you’ll work with observability features like alerting rules and integration with PagerDuty and Slack. As you progress, the book addresses the administrative aspects of Grafana, from configuring users and organizations to implementing user authentication with Okta and LDAP, as well as organizing dashboards into folders, and more.

By the end of this book, you’ll have gained all the knowledge you need to start building interactive dashboards.
What you will learn

Learn the techniques of data visualization using Grafana
Get familiar with the major components of Time series visualization
Explore data transformation operations, query inspector, and time interval settings
Work with advanced dashboard features, such as annotations, variable-based templating, and dashboard linking and sharing
Connect user authentication through Okta, Google, GitHub, and other external providers
Discover Grafana’s monitoring support for cloud service infrastructures

Who this book is for

This book is for business intelligence developers, business analysts, data analysts, and anyone interested in performing time-series data analysis and monitoring using Grafana. You’ll also find this book useful if you’re looking to create and share interactive dashboards or get up to speed with the latest features of Grafana. Although no prior knowledge of Grafana is required, basic knowledge of data visualization and some Python programming experience will help you understand the concepts covered in the book.

✦ Table of Contents

Cover
Title Page
Copyright and Credits
Contributors
Table of Contents
Preface
Part 1 – Getting Started with Grafana
Chapter 1: Introducing Data Visualization with Grafana
Technical requirements
Appreciating data and visualization
Storing, retrieving, and visualizing data
Why Grafana?
Exploration
Analysis
Presentation
Observability
Choosing Grafana
Installing Grafana
Grafana in a Docker container
Make and Makefile
Grafana for macOS
Grafana for Linux
Grafana for Windows
Grafana Cloud
Connecting to the Grafana server
Summary
Further reading
Chapter 2: Touring the Grafana Interface
Technical requirements
Exploring Grafana—the Home dashboard
Introducing the Grafana search bar
Grafana logo
Organization menu
Search box
+ (add) menu
Help
Grafana blog
User menu
Expanding Grafana’s main menu
Home
Starred
Dashboards
Explore
Alerting
Connections
Administration
Exploring the Grafana dashboard UI
Grafana dashboard title bar
Dashboard content
Summary
Chapter 3: Diving into Grafana's Time Series Visualization
Touring the Grafana panel UI
Creating a simple data source
Creating a graph panel
Generating data series in the Query tab
What is a query?
Query tab features
Duplicating an existing query
Transform
Editing the panel settings
Selecting panel visualizations
The Panel options section
The Tooltip section
The Legend section
The Axis section
The Graph styles section
The Standard options section
The Data links section
The Value mappings section
The Thresholds section
Monitoring with the Alert tab
Further exploration
Summary
Part 2 – Real-World Grafana
Chapter 4: Connecting Grafana to a Prometheus Data Source
Technical requirements
Installing the Prometheus server
Installing Prometheus from Docker
Configuring the Prometheus data source
Exploring Prometheus
Using Explore for investigation
Configuring Grafana metrics
Querying the Prometheus data source
Typing in a metrics query
Querying for process metrics
Querying for memory metrics
Detecting trends with aggregations
Applying aggregations to our query data
Understanding the data source limitations
Querying limits for series aggregations
Querying limits for time aggregations
Exploring data source dashboards
Summary
Chapter 5: Extracting and Visualizing Data with InfluxDB and Grafana
Technical requirements
Making advanced queries
Launching server Docker containers
Writing the ETL script
Running the script
Setting up an InfluxDB database
Configuring the InfluxDB data source
Understanding the time series data visualization
Displaying time-aggregated data
Debugging queries with Query inspector
Observing time interval effects
Setting the minimum interval
Setting the axis
Setting axis units
Converting temperature into Fahrenheit
Autoscaling the Y axis
Dual y axis display
Working with legends
Setting legend contents
Enabling legend aggregations
Summary
Chapter 6: Shaping Data with Grafana Transformations
About Grafana DataFrames and transformations
Exploring the various transformation functions
Installing the TestData data source
Selecting transformations
Adding fields
Modifying fields
Filtering results
Reducing rows
Combining tables
Expanding analysis with a transformation
Chaining transformations into a visualization pipeline
Limitations of transformations
Summary
Chapter 7: Surveying Key Grafana Visualizations
Technical requirements
Launching server Docker containers
Setting up the InfluxDB database
Initializing the InfluxDB server
Generating an API token
Configuring the InfluxDB data source
Building the Python Docker container
Loading the data
Reviewing the table visualization
Comparing aggregations
Overriding field settings
Setting a display name
Introducing the stat visualization
Creating a stat visualization panel
Setting stat styles
Defining value mappings in a stat visualization
Adding visual interest with a gauge
Exploring the gauge options
Setting the threshold values and colors
Going linear with a bar gauge
Exploring the bar gauge settings
Building a bar gauge
Summary
Chapter 8: Surveying Additional Grafana Visualizations
Technical requirements
Launching server Docker containers
Setting up the InfluxDB database
Initializing the InfluxDB server
Generating an API token
Configuring the InfluxDB data source
Building the Python Docker container
Exploring spatial data with the Geomap visualization
Ingesting a new earthquake dataset
Mapping earthquake data with the Geomap visualization
Displaying category data with a bar chart visualization
Understanding histograms
Producing histograms with transformations
Displaying histogram data with the bar chart visualization
Visualizing histogram data over time with the heatmap
Summary
Chapter 9: Creating Insightful Dashboards
Technical requirements
Launching server Docker containers
Setting up the InfluxDB database
Designing a dashboard
Conveying information
Determining the visual context
Prioritizing elements of importance
Creating a high information-density dashboard
Ingesting the weather data
Designing the dashboard
Adding the first row
Adding the second row
Adding the third row
Adding the fourth row
Adding the fifth row
Creating a high-information visibility dashboard
Designing the dashboard
Ingesting the earthquake data
Adding the first row
Adding the second row
Adding the third row
Parting thoughts
Considering layout
Automating ingestion
Summary
Chapter 10: Working with Advanced Dashboard Features and Elasticsearch
Technical requirements
Building the data server
Connecting Grafana to Elasticsearch
Querying with Elasticsearch
Creating a template variable
Adding template variables to the graph panel
Templating additional variables
Creating ad hoc filters
Repeating rows and panels with template variables
Creating a new dashboard
Setting up the template variables
Configuring the panels
Linking dashboards
Adding dashboard tags
Locking down a template variable
Creating dashboard links
Annotating dashboards
Annotating the graph panel
Querying tagged annotations
Creating Elasticsearch annotation queries
Sharing dashboards
Sharing dashboard links
Sharing dashboards by exporting
Sharing dashboard snapshots
Summary
Chapter 11: Streaming Real-Time IoT Data from Telegraf Agent to Grafana Live
Technical requirements
Streaming real-time data from Telegraf to Grafana
Setting up Grafana for streaming
Installing the Telegraf agent
Configuring the Telegraf agent
Running Telegraf
Streaming IoT data with MQTT and Mosquitto
Describing the pipeline architecture
Configuring the Telegraf MQTT consumer
Installing the Mosquitto broker
Generating messages to an IoT data pipeline
Examining the simulated metrics data
Reviewing the mqtt_pub.py script
Running the mqtt_pub.py script
Exploring the IoT data
Summary
Chapter 12: Monitoring Data Streams with Grafana Alerts
Technical requirements
Monitoring and observability
Monitoring processes
Monitoring system resources
Monitoring applications
Capturing metrics for alerting
Alerting in Grafana
Alert rules
Labels
Notification policies
Contact points
Defining alert rules
What condition…
What metrics…
How long…
Monitoring systems
Monitoring applications
Alert messaging to contact points
Configuring an email contact point
Configuring a PagerDuty contact point
Configuring a Slack contact point
Routing alerts with notification policies
Specific routing
Mute timings
On your own
Summary
Chapter 13: Exploring Log Data with Grafana’s Loki
Technical requirements
Loading system logs into Loki
Networking our services
Installing Promtail
Installing Loki
Launching Grafana
Creating a Loki data source
Visualizing Loki log data with Explore
Simulating logs with flog
Configuring promtail
Promtail in Docker Compose
Alternative Docker log capture
Querying logs and metrics with Explore
Summary
Part 3 – Managing Grafana
Chapter 14: Organizing Dashboards and Folders
Managing dashboards and folders
Naming a dashboard
Dashboard naming tips
Working with dashboard folders
Tips to manage dashboard folders
Starring and tagging dashboards
Marking dashboards as favorites
Tagging dashboards
Building and running dashboard playlists
Creating a playlist
Displaying a playlist
Displaying playlists in normal mode
Displaying playlists in TV mode
Displaying playlists in Kiosk mode
Displaying playlists with auto fit panels
Editing a playlist
Exploring the Dashboard list panel
Setting Dashboard list panel options
Duplicating dashboards
Internal dashboard duplications
External dashboard duplications
Summary
Chapter 15: Managing Permissions for Users, Teams, and Organizations
Understanding key permission concepts
Organizations
Users
Roles
Teams
Adding users
Adding users – by invitation only
Adding users – a self-service model
Setting permissions
Setting organization roles
Setting folder permissions
Setting dashboard permissions
Establishing teams
Setting up a team
Permission rules
Administering users and organizations
Managing users
Organization admin and Super Admin roles
Managing organizations
Summary
Chapter 16: Authenticating Grafana Logins Using LDAP or OAuth 2 Providers
Authenticating with OpenLDAP
Setting up an OpenLDAP server
Configuring Grafana to use LDAP
Securing the bind password
Testing the Grafana configuration
Adding a user to OpenLDAP
Looking up a user in Grafana
Authenticating with OAuth 2
GitHub OAuth 2 authentication
Google OAuth 2 authentication
Authenticating with Okta
Summary
Chapter 17: Cloud Monitoring AWS, Azure, and GCP
Configuring an AWS CloudWatch data source
Creating the policy
Creating the user
Configuring a new Grafana connection for AWS CloudWatch
Configuring a Microsoft Azure Monitor data source
Registering the Grafana application
Setting the application role
Generating application secrets
Configuring a new Grafana connection for Azure Monitor
Configuring Azure Log Analytics
Configuring a GCM data source
Enabling a Google Cloud API
Creating a Google service account
Configuring a new Grafana connection for GCM
Summary
Index
Other Books You May Enjoy