Cloud Native Observability

✍ Scribed by Kenichi Shibata, Rob Skillington, and Martin Mao

• Publisher: O'Reilly Media, Inc.
• Year: 2024
• Tongue: English
• Leaves: 61
• Category: Library

✦ Synopsis

With this insightful guide, authors Kenichi Shibata, Rob Skillington, and Martin Mao take you through the differences between traditional and cloud native system observability. SREs, cloud native engineers, CIOs, and CTOs will learn that while many principles of cloud native and traditional systems are similar, highly scalable and dynamic cloud native systems present unique challenges to overcome.

✦ Table of Contents

1. The Cloud Native Impact on Observability
   Challenges of Cloud Native Observability
   Deep Dive into Observability Data
   Observability Data Is Growing in Scale
   Understanding Cardinality and Dimensionality
   Cloud Native Systems Are Flexible and Ephemeral
   The Goldilocks Zone of Cloud Native Observability
   Cloud Native Environments Emit Exponentially More Data Than Traditional Environments
   Delivering Reduced Business Outcomes
   Observability Practitioners Lose Focus
   Increasing Cost of Observability Data
   The Cloud Native Impact
   Slower Troubleshooting
   Tools Become Unreliable
   Use Context to Troubleshoot Faster
   The Three Phases of Observability: An Outcome-Focused Approach
   Remediating at Any Phase, with Any Signal
   Conclusion
2. Cloud Native Challenges in the Real World
   Impact of Uncontrolled Data Growth on System Performance
   Controlling Cost
   Case Study 1: Improving Performance While Gaining Huge Cost Savings
   The Challenge
   Approach
   Impact of Uncontrolled Data Growth on Observability Reliability
   Poor Developer Experience Caused by Poor Observability Data
   Case Study 2: Increased Observability Reliability and Improved Developer Experience
   The Challenge
   Approach
   Making Way for Fast-Paced Innovation
   Regulatory Requirements
   Case Study 3: Navigating Observability Challenges in Balancing Rapid Fintech Growth and SLA Compliance
   The Challenge
   Approach
   Conclusion
3. Strategies for Controlling Observability Data Growth and Complexity
   Emerging Solution Using a Repeatable Framework
   Using FinOps as an Inspiration
   Observability Data Optimization Cycle
   Step 0: Centralized Governance
   Autonomy and Allocations to Increase Responsibility and Improve Responsiveness
   Usable Capacity by Allocation to Optimize Use Cases
   Using Observability Team as Consultants Instead of as Bottlenecks
   Framework Components
   Step 1: Analyze
   Traffic Analysis
   Usage Analysis
   Combining Traffic and Usage Analysis to Make Decisions
   Output of Analyze Step
   Step 2: Refine
   Dropping
   Retention
   Resolution
   Downsampling
   Aggregation
   Output of Refine Step
   Step 3: Operate
   Expanding Visibility and Coverage
   Freeing Up More of the Observability Team’s Time to Tackle Strategic Projects
   Conclusion
4. Open Source Telemetry Standards: Prometheus, OpenTelemetry, and Beyond
   Instrumentation Before Prometheus and OTel
   Data Collection Is Controlled by Users
   Prometheus
   Interoperability Between Different Observability Tools
   Standardization to Prometheus
   Prometheus Reliability
   Prometheus: The Good
   Prometheus: The Not-So-Good
   OpenTelemetry
   What Is OTel?
   The OTel Specification
   OTel SDK
   OpenTelemetry Collector
   OTel: The Promise
   OTel: The Reality
   Limitations of maturity
   Backend support
   Where to Start with OTel
   Implications of OTel’s Approach
   Fluent Bit
   Conclusion