๐”– Scriptorium
โœฆ   LIBER   โœฆ

๐Ÿ“

Pro Python system administration

โœ Scribed by Sileika, Rytis


Publisher
Apress, Distributed to the Book trade worldwide by Springer Science+Business Media New York
Year
2014
Tongue
English
Leaves
411
Series
Expert's voice in Python
Edition
Second edition
Category
Library

โฌ‡  Acquire This Volume

No coin nor oath required. For personal study only.

โœฆ Synopsis


As time goes on, system administrators are presented with increasingly complicated challenges. In the early days, a team of engineers might have had to look after one or two systems. These days, one engineer can administer hundreds or thousands of systems.

System administrators are gradually replacing their tools with more advanced and flexible ones. One of the choices is Python. Structurally, Python is a modern, high-level language with a very clean syntax. Python comes with many built-in libraries that can make automation tasks easier. It also has extensive set of third-party libraries and a very active development community. This flexibility makes Python a good choice for a wide variety of tasks, from prototyping science applications to system maintenance and administration jobs.

This book explains and shows how to apply Python scripting in practice. Unlike the majority of the Python books, it will show you how to approach and resolve real-world issues that most system administrators will come across in their careers. In this book, you will find several projects in the categories of network administration, web server administration, and monitoring and database management. In each project, we will define the problem, design the solution, and go through the more interesting implementation steps. Each project is accompanied with the source code of a fully working prototype, which you'll be able to use immediately or adapt to your requirements and environment.

โœฆ Table of Contents


Contents at a Glance......Page 3
Contents......Page 399
About the Author......Page 409
About the Technical Reviewers......Page 410
Acknowledgments......Page 411
Introduction......Page 4
Specifying the Requirements......Page 8
Introduction to SNMP......Page 9
The System SNMP Variables Node......Page 11
The Interfaces SNMP Variables Node......Page 12
Querying SNMP from the Command Line......Page 14
Configuring the Application......Page 18
The SNMP GET Command......Page 20
The SNMP SET Command......Page 21
The SNMP GETNEXT Command......Page 22
Implementing the SNMP Read Functionality......Page 23
Introduction to RRDTool......Page 24
Creating a Round Robin Database......Page 26
Writing and Reading Data from the Round Robin Database......Page 28
Plotting Graphs with RRDTool......Page 31
Integrating RRDTool with the Monitoring Solution......Page 34
Creating Web Pages with the Jinja2 Templating System......Page 36
Loading Template Files with Jinja2......Page 37
Accessing Variables......Page 38
Flow Control Statements......Page 39
Generating Website Pages......Page 40
Summary......Page 43
What Is the SOAP API?......Page 44
Requesting Services with SOAP......Page 45
Finding Information About Available Services with WSDL......Page 46
Converting WSDL Schema to Python Helper Module......Page 48
Defining Requirements for Our Load Balancer Tool......Page 49
Code Structure......Page 50
Configuration......Page 51
Fixing Issues with Citrix Netscaler WSDL......Page 52
Creating a Connection Object......Page 53
Logging In: Our First SOAP Call......Page 55
Finding What Is Being Returned in the Response......Page 57
How Is the Session Maintained After We Have Logged On?......Page 59
Gathering Performance Statistics Data......Page 60
SOAP Methods for Reading Statistical Data and its Return Values......Page 61
Reading System Health Data......Page 62
Reading Service Status Data......Page 63
Device Configuration SOAP Methods......Page 66
Setting a Service State......Page 67
Logging Levels and Scope......Page 69
Configuring and Using the Logger......Page 70
Handling Exceptions......Page 71
Download......Page 73
Module Layout......Page 74
Logging On......Page 80
Gathering Statistical Data......Page 81
Performing Administration Tasks......Page 84
Summary......Page 85
Setting out the Requirements......Page 86
Defining the Database Schema......Page 87
The Add Function......Page 90
What Is Django?......Page 91
The View Component......Page 92
Installing the Django Framework......Page 93
The Structure of a Django Application......Page 94
Using Django with Apache Web Server......Page 96
Defining the Database Model......Page 98
URL Configuration......Page 101
Enabling the Management Interface......Page 103
Allowing the Administration Plug-in to Manage New Models......Page 105
Viewing Records......Page 106
Using Templates......Page 108
Deleting Records......Page 111
Adding New Records......Page 112
Modifying Existing Records......Page 115
Summary......Page 116
Extending the Design and Requirements......Page 117
Extending the Database Schema......Page 119
The Data Models......Page 120
The Add Function......Page 121
Extending the DHCP Configuration with Address Pools......Page 123
The DHCP Network Details......Page 124
Generation of URLs in the Model Class......Page 126
Assignment of Names to URL Patterns......Page 127
Use of URL References in the Templates......Page 128
Additions to the Data Model......Page 129
Template Inheritance......Page 130
Display of a List of Objects......Page 131
Detailed View of the Object......Page 132
New Objects Added or Modified......Page 134
Deletion of Objects......Page 135
Generating the DHCP Configuration File......Page 136
Checking Whether the Address Is in Use......Page 140
Employ Python Interface to OMAPI......Page 143
Set up the ISC DHCP Server......Page 144
Add a New Host Lease Record......Page 145
Delete a Host Lease Record......Page 146
Summary......Page 147
Functional Requirements......Page 148
Apache Configuration......Page 149
Configuring the Application......Page 150
Defining the URL Structure......Page 152
The Data Model......Page 153
The Basic Model Structure......Page 154
Customizing the Class Names......Page 157
Adding New Fields to the Object List......Page 158
Reorganizing the Form Fields......Page 160
Adding Custom Object Actions......Page 162
Generating the Configuration File......Page 164
Summary......Page 166
Application Structure and Functionality......Page 167
Plug-in Framework Implementation in Python......Page 168
Interface Model......Page 169
Creating the Plug-in Framework......Page 170
Discovery and Registration......Page 171
Format of Apache Log Files......Page 174
Log File Reader......Page 176
Calling the Plug-in Methods......Page 179
Plug-in Methods and the Call Mechanism......Page 180
Installing the Required Libraries......Page 182
Using the GeoIP Python Bindings......Page 183
Writing the Plug-in Code......Page 184
Installing Required Libraries and Data Files......Page 185
Working with Shapefile......Page 186
Displaying the Requests Data on the World Map......Page 188
Summary......Page 191
Defining the Problem......Page 192
Why We Use Exceptions......Page 194
Why We Should Analyze Exceptions......Page 195
What Can We Find in a Typical Log File?......Page 196
The Structure of an Exception Stack Trace Log......Page 197
Handling Multiple Files......Page 199
Handling Multiple Files......Page 200
Using the Built-in Bzip2 Library......Page 202
What Are Generators, and How Are They Used?......Page 203
Detecting the Exceptions......Page 204
Detecting the Potential Candidates......Page 205
Storing Data in Data Structures......Page 206
Generating an Exception Fingerprint for Unknown Exceptions......Page 207
The Configuration File......Page 210
Parsing XML Files with Python......Page 211
Storing and Applying Filters......Page 213
The Benefits of a Precompiled Search over a Plain-Text Search......Page 214
Grouping the Exceptions......Page 215
Producing Differently Formatted Outputs for the Same Dataset......Page 216
Calculating Group Statistics......Page 217
Summary......Page 218
Requirements for the Check System......Page 219
Nagios Plug-In Architecture......Page 220
Installing the Beautiful Soup HTML Parsing Library......Page 221
Retrieving a Web Page......Page 222
Parsing the HTML Pages with Beautiful Soup......Page 223
Adding the New Check to the Nagios System......Page 229
Emulating the User Login Process......Page 231
Basic Usage......Page 236
Rewriting Site Logon Check Script......Page 239
Summary......Page 241
The Components......Page 242
The Monitoring Server......Page 243
Configuration......Page 244
Introduction to Data Normalization......Page 245
Configuration Data......Page 248
Scheduling......Page 250
Site Configuration......Page 251
Communication Flows......Page 252
Structure......Page 253
Python Support......Page 254
CherryPy......Page 255
Initializing the Database File......Page 256
Accepting Sensor Readings......Page 261
Supplying a New Configuration......Page 262
Actions......Page 263
Multithreading, Multiprocessing, and GIL......Page 264
Basic Usage Patterns and Examples......Page 265
A Simple Clock Implementation......Page 268
A Cron-Like Scheduler......Page 270
Ticket Dispatcher......Page 272
Summary......Page 274
Accepting a New Configuration......Page 275
The Security Model......Page 276
The File Format......Page 277
Using the ConfigParser Class Methods......Page 278
The Configuration Class Wrapper......Page 282
The Sensor Design......Page 286
Using the Subprocess Library......Page 287
Controlling the Running Processes......Page 290
Using File Objects......Page 293
Using the Pipe Objects......Page 294
Redirecting Standard Error......Page 295
Sending and Receiving Binary Data with XML-RPC......Page 296
Working with Files and Archives (TAR and BZip2)......Page 297
Summary......Page 299
Using the NumPy Library......Page 300
Working with Arrays......Page 301
Basic Mathematical and Statistical Operations......Page 304
Calculating the Mean and Standard Deviation......Page 305
Finding the Trend Line of a Dataset......Page 309
Reading and Writing Data to Files......Page 311
Installing matplotlib......Page 312
Plotting Graphs......Page 313
Changing the Appearance of the Plot Primitives......Page 314
Drawing Bars and Using Multiple Axes......Page 315
Working with Text Strings......Page 317
Saving Plots to a File......Page 318
Displaying Available Hosts......Page 319
Drawing Timescale Graphs......Page 320
The Index Page......Page 321
Host Details Page......Page 322
Graph Collection Pages......Page 324
Plotting Performance Graphs......Page 326
Summary......Page 328
Task Queuing Systems......Page 329
Message Queuing Systems......Page 330
Installing and Setting up RabbitMQ......Page 331
Installing and Setting up Celery......Page 332
Create Celery Project Directory and Sample Application......Page 333
Create Systemd Configuration Files......Page 334
Layout of a Typical Celery Application......Page 336
Worker and Master Process Application Files......Page 337
Overview of the Celery Configuration File......Page 338
The Main Celery Application File......Page 339
Systemd Configuration......Page 340
Inside a Message Queue System......Page 341
Binding Worker Node to Specific Queues......Page 342
Sending Broadcast Messages......Page 344
Summary......Page 345
Requirements Specification and Design......Page 346
System Design......Page 347
Changes to the Host Application......Page 348
Accessing the MySQL Database from Python Applications......Page 350
Querying the Configuration Variables......Page 354
Querying the Server Status Variables......Page 356
Collecting the Host Configuration Data......Page 357
Checking the MySQL Version......Page 358
Checking the Key Buffer Size Setting......Page 360
Checking the Slow Queries Counter......Page 361
Summary......Page 363
The Problem......Page 364
The Amazon EC2 and S3 Crash Course......Page 365
Access Credentials......Page 366
The Simple Storage System Concepts......Page 367
Amazon Machine Images and Instances......Page 368
Elastic Block Store......Page 371
Creating a Custom EC2 Image......Page 372
Reusing Existing Images......Page 373
Create and Set Up an Elastic Block Store Volume......Page 374
Configure the MySQL Instance......Page 375
Bundling the New AMI......Page 376
Controlling the EC2 Using the Boto Python Module......Page 378
Initializing the EC2 Instance Programmatically......Page 379
Attaching the EBS Volume......Page 381
Mounting the EBS Device......Page 382
Shutting Down the MySQL Instance......Page 383
Taking a Snapshot of the Volume......Page 384
The Control Sequence......Page 385
Summary......Page 386
Index......Page 387


๐Ÿ“œ SIMILAR VOLUMES


Pro Python System Administration
โœ Rytis Sileika ๐Ÿ“‚ Library ๐Ÿ“… 2010 ๐Ÿ› Apress ๐ŸŒ English

<p>As time goes on, system administrators are presented with increasingly complicated challenges. In the early days, a team of engineers might have had to look after one or two systems. These days, one engineer can administer hundreds or thousands of systems.</p> <p>System administrators are grad

Pro Python System Administration
โœ Rytis Sileika ๐Ÿ“‚ Library ๐Ÿ“… 2010 ๐Ÿ› Apress ๐ŸŒ English

As time goes on, system administrators are presented with increasingly complicated challenges. In the early days, a team of engineers might have had to look after one or two systems. These days, one engineer can administer hundreds or thousands of systems.System administrators are gradually replacin

Pro Python System Administration
โœ Rytis Sileika ๐Ÿ“‚ Library ๐Ÿ“… 2010 ๐Ÿ› Apress ๐ŸŒ English

As time goes on, system administrators are presented with increasingly complicated challenges. In the early days, a team of engineers might have had to look after one or two systems. These days, one engineer can administer hundreds or thousands of systems. System administrators are gradually replaci

Pro Python System Administration
โœ Rytis Sileika ๐Ÿ“‚ Library ๐Ÿ“… 2010 ๐Ÿ› Apress ๐ŸŒ English

As time goes on, system administrators are presented with increasingly complicated challenges. In the early days, a team of engineers might have had to look after one or two systems. These days, one engineer can administer hundreds or thousands of systems.</p><p>System administrators are gradually r

Pro Python system administration
โœ Sileika, Rytis ๐Ÿ“‚ Library ๐Ÿ“… 2010 ๐Ÿ› Apress, Distributed by Springer Science+Business M ๐ŸŒ English

As time goes on, system administrators are presented with increasingly complicated challenges. In the early days, a team of engineers might have had to look after one or two systems. These days, one engineer can administer hundreds or thousands of systems.<br /><br />System administrators are gradua