Mastering Python Scripting for System Administrators: Write scripts and automate them for real-world administration tasks using Python

✍ Scribed by Ganesh Sanjiv Naik

• Publisher: Packt Publishing
• Year: 2019
• Tongue: English
• Leaves: 308
• Category: Library

✦ Synopsis

Leverage the features and libraries of Python to administrate your environment efficiently.

Key Features

• Learn how to solve problems of system administrators and automate routine activities
• Learn to handle regular expressions, network administration
• Building GUI, web-scraping and database administration including data analytics

Book Description

Python has evolved over time and extended its features in relation to every possible IT operation. Python is simple to learn, yet has powerful libraries that can be used to build powerful Python scripts for solving real-world problems and automating administrators' routine activities. The objective of this book is to walk through a series of projects that will teach readers Python scripting with each project.

This book will initially cover Python installation and quickly revise basic to advanced programming fundamentals. The book will then focus on the development process as a whole, from setup to planning to building different tools. It will include IT administrators' routine activities (text processing, regular expressions, file archiving, and encryption), network administration (socket programming, email handling, the remote controlling of devices using telnet/ssh, and protocols such as SNMP/DHCP), building graphical user interface, working with websites (Apache log file processing, SOAP and REST APIs communication, and web scraping), and database administration (MySQL and similar database data administration, data analytics, and reporting).

By the end of this book, you will be able to use the latest features of Python and be able to build powerful tools that will solve challenging, real-world tasks

What you will learn

• Understand how to install Python and debug Python scripts
• Understand and write scripts for automating testing and routine administrative activities
• Understand how to write scripts for text processing, encryption, decryption, and archiving
• Handle files, such as pdf, excel, csv, and txt files, and generate reports
• Write scripts for remote network administration, including handling emails
• Build interactive tools using a graphical user interface
• Handle Apache log files, SOAP and REST APIs communication
• Automate database administration and perform statistical analysis

Who this book is for

This book would be ideal for users with some basic understanding of Python programming and who are interested in scaling their programming skills to command line scripting and system administration.

Prior knowledge of Python would be necessary.

Table of Contents

1. Python Scripting Overview
2. Debugging and Profiling Python Scripts
3. Unit Testing - Introduction to the Unit Testing Framework
4. Automating Regular Administrative Activities
5. Handling Files, Directories, and Data
6. File Archiving, Encrypting, and Decrypting
7. Text Processing and Regular Expressions
8. Documentation and Reporting
9. Working with Various Files
10. Basic Networking - Socket Programming
11. Handling Emails Using Python Scripting
12. Remote Monitoring of Hosts Over Telnet and SSH
13. Building Graphical User Interfaces
14. Working with Apache and Other Log Files
15. SOAP and REST API Communication
16. Web Scraping - Extracting Useful Data from Websites
17. Statistics Gathering and Reporting
18. MySQL and SQLite Database Administrations
19. Assessments

✦ Table of Contents

Cover
Title Page
Copyright and Credits
Dedication
About Packt
Contributors
Table of Contents
Preface
Chapter 1: Python Scripting Overview
Technical requirements
Why Python?
Python syntax compared to other programming languages
Python installation
Installation on the Linux platform
Installation on the Windows platform
Installing and using pip to install packages
Installation on Mac
Installing Jupyter notebook
Installing and using the virtual environment
Installing Geany and PyCharm
Python interpreter
The difference between Python and Bash scripting
Starting the interactive console
Writing scripts with the Python interactive console
Multiple lines
Importing modules through the Python interpreter
Exiting the Python console
The keyboard shortcut
Using the quit() or exit() functions
Indentation and tabs
Variables
Creating and assigning values to variables
Numbers
Number type conversion
Strings
Concatenation (+) and repetition (*)
String slicing
Accessing values in strings
Updating strings
Escape characters
Special string operators
%  string formatting operator
Triple quotes in Python
Strings are immutable
Understanding lists
Accessing values in lists
Updating lists
Deleting list elements
Basic list operations
List operations
Indexing, slicing, and matrices
Tuples
Accessing values in tuples
Updating tuples
Deleting tuple elements
Basic tuple operations
Indexing, slicing, and matrices
max() and min()
Sets
Dictionaries
Parsing command-line arguments
Command-line arguments in Python
Sys.argv
Decision making
Python if statement syntax
Python if...else statement syntax
Python if...elif...else statement
Loops
for loop
The range() function
while loop
Iterators
Generators
How to create a generator in Python?
Functions
The return statement
Lambda functions
Modules
Importing modules
Summary
Questions
Further reading
Chapter 2: Debugging and Profiling Python Scripts
What is debugging?
Python debugging techniques
Error handling (exception handling)
Debuggers tools
The pdb debugger
Within an interpreter
From a command line
Within a Python script
Debugging basic program crashes
Profiling and timing programs
The cProfile module
timeit
Making programs run faster
Summary
Questions
Further reading
Chapter 3: Unit Testing - Introduction to the Unit Testing Framework
What is unittest?
Creating unit tests
Methods used in unit testing
Summary
Questions
Further reading
Chapter 4: Automating Regular Administrative Activities
Accepting input by redirection, pipe, and input files
Input by redirection
Input by pipe
Input by input file
Handling passwords at runtime in scripts
Executing external commands and getting their output
Capturing output using the subprocess module
Prompting for passwords during runtime and validation
Reading configuration files
Adding logging and warning code to scripts
Generating warnings
Putting limits on CPU and memory usage
Launching webbrowser
Using the os module for handling directory and files
Creating and deleting the directory
Examining the content of a filesystem
Making backups (with rsync)
Summary
Questions
Further reading
Chapter 5: Handling Files, Directories, and Data
Using the os module to work with directories
Get the working directory
Changing the directory
Listing files and directories
Renaming a directory
Copying, moving, renaming, and deleting data
Copying the data
Moving the data
Renaming data
Deleting data
Working with paths
Comparing data
Merging data
Pattern matching files and directories
Metadata: data about data
Compressing and restoring
Using the tarfile module to create TAR archives
Using a tarfile module to examine the contents of TAR files
Summary
Questions
Further reading
Chapter 6: File Archiving, Encrypting, and Decrypting
Creating and unpacking archives
Creating archives
Unpacking archives
Tar archives
ZIP creation
File encryption and decryption
Summary
Questions
Further reading
Chapter 7: Text Processing and Regular Expressions
Text wrapping
The wrap() function
The fill() function
The dedent() function
The indent() function
The shorten() function
Regular expressions
The match() function
The search() function
The findall() function
The sub() function
Unicode strings
Unicode code point
Encoding
Decoding
Avoiding UnicodeDecodeError
Summary
Questions
Further reading
Chapter 8: Documentation and Reporting
Standard input and output
Information formatting
Sending email
Summary
Questions
Further reading
Chapter 9: Working with Various Files
Working with PDF files
Reading a PDF document and getting the number of pages
Extracting text
Rotating PDF pages
Working with Excel files
Using the xlrd module
Reading an Excel file
Extracting the names of columns
Using pandas 
Reading an Excel file
Reading specific columns in an Excel file
Using openpyxl
Creating a new Excel file
Appending values
Reading multiple cells
Working with CSV files
Reading a CSV file
Writing into a CSV file
Working with txt files
The open() function
File opening
The close() function
Writing a text file
Reading a text file
Summary
Questions
Further reading
Chapter 10: Basic Networking - Socket Programming
Sockets
The http package
The http.client module
The http.server module
The ftplib module
Downloading files
Getting a welcome message using getwelcome():
Sending commands to the server using the sendcmd() function
The urllib package
Python urllib response headers
Summary
Questions
Further reading
Chapter 11: Handling Emails Using Python Scripting
Email message format
Adding HTML and multimedia content
POP3 and IMAP servers
Receiving email using the poplib library
Receiving email using the imaplib library
Summary
Questions
Chapter 12: Remote Monitoring of Hosts Over Telnet and SSH
The telnetlib() module
SSH
The subprocess.Popen() module
SSH using fabric module
SSH using the Paramiko library
SSH using the Netmiko library
Summary
Questions
Further reading
Chapter 13: Building Graphical User Interfaces
Introduction to GUI
Using a library to create a GUI-based application
Installing and using the Apache Log Viewer app
Summary
Questions
Further reading
Chapter 14: Working with Apache and Other Log Files
Parsing complex log files
The need for exceptions
Analyzing exceptions
Tricks for parsing different files
Error log
Access log
Common log format
Parsing other log files
Summary
Questions
Further reading
Chapter 15: SOAP and REST API Communication
What is SOAP?
Using libraries for SOAP
What is a RESTful API?
Using standard libraries for RESTful APIs
Working with JSON data
Summary
Questions
Further reading
Chapter 16: Web Scraping - Extracting Useful Data from Websites
What is web scraping?
Data extraction
The requests library
The beautifulsoup library
Extracting information from Wikipedia
Summary
Questions
Further reading
Chapter 17: Statistics Gathering and Reporting
NumPY module
Using arrays and scalars
Array indexing
Indexing a 2D array
Universal array functions
Pandas module
Series
DataFrames
Data visualization
Matplotlib
Histograms
Scatter plots
Bar charts
Plotly
Scatter plots
Line scatter plots
Box plots
Contour plots
Summary
Questions
Further reading
Chapter 18: MySQL and SQLite Database Administrations
MySQL database administration
Getting a database version
Creating a table and inserting data
Retrieving the data
Updating the data
Deleting the data
SQLite database administration
Connecting to the database
Creating a table
Inserting the data
Retrieving the data
Updating the data
Deleting the data
Summary
Questions
Further reading
Assessments
Other Books You May Enjoy
Index