Software Architecture with Python

✍ Scribed by Anand Balachandran Pillai

• Year: 2017
• Tongue: English
• Leaves: 557
• Category: Library

✦ Synopsis

Architect and design highly scalable, robust, clean, and highly performant applications in PythonAbout This Book Identify design issues and make the necessary adjustments to achieve improved performance Understand practical architectural quality attributes from the perspective of a practicing engineer and architect using Python Gain knowledge of architectural principles and how they can be used to provide accountability and rationale for architectural decisionsWho This Book Is ForThis book is for experienced Python developers who are aspiring to become the architects of enterprise-grade applications or software architects who would like to leverage Python to create effective blueprints of applications.What You Will Learn Build programs with the right architectural attributes Use Enterprise Architectural Patterns to solve scalable problems on the Web Understand design patterns from a Python perspective Optimize the performance testing tools in Python Deploy code in remote environments or on the Cloud using Python* Secure architecture applications in PythonIn DetailThis book starts off by explaining how Python fits into an application architecture. As you move along, you will understand the architecturally significant demands and how to determine them. Later, you'll get a complete understanding of the different architectural quality requirements that help an architect to build a product that satisfies business needs, such as maintainability/reusability, testability, scalability, performance, usability, and security.You will use various techniques such as incorporating DevOps, Continuous Integration, and more to make your application robust. You will understand when and when not to use object orientation in your applications. You will be able to think of the future and design applications that can scale proportionally to the growing business.The focus is on building the business logic based on the business process documentation and which frameworks are to be used when. We also cover some important patterns that are to be taken into account while solving design problems as well as those in relatively new domains such as the Cloud.This book will help you understand the ins and outs of Python so that you can make those critical design decisions that not just live up to but also surpass the expectations of your clients.Style and approachFilled with examples and use cases, this guide takes a no-nonsense approach to help you with everything it takes to become a successful software architect.

✦ Table of Contents

Cover
Copyright
Credits
About the Author
About the Reviewer
www.PacktPub.com
Customer Feedback
Table of Contents
Preface
Chapter 1: Principles of Software Architecture
Defining Software Architecture
Software Architecture versus design
Aspects of Software Architecture
Characteristics of Software Architecture
An architecture defines a structure
An architecture picks a core set of elements
An architecture captures early design decisions
An architecture manages stakeholder requirements
An architecture influences the organizational structure
An architecture is influenced by its environment
An architecture documents the system
An architecture often conforms to a pattern
Importance of Software Architecture
System versus enterprise architecture
Architectural quality attributes
Modifiability
Testability
Scalability
Performance
Availability
Security
Deployability
Summary
Chapter 2: Writing Modifiable and Readable Code
What is modifiability?
Aspects related to Modifiability
Understanding readability
Python and readability
Readability - antipatterns
Techniques for readability
Document your code
Follow coding and style guidelines
Review and refactor code
Commenting the code
Fundamentals of Modifiability – Cohesion & Coupling
Measuring cohesion and coupling
Measuring cohesion and coupling – string and text processing
Exploring strategies for modifiability
Providing explicit interfaces
Reducing two-way dependencies
Abstract common services
Using inheritance techniques
Using late binding techniques
Metrics – tools for static analysis
What are code smells?
Cyclomatic complexity – the McCabe metric
Testing for metrics
Running Static Checkers
Refactoring Code
Refactoring code – fixing complexity
Refactoring code - fixing code smells
Refactoring code - fixing styling and coding issues
Summary
Chapter 3: Testability – Writing Testable Code
Understanding testability
Software testability and related attributes
Testability – architectural aspects
Testability – strategies
Reduce system complexity
Improving predictability
Control and isolate external dependencies
White-box testing principles
Unit testing
Unit testing in action
Extending our unit test case
Nosing around with nose2
Testing with py.test
Code coverage
Measuring coverage using coverage.py
Measuring coverage using nose2
Measuring coverage using py.test
Mocking things up
Tests inline in documentation – doctests
Integration tests
Test automation
Test automation using Selenium Web Driver
Test-Driven Development
TDD with palindromes
Summary
Chapter 4: Good Performance is Rewarding!
What is performance?
Software performance engineering
Performance testing and measurement tools
Performance complexity
Measuring performance
Measuring time using a context manager
Timing code using the timeit module
Measuring the performance of our code using timeit
Finding out time complexity – graphs
Measuring CPU time with timeit
Profiling
Deterministic profiling
Profiling with cProfile and profile
Prime number iterator class – performance tweaks
Profiling – collecting and reporting statistics
Third-party profilers
Line profiler
Memory profiler
Substring (subsequence) problem
Other tools
Objgraph
Pympler
Programming for performance – data structures
Mutable containers – lists, dictionaries, and sets
Lists
Dictionaries
Sets
Immutable containers – tuples
High performance containers – the collections module
deque
defaultdict
OrderedDict
Counter
ChainMap
namedtuple
Probabilistic data structures – bloom filters
Summary
Chapter 5: Writing Applications That Scale
Scalability and performance
Concurrency
Concurrency versus parallelism
Concurrency in Python – multithreading
Thumbnail generator
Thumbnail generator – producer/consumer architecture
Thumbnail generator – resource constraint using locks
Thumbnail generator – resource constraint using semaphores
Resource constraint – semaphore versus lock
Thumbnail generator – URL rate controller using conditions
Multithreading – Python and GIL
Concurrency in Python – multiprocessing
A primality checker
Sorting disk files
Sorting disk files – using a counter
Sorting disk files – using multiprocessing
Multithreading versus multiprocessing
Concurrecy in Python - Asynchronous Execution
Pre-emptive versus cooperative multitasking
The asyncio module in Python
Waiting for a future – async and await
Concurrent futures – high-level concurrent processing
Disk thumbnail generator
Concurrency options – how to choose?
Parallel processing libraries
Joblib
PyMP
Fractals – the Mandelbrot set
Fractals – Scaling the Mandelbrot set implementation
Scaling for the Web
Scaling workflows – message queues and task queues
Celery – a distributed task queue
The Mandelbrot set using Celery
Serving with Python on the Web—WSGI
uWSGI – WSGI middleware on steroids
Gunicorn – unicorn for WSGI
Gunicorn versus uWSGI
Scalability architectures
Vertical scalability architectures
Horizontal scalability architectures
Summary
Chapter 6: Security – Writing Secure Code
Information Security architecture
Secure coding
Common security vulnerabilities
Is Python secure?
Reading input
Evaluating arbitrary input
Overflow errors
Serializing objects
Security issues with web applications
Server Side Template Injection
Server-Side Template Injection – mitigation
Denial of Service
Cross-Site Scripting(XSS)
Mitigation – DoS and XSS
Strategies for security – Python
Secure coding strategies
Summary
Chapter 7: Design Patterns in Python
Design patterns - Elements
Categories of design patterns
Pluggable hashing algorithms
Summing up pluggable hashing algorithm
Patterns in Python – Creational
The Singleton pattern
The Singleton – do we need a Singleton?
State sharing – Borg versus Singleton
The Factory pattern
The Prototype pattern
Prototype – deep versus shallow copy
Prototype using metaclasses
Combining patterns using metaclasses
The Prototype factory
The Builder pattern
Patterns in Python – Structural
The Adapter pattern
The Facade pattern
Facades in Python
The Proxy pattern
An instance-counting proxy
Patterns in Python – Behavioral
The Iterator pattern
The Observer pattern
The State pattern
Summary
Chapter 8: Python – Architectural Patterns
Introducing MVC
Model Template View (MTV) – Django
Django admin – automated model-centric views
Flexible Microframework – Flask
Event-driven programming
Chat server and client using I/O multiplexing with the select module
Event-driven programming versus Concurrent programming
Twisted
Twisted – a simple web client
Chat Server using Twisted
Eventlet
Greenlets and Gevent
Microservice architecture
Microservice frameworks in Python
Microservices example – restaurant reservation
Microservices – advantages
Pipe and Filter architectures
Pipe and Filter in Python
Summary
Chapter 9: Deploying Python Applications
Deployability
Factors affecting Deployability
Tiers of software deployment architecture
Software deployment in Python
Packaging Python code
Pip
Virtualenv
Virtualenv and pip
Relocatable virtual environments
PyPI
Packaging and submission of an application
The init.py files
The setup.py file
Installing the package
Submitting the package to PyPI
PyPA
Remote deployments using Fabric
Remote deployments using Ansible
Managing remote daemons using Supervisor
Deployment – patterns and best practices
Summary
Chapter 10: Techniques for Debugging
Maximum subarray problem
The power of "print"
Analysis and rewrite
Timing and optimizing the code
Simple debugging tricks and techniques
Word searcher program
Word searcher program—debugging step 1
Word searcher program—debugging step 2
Word searcher program—final code
Skipping blocks of code
Stopping execution
External dependencies—using wrappers
Replacing functions with their return value/data (Mocking)
Saving to / loading data from files as cache
Saving to / loading data from memory as cache
Returning random/mock data
Logging as a debugging technique
Simple application logging
Advanced logging—logger objects
Advanced logging—custom formatting and loggers
Advanced logging—writing to syslog
Debugging tools—using debuggers
A debugging session with pdb
Pdb—similar tools
iPdb
Pdb++
Advanced debugging—tracing
The trace module
The lptrace program
System call tracing using strace
Summary
Index