Python Microservices Development

✍ Scribed by Ziadé, Tarek

• Publisher: Packt Publishing Limited
• Year: 2017
• Tongue: English
• Leaves: 478
• Category: Library

✦ Synopsis

Key Features A very useful guide for Python developers who are shifting to the new microservices-based development A concise, up-to-date guide to building efficient and lightweight microservices in Python using Flask, Tox, and other tools Learn to use Docker containers, CoreOS, and Amazon Web Services to deploy your services Book DescriptionWe often deploy our web applications into the cloud, and our code needs to interact with many third-party services. An efficient way to build applications to do this is through microservices architecture. But, in practice, it's hard to get this right due to the complexity of all the pieces interacting with each other.This book will teach you how to overcome these issues and craft applications that are built as small standard units, using all the proven best practices and avoiding the usual traps. It's a practical book: you'll build everything using Python 3 and its amazing tooling ecosystem. You will understand the principles of TDD and apply them.You will use Flask, Tox, and other tools to build your services using best practices. You will learn how to secure connections between services, and how to script Nginx using Lua to build web application firewall features such as rate limiting. You will also familiarize yourself with Docker's role in microservices, and use Docker containers, CoreOS, and Amazon Web Services to deploy your services.This book will take you on a journey, ending with the creation of a complete Python application based on microservices. By the end of the book, you will be well versed with the fundamentals of building, designing, testing, and deploying your Python microservices.What you will learn Explore what microservices are and how to design them Use Python 3, Flask, Tox, and other tools to build your services using best practices Learn how to use a TDD approach Discover how to document your microservices Configure and package your code in the best way Interact with other services Secure, monitor, and scale your services Deploy your services in Docker containers, CoreOS, and Amazon Web Services About the AuthorTarek Ziadeis a Python developer, located in the countryside near Dijon, France. He works at Mozilla in the services team. He founded a French Python user group called Afpy, and he has written several books about Python in French and English. When he is not hacking on his computer or hanging out with his family, he's spending time between his two passions, running and playing the trumpet.You can visit his personal blog (Fetchez le Python) and follow him on Twitter (@tarek_ziade). You can also take a look at one of his books on Amazon, Expert Python Programming, published by Packt.Table of Contents Understanding Microservices Working with flask Test-driven development and documentation, the virtuous cycle Designing Forrest Interacting with other services Monitoring your services Securing your services Consuming the Microservices Packaging Forrest Dockerizing your services Deploying on AWS Thinking ahead

✦ Table of Contents

Preface......Page 19
What this book covers......Page 20
What you need for this book......Page 22
Who this book is for......Page 23
Conventions......Page 24
Reader feedback......Page 26
Customer support......Page 27
Downloading the example code......Page 28
Errata......Page 29
Piracy......Page 30
Questions......Page 31
Understanding Microservices......Page 32
Origins of Service-Oriented Architecture......Page 34
The monolithic approach......Page 36
The microservice approach......Page 41
Microservice benefits......Page 44
Separation of concerns......Page 45
Smaller projects......Page 46
Scaling and deployment......Page 47
Microservices pitfalls......Page 48
Illogical splitting......Page 49
More network interactions......Page 50
Data storing and sharing......Page 51
Compatibility issues......Page 52
Testing......Page 53
Implementing microservices with Python......Page 54
The WSGI standard......Page 56
Greenlet and Gevent......Page 59
Twisted and Tornado......Page 62
asyncio......Page 64
Language performances......Page 67
Summary......Page 71
Discovering Flask......Page 72
Which Python?......Page 74
How Flask handles requests......Page 75
Routing......Page 81
Variables and converters......Page 84
The url_for function......Page 89
Request......Page 90
Response......Page 92
Flask built-in features......Page 95
The session object......Page 96
Globals......Page 97
Signals......Page 99
Extensions and middlewares......Page 101
Templates......Page 104
Configuration......Page 107
Blueprints......Page 110
Error handling and debugging......Page 112
Custom error handler......Page 113
The debug mode......Page 117
A microservice skeleton......Page 119
Summary......Page 122
Coding, Testing, and Documenting - the Virtuous Cycle......Page 123
Different kinds of tests......Page 126
Unit tests......Page 127
Functional tests......Page 132
Integration tests......Page 136
Load tests......Page 138
End-to-end tests......Page 142
Using WebTest......Page 144
Using pytest and Tox......Page 147
Developer documentation......Page 152
Continuous Integration......Page 159
Travis-CI......Page 160
ReadTheDocs......Page 162
Coveralls......Page 163
Summary......Page 166
Designing Runnerly......Page 167
The Runnerly application......Page 168
User stories......Page 169
Monolithic design......Page 171
Model......Page 172
View and Template......Page 174
Background tasks......Page 179
Strava token......Page 184
Authentication and authorization......Page 186
Putting together the monolithic design......Page 191
Splitting the monolith......Page 193
Data Service......Page 195
Using Open API 2.0......Page 197
More splitting......Page 201
Summary......Page 203
Interacting with Other Services......Page 204
Synchronous calls......Page 206
Using Session in a Flask app......Page 208
Connection pooling......Page 214
HTTP cache headers......Page 216
Improving data transfer......Page 222
GZIP compression......Page 223
Binary payloads......Page 226
Putting it together......Page 231
Asynchronous calls......Page 232
Task queues......Page 233
Topic queues......Page 234
Publish/subscribe......Page 242
RPC over AMQP......Page 243
Putting it together......Page 244
Testing......Page 245
Mocking synchronous calls......Page 246
Mocking asynchronous calls......Page 248
Mocking Celery......Page 249
Mocking other asynchronous calls......Page 252
Summary......Page 253
Monitoring Your Services......Page 254
Centralizing logs......Page 255
Setting up Graylog......Page 257
Sending logs to Graylog......Page 261
Adding extra fields......Page 264
Performance metrics......Page 266
System metrics......Page 267
Code metrics......Page 269
Web server metrics......Page 272
Summary......Page 274
Securing Your Services......Page 275
The OAuth2 protocol......Page 277
Token-based authentication......Page 279
The JWT standard......Page 280
PyJWT......Page 283
X.509 certificate-based authentication......Page 285
The TokenDealer microservice......Page 288
The POST/oauth/token implementation......Page 290
Using TokenDealer......Page 293
Web application firewall......Page 296
OpenResty - Lua and nginx......Page 299
Rate and concurrency limiting......Page 303
Other OpenResty features......Page 306
Securing your code......Page 307
Asserting incoming data......Page 308
Limiting your application scope......Page 312
Using Bandit linter......Page 314
Bringing It All Together......Page 318
Building a ReactJS dashboard......Page 320
The JSX syntax......Page 321
React components......Page 323
ReactJS and Flask......Page 329
Using Bower, npm, and Babel......Page 331
Cross-origin resource sharing......Page 337
Authentication and authorization......Page 342
Interacting with Data Service......Page 343
Getting the Strava token......Page 345
JavaScript authentication......Page 348
Packaging and Running Runnerly......Page 352
The packaging toolchain......Page 354
A few definitions......Page 356
Packaging......Page 357
The setup.py file......Page 358
The requirements.txt file......Page 365
The MANIFEST.in file......Page 370
Versioning......Page 372
Releasing......Page 375
Distributing......Page 379
Running all microservices......Page 383
Process management......Page 388
Summary......Page 393
Containerized Services......Page 394
What is Docker?......Page 396
Docker 101......Page 399
Running Flask in Docker......Page 403
The full stack - OpenResty, Circus and Flask......Page 406
OpenResty......Page 408
Circus......Page 412
Docker-based deployments......Page 418
Docker Compose......Page 421
Introduction to Clustering and Provisioning......Page 425
Summary......Page 428
Deploying on AWS......Page 429
AWS overview......Page 431
Routing - Route53, ELB, and AutoScaling......Page 433
Execution - EC2 and Lambda......Page 434
Storage - EBS, S3, RDS, ElasticCache, and CloudFront......Page 436
Messaging - SES, SQS, and SNS......Page 438
Simple Email Service (SES)......Page 439
Simple Queue Service (SQS)......Page 440
Simple Notification Service (SNS)......Page 441
Provisioning and deployment - CloudFormation and ECS......Page 442
Deploying on AWS - the basics......Page 443
Setting up your AWS account......Page 444
Deploying on EC2 with CoreOS......Page 447
Deploying with ECS......Page 454
Route53......Page 460
Summary......Page 462
What Next?......Page 463
Iterators and generators......Page 465
Coroutines......Page 469
The asyncio library......Page 471
The aiohttp framework......Page 473
Sanic......Page 474
Asynchronous versus synchronous......Page 476

✦ Subjects

Computer Science;Programming;Nonfiction;Computers