Introduction to Python for Econometrics, Statistics and Data Analysis

✍ Scribed by Kevin Sheppard

• Publisher: University of Oxford
• Year: 2021
• Tongue: English
• Leaves: 427
• Category: Library

✦ Synopsis

These notes are designed for someone new to statistical computing wishing to develop a set of skills necessary to perform original research using Python. They should also be useful for students, researchers or practitioners who require a versatile platform for econometrics, statistics or general numerical analysis (e.g. numeric solutions to economic models or model simulation).

Python is a popular general purpose programming language which is well suited to a wide range of problems.

Recent developments have extended Python's range of applicability to econometrics, statistics and general numerical analysis. Python – with the right set of add-ons – is comparable to domain-specific languages such as R, MATLAB or Julia.

✦ Table of Contents

Introduction
Background
Conventions
Important Components of the Python Scientific Stack
Setup
Using Python
Exercises
Additional Installation Issues
Python 2.7 vs. 3 (and the rest)
Python 2.7 vs. 3.x
Intel Math Kernel Library and AMD's GPUOpen Libraries
Other Variants
Relevant Differences between Python 2.7 and 3
Built-in Data Types
Variable Names
Core Native Data Types
Additional Container Data Types in the Standard Library
Python and Memory Management
Exercises
Arrays and Matrices
Array
Matrix
1-dimensional Arrays
2-dimensional Arrays
Multidimensional Arrays
Concatenation
Accessing Elements of an Array
Slicing and Memory Management
import and Modules
Calling Functions
Exercises
Basic Math
Operators
Broadcasting
Addition (+) and Subtraction (-)
Multiplication ()
Matrix Multiplication (@)
Array and Matrix Division (/)
Exponentiation ()
Parentheses
Transpose
Operator Precedence
Exercises
Basic Functions and Numerical Indexing
Generating Arrays and Matrices
Rounding
Mathematics
Complex Values
Set Functions
Sorting and Extreme Values
Nan Functions
Functions and Methods/Properties
Exercises
Special Arrays
Exercises
Array and Matrix Functions
Views
Shape Information and Transformation
Linear Algebra Functions
Exercises
Importing and Exporting Data
Importing Data using pandas
Importing Data without pandas
Saving or Exporting Data using pandas
Saving or Exporting Data without pandas
Exercises
Inf, NaN and Numeric Limits
inf and NaN
Floating point precision
Exercises
Logical Operators and Find
>, >=, <, <=, ==, !=
and, or, not and xor
Multiple tests
is
Exercises
Advanced Selection and Assignment
Numerical Indexing
Logical Indexing
Performance Considerations and Memory Management
Assignment with Broadcasting
Exercises
Flow Control, Loops and Exception Handling
Whitespace and Flow Control
if … elif … else
for
while
try … except
List Comprehensions
Tuple, Dictionary and Set Comprehensions
Exercises
Dates and Times
Creating Dates and Times
Dates Mathematics
Numpy
Graphics
seaborn
2D Plotting
Advanced 2D Plotting
3D Plotting
General Plotting Functions
Exporting Plots
Exercises
pandas
Data Structures
Statistical Functions
Time-series Data
Importing and Exporting Data
Graphics
Examples
Structured Arrays
Mixed Arrays with Column Names
Record Arrays
Custom Function and Modules
Functions
Variable Scope
Example: Least Squares with Newey-West Covariance
Anonymous Functions
Modules
Packages
PYTHONPATH
Python Coding Conventions
Exercises
Listing of econometrics.py
Probability and Statistics Functions
Simulating Random Variables
Simulation and Random Number Generation
Statistics Functions
Continuous Random Variables
Select Statistics Functions
Select Statistical Tests
Exercises
Statistical Analysis with statsmodels
Regression
Non-linear Function Optimization
Unconstrained Optimization
Derivative-free Optimization
Constrained Optimization
Scalar Function Minimization
Nonlinear Least Squares
Exercises
String Manipulation
String Building
String Functions
Formatting Numbers
Regular Expressions
Safe Conversion of Strings
File System Operations
Changing the Working Directory
Creating and Deleting Directories
Listing the Contents of a Directory
Copying, Moving and Deleting Files
Executing Other Programs
Creating and Opening Archives
Reading and Writing Files
Exercises
Performance and Code Optimization
Getting Started
Timing Code
Vectorize to Avoid Unnecessary Loops
Alter the loop dimensions
Utilize Broadcasting
Use In-place Assignment
Avoid Allocating Memory
Inline Frequent Function Calls
Consider Data Locality in Arrays
Profile Long Running Functions
Exercises
Improving Performance using Numba
Quick Start
Supported Python Features
Supported NumPy Features
Diagnosing Performance Issues
Replacing Python function with C functions
Other Features of Numba
Exercises
Improving Performance using Cython
Diagnosing Performance Issues
Interfacing with External Code
Exercises
Executing Code in Parallel
map and related functions
multiprocessing
joblib
IPython's Parallel Cluster
Converting a Serial Program to Parallel
Other Concerns when executing in Parallel
Object-Oriented Programming (OOP)
Introduction
Class basics
Building a class for Autoregressions
Exercises
Other Interesting Python Packages
scikit-learn
mlpy
NLTK
pymc
pystan
pytz and babel
rpy2
PyTables and h5py
Theano
Examples
Estimating the Parameters of a GARCH Model
Estimating the Risk Premia using Fama-MacBeth Regressions
Estimating the Risk Premia using GMM
Outputting LaTeX
Quick Reference
Built-ins
NumPy (numpy)
SciPy
Matplotlib
pandas
IPython