The Definitive ANTLR Reference – Building Domain– Specific Languages

✍ Scribed by Terence Parr

• Publisher: O′Reilly
• Year: 2007
• Tongue: English
• Leaves: 369
• Category: Library

✦ Synopsis

ANTLR v3 is the most powerful, easy-to-use parser generator built to date, and represents the culmination of more than 15 years of research by Terence Parr. This book is the essential reference guide to using this completely rebuilt version of ANTLR, with its amazing new LL() parsing technology, tree construction facilities, StringTemplate code generation template engine, and sophisticated ANTLRWorks GUI development environment. Learn to use ANTLR directly from the author! ANTLR is a parser generator-a program that generates code to translate a specified input language into a nice, tidy data structure. You might think that parser generators are only used to build compilers. But in fact, programmers usually use parser generators to build translators and interpreters for domain-specific languages such as proprietary data formats, common network protocols, text processing languages, and domain-specific programming languages. Domain-specific languages are important to software development because they represent a more natural, high fidelity, robust, and maintainable means of encoding a problem than simply writing software in a general-purpose language. For example, NASA uses domain-specific command languages for space missions to improve reliability, reduce risk, reduce cost, and increase the speed of development. Even the first Apollo guidance control computer from the 1960s used a domain-specific language that supported vector computations. This book is the definitive guide to using the completely rebuilt ANTLR v3 and describes all features in detail, including the amazing new LL() parsing technology, tree construction facilities, StringTemplate code generation template engine, and sophisticated ANTLRWorks GUI development environment. You'll learn all about ANTLR grammar syntax, resolving grammar ambiguities, parser fault tolerance and error reporting, embedding actions to interpret or translate languages, building intermediate-form trees, extracting information from trees, generating source code, and how to use the ANTLR Java API.

✦ Table of Contents

Contents
Acknowledgments
Preface
Why a Completely New Version of ANTLR?
Who Is This Book For?
What's in This Book?
Introducing ANTLR and Computer Language Translation
Getting Started with ANTLR
The Big Picture
An A-mazing Analogy
Installing ANTLR
Executing ANTLR and Invoking Recognizers
ANTLRWorks Grammar Development Environment
The Nature of Computer Languages
Generating Sentences with State Machines
The Requirements for Generating Complex Language
The Tree Structure of Sentences
Enforcing Sentence Tree Structure
Ambiguous Languages
Vocabulary Symbols Are Structured Too
Recognizing Computer Language Sentences
A Quick Tour for the Impatient
Recognizing Language Syntax
Using Syntax to Drive Action Execution
Evaluating Expressions via an AST Intermediate Form
ANTLR Reference
ANTLR Grammars
Describing Languages with Formal Grammars
Overall ANTLR Grammar File Structure
Rules
Tokens Specification
Global Dynamic Attribute Scopes
Grammar Actions
ANTLR Grammar-Level Options
language Option
output Option
backtrack Option
memoize Option
tokenVocab Option
rewrite Option
superClass Option
filter Option
ASTLabelType Option
TokenLabelType Option
k Option
Attributes and Actions
Introducing Actions, Attributes, and Scopes
Grammar Actions
Token Attributes
Rule Attributes
Dynamic Attribute Scopes for Interrule Communication
References to Attributes within Actions
Tree Construction
Proper AST Structure
Implementing Abstract Syntax Trees
Default AST Construction
Constructing ASTs Using Operators
Constructing ASTs with Rewrite Rules
Tree Grammars
Moving from Parser Grammar to Tree Grammar
Building a Parser Grammar for the C- Language
Building a Tree Grammar for the C- Language
Generating Structured Text with Templates and Grammars
Why Templates Are Better Than Print Statements
Embedded Actions and Template Construction Rules
A Brief Introduction to StringTemplate
The ANTLR StringTemplate Interface
Rewriters vs. Generators
A Java Bytecode Generator Using a Tree Grammar and Templates
Rewriting the Token Buffer In-Place
Rewriting the Token Buffer with Tree Grammars
References to Template Expressions within Actions
Error Reporting and Recovery
A Parade of Errors
Enriching Error Messages during Debugging
Altering Recognizer Error Messages
Exiting the Recognizer upon First Error
Manually Specifying Exception Handlers
Errors in Lexers and Tree Parsers
Automatic Error Recovery Strategy
Understanding Predicated-LL() Grammars
LL() Parsing
The Relationship between Grammars and Recognizers
Why You Need LL()
Toward LL() from LL(k)
LL() and Automatic Arbitrary Regular Lookahead
Ambiguities and Nondeterminisms
Using Semantic and Syntactic Predicates
Syntactic Ambiguities with Semantic Predicates
Resolving Ambiguities and Nondeterminisms
Semantic Predicates
Resolving Non-LL() Conflicts
Gated Semantic Predicates Switching Rules Dynamically
Validating Semantic Predicates
Limitations on Semantic Predicate Expressions
Syntactic Predicates
How ANTLR Implements Syntactic Predicates
Using ANTLRWorks to Understand Syntactic Predicates
Nested Backtracking
Auto-backtracking
Memoization
Grammar Hazards with Syntactic Predicates
Issues with Actions and Syntactic Predicates
Bibliography
Index
Symbols
A
B
C
D
E
F
G
H
I
J
K
L
M
N
O
P
Q
R
S
T
U
V
W
Z