Learning and Adaption in Multi-Agent Systems: First International Workshop, LAMAS 2005, Utrecht, The Netherlands, July 25, 2005, Revised Selected

✍ Scribed by Karl Tuyls, Pieter Jan 't Hoen, Katja Verbeeck, Sandip Sen

• Publisher: Springer
• Year: 2006
• Tongue: English
• Leaves: 224
• Series: Lecture Notes in Artificial Intelligence 3898
• Edition: 1
• Category: Library

✦ Synopsis

This book constitutes the thoroughly refereed post-proceedings of the First International Workshop on Learning and Adaption in Multi-Agent Systems, LAMAS 2005, held in Utrecht, The Netherlands, in July 2005, as an associated event of AAMAS 2005.

The 13 revised papers presented together with two invited talks were carefully reviewed and selected from the lectures given at the workshop. The papers increase awareness and interest in adaptive agent research, encourage collaboration between machine learning experts and agent system experts, and give a representative overview of current research in the area of adaptive agents.

✦ Table of Contents

Front matter......Page 1
Introduction......Page 9
Setting......Page 11
Types of Games......Page 12
Repeated Play......Page 14
Cooperative MASs......Page 17
Team Learning......Page 18
Concurrent Learning......Page 19
Credit Assignment......Page 20
The Dynamics of Learning......Page 21
Learning and Communication......Page 22
Preamble......Page 23
Design of Adaptive Software Agents for Market-Based Multi-agent Games......Page 24
MARL......Page 29
Contributions of This Book......Page 34
Open Research Issues......Page 37
Introductory Notions from (Evolutionary) Game Theory......Page 49
Strategic Games......Page 50
Minimax and Maximin......Page 51
Evolutionary Stable Strategies......Page 52
Population Dynamics......Page 53
Introduction......Page 55
Related Work......Page 56
Continuous Area Sweeping Formulation......Page 58
Learning the Expected Reward......Page 59
Choosing Actions......Page 61
Correctness of the Approach......Page 63
Multi-robot Continuous Area Sweeping......Page 64
Single Robot Results......Page 67
Single-Robot Configuration $I$ with a Real Robot......Page 68
Single-Robot Configuration $II$ with a Real Robot......Page 69
Single-Robot Simulation......Page 70
Multi-robot Configuration $I$ in Simulation......Page 73
Multi-robot Configuration $II$ in Simulation......Page 75
Conclusions......Page 77
Introduction......Page 79
Basic LA Schemes......Page 80
Illustration of LA Behavior......Page 82
Some Convergence Properties More Formally......Page 84
Automata Games Games: Single Stage, Multi-agent......Page 85
The Solution Concept......Page 86
Update Rule for Interconnected LA......Page 87
Interconnected LA and Their Relationship to Ant Colony Optimization......Page 88
Markov Games: Multi-stage, Multi Agent......Page 90
Conclusion......Page 92
Introduction......Page 94
Related Work......Page 97
Game Protocol and Learners......Page 98
Learners......Page 99
Results on the Testbed......Page 101
Results on Randomly Generated Matrices......Page 105
Conclusion and Future Work......Page 106
Introduction......Page 108
Multiagent Reinforcement Learning......Page 109
Related Work......Page 110
Our Approach: ReDVaLeR with Variable $ igma$......Page 111
Analysis of RV_$ igma(t)$......Page 112
Convergence Against Eventually Stationary Opponents......Page 113
No-Regret Property......Page 115
Convergence in Self-play......Page 116
Conclusion......Page 121
Introduction......Page 123
Social Attachments and Networks......Page 124
Description of the Information Sharing Scenarios......Page 125
Scenarios Without Recommendation......Page 128
Scenarios with Recommendation......Page 132
Final Remarks About All Scenarios......Page 134
Conclusions and Future Work......Page 135
Introduction......Page 137
Desired Properties......Page 138
The Reservation System......Page 139
Communication Protocol......Page 140
Learning Opportunities for the Intersection Manager......Page 141
Vehicles with Priorities......Page 142
Learning Opportunities for the Driver Agent......Page 143
Making Better Reservations......Page 144
Conclusion......Page 145
Introduction......Page 147
Decision Making in a Stochastic World......Page 148
Q-Learning......Page 149
The Agent......Page 150
The Learning Algorithm......Page 154
Cooperative Learning in Environments with Errors......Page 155
Related Work......Page 159
Conclusions and Future Work......Page 160
Introduction......Page 163
Related Work......Page 164
Tags in Prisoner's Dilemma and Related Games......Page 165
Effect of Tags Where Cooperation $\neq$ Mimicry......Page 169
Conclusion......Page 171
Introduction......Page 173
The Firm Model......Page 174
XCS and Adaptive Firms......Page 175
Exploration Techniques......Page 176
Wilson Techniques......Page 177
Experimental Results......Page 178
Exploration Techniques......Page 179
Exploration-Exploitation Techniques......Page 180
Conclusion......Page 183
Introduction......Page 185
Rover Reward Function Properties......Page 186
Difference Reward Functions......Page 187
Continuous Rover Problem......Page 188
Rover Capabilities......Page 189
Rover Policy Selection......Page 190
Learning Control Strategies in a Collective......Page 191
Results......Page 192
Learning Rates......Page 193
Scaling Properties......Page 194
Learning with Communication Limitations......Page 195
Leaning in Changing Episodic Environment......Page 197
Discussion......Page 198
Introduction......Page 200
Reinforcement Learning......Page 202
Relational Multi-agent Reinforcement Learning......Page 203
Complexity Factors......Page 204
Settings......Page 205
Experimental Setup......Page 208
Experimental Results......Page 210
Conclusions......Page 212
Backup Trees......Page 215
Ant Colony Optimization......Page 217
Multi-type ACO......Page 218
Our Algorithm......Page 219
Experimental Results......Page 220
The Complete Algorithm......Page 221
Conclusion......Page 222
Back matter......Page 224