We propose to train trading systems and portfolios by optimizing objective functions that directly measure trading and investment performance. Rather than basing a trading system on forecasts or training via a supervised learning algorithm using labelled trading data, we train our systems using recurrent reinforcement learning (RRL) algorithms. The performance functions that we consider for reinforcement learning are proยฎt or wealth, economic utility, the Sharpe ratio and our proposed dierential Sharpe ratio. The trading and portfolio management systems require prior decisions as input in order to properly take into account the eects of transactions costs, market impact, and taxes. This temporal dependence on system state requires the use of reinforcement versions of standard recurrent learning algorithms. We present empirical results in controlled experiments that demonstrate the ecacy of some of our methods for optimizing trading systems and portfolios. For a long/short trader, we ยฎnd that maximizing the dierential Sharpe ratio yields more consistent results than maximizing proยฎts, and that both methods outperform a trading system based on forecasts that minimize MSE. We ยฎnd that portfolio traders trained to maximize the dierential Sharpe ratio achieve better risk-adjusted returns than those trained to maximize proยฎt. Finally, we provide simulation results for an S&P 500/TBill asset allocation system that demonstrate the presence of out-of-sample predictability in the monthly S&P 500 stock index for the 25 year period 1970 through 1994.