We propose an analysis of the inverse scattering problem for chaotic Hamiltonian systems. Our main goal will be the reconstruction of the structure of the chaotic saddle from asymptotic data. We will also address the question how to obtain thermodynamic measures and a partition from these data. An essential step in achieving this is the reconstruction of the hierarchical order of the fractal structure of singularities in scattering functions solely from knowledge of asymptotic data. This provides a branching tree which coincides with the branching tree derived from the hyperbolic component of the horseshoe in the Poincare map taken in the interaction region. We achieve our goal explicitly for two types of systems governed by an external or an internal clock, respectively. Once we have achieved this goal, a discrete arbitrariness remains for the reconstruction of the horseshoe. Here symmetry considerations can help. We discuss the implications for the inverse scattering problem of the effects of finite resolution and the possible use of nonhyperbolic effects. The connection between the formal development parameter of the horseshoe and the topological entropy proves helpful in the systems discussed.