This paper presents a novel interface element for the geometric and material non-linear analysis of unreinforced brick-masonry structures. In the proposed modelling approach, the blocks are modelled using 3D continuum solid elements, whereas the mortar and brick-mortar interfaces are modelled by means of the 2D non-linear interface element. This enables the representation of any 3D arrangement for brick-masonry, accounting for the in-plane stacking mode and the through-thickness geometry, and importantly it allows the investigation of both the in-plane and the out-of-plane responses of unreinforced masonry panels. A co-rotational approach is employed for the interface element, which shifts the treatment of geometric non-linearity to the level of discrete entities, and enables the consideration of material non-linearity within a simplified local framework employing first-order kinematics. In this respect, the internal interface forces are modelled by means of elasto-plastic material laws based on work-softening plasticity and employing multi-surface plasticity concepts. Following the presentation of the interface element formulation details, several experimental-numerical comparisons are provided for the in-plane and out-of-plane static behaviours of brick-masonry panels. The favourable results achieved demonstrate the accuracy and the significant potential of using the developed interface element for the non-linear analysis of brick-masonry structures under extreme loading conditions.