We define the concept of probabilistic aligned hypergraph bimorphism. Each such bimorphism consists of a probabilistic regular tree grammar, two hypergraph algebras in which the generated trees are interpreted, and a family of alignments between the two interpretations. It generates a set of bihypergraphs each consisting of two hypergraphs and an alignment between them; for instance, discontinuous phrase structures and non-projective dependency structures are bihypergraphs. We show an EM-training algorithm which takes a corpus of bihypergraphs and an aligned hypergraph bimorphism as input and calculates a probability assignment to the rules of the regular tree grammar such that in the limit the maximum-likelihood of the corpus is approximated.