We are interested in group extensions 1 --> N --> E --> F --> 1, for which the corresponding abstract kernel F --> Out(N) is faithful. For these groups E, we develop commutative diagrams which are helpful to understand and to compute Aut(E, N) (the group of all E-automorphisms mapping N into itself) and Out(E, N) = Aut(E, N)/Inn(E). Of course, if N is characteristic in E, Aut(E, N) = Aut(E) and Out(E,N) = Out(E). These conditions occur e.g. when studying almost crystallographic groups, which were in fact the initiating cases to us. Similar work has been done previously by Conner and Raymond () (several types where N = Z(k)) and by Charlap () (for crystallographic groups). Although the approach in both works is rather different, we did an effort to obtain a description covering most aspects of both previously developed pictures. We include the results of an example computation for one family of isomorphism types of 3-dimensional almost crystallographic groups. For K(E, 1)-manifolds it is known that Out(E) has an important geometric meaning. In the closing section and for certain K(E, 1)-manifolds, we establish a geometric interpretation of Out(E, N) and its subgroups from the determining diagrams.