Using the method developed in hep-th/0103015, we determine the non-abelian Born-Infeld action through O(alpha'^3). We start from solutions to a Yang-Mills theory which define a stable holomorphic vector bundle. In D-brane context this corresponds to BPS configurations in the limit of small background fields. Subsequently we investigate its deformation away from this limit. Through O(alpha'^2), a unique, modulo field redefinitions, solution emerges. At O(alpha'^3) we find a one-parameter family of allowed deformations. The presence of derivative terms turns out to be essential. Finally, we present a detailed comparison of our results to existing, partial results.