We present a gestural control system to augment harp performance with real-time control of computer-based audio affects and processing. While the lightweight system was designed for use alongside any instrument, our choice of the concert harp represented a unique case study in gestural control of music. The instrument’s large size and physically demanding playing technique leaves the performer with little spare bandwidth to devote to other tasks. A motion capture study analyzed instrumental and ancillary gestures of natural harp performances that could be mapped effectively to control of additional signal processing parameters. The initial findings of the study helped to guide the design of custom gesture control devices and user software, and a new work for solo harpist and electronics was created. We discuss our findings, successes and challenges in the study and design of gesture control for augmented instrumental performance, with particular focus on the concert harp.