PurposeThe lack of progress toward equity in the U.S. is evident across many spheres of society, academia notwithstanding. Women academicians, in particular, face many barriers that prevent them from advancing–including a continued unsupportive climate, competing work and family demands, and interpersonal discrimination. This paper reflects on a collaborative research effort in the United States to enhance allyship for women in academia.Design/methodology/approachThe authors partnered with a major university to hold ally training for department chairs during a university-wide department chair meeting. The authors developed a methodology for creating and implementing training content using a focus-group-based training needs analysis and a diversity science grounded approach to allyship training. The authors followed this up with surveys to assess impact.FindingsParticipants indicated that they learned from the training, but participation in follow-up data collection was limited, hampering the ability to conduct rigorous quantitative analyses around intervention impact.Research limitations/implicationsAlthough the sample size may have been too limited to detect effects, the current study provides an approach that furthers the way in which researchers and practitioners can better assess the impact of allyship to women academicians.Practical implicationsPublished research on allies is very limited. The current research examines allies in the context of helping women in academia.Originality/valueDespite widespread recognition of the importance of first-line supervisors in support of diversity, limited intervention designs are available. The authors add to the extant literature on diversity interventions, while highlighting barriers to rigorous intervention evaluation.