A science education model for large collaborative centers

Scientific breakthroughs often require the collaborative efforts of researchers from several fields of study working together toward a common goal.¹ This has been recognized by national funding agencies, which fund several Center or Institute programs designed to promote collaboration and advance research and train the next generation of scientists. The majority of these funding calls require a significant investment in educational programing and outreach, but rarely provide specific examples or guidance on how to develop supplemental education programs. We describe here a model education program that addresses many issues faced by large, interdisciplinary Centers and provide examples of the design, implementation, and outcomes of relevant initiatives developed for a National Science Foundation (NSF) Science and Technology Center.

The emergence of new experimental techniques utilizing X-ray Free Electron Lasers (XFELs), such as time-resolved serial femtosecond crystallography (SFX)^2,3 and single particle imaging, has had a profound impact on the scientific understanding of dynamic biological systems.^4,5 These achievements have led to advancements in many fields from biochemistry to physics, enabling scientists to visualize catalytic reactions in real time at atomic resolution⁶ and reconstruct 3D images of single particles.^7,8 Successful utilization of XFELs requires a broad skill set, typically delivered by a diverse team of researchers. These teams, with members trained in a wide range of disciplines, are required to communicate and collaborate using a common language to effectively manage the team, coordinate experiments, and interpret results. As a highly interdisciplinary field, XFEL science presents unique challenges in preparing the next generation of scientists.

Here, we present a progressive education program developed by the NSF Science and Technology Center (STC), Biology with X-ray Free Electron Lasers (BioXFEL). The scientific goal of BioXFEL is to invent, discover, develop, and provide new tools and training to bring X-ray laser technology to the wider scientific community, specifically to study the structure and dynamics of biological macromolecules. The Center is composed of several institutions distributed across the country, including the State University of New York at Buffalo (UB), Arizona State University (ASU), the University of Wisconsin-Milwaukee (UWM), the University of California-San Francisco (UCSF), Rice University, Cornell University, Stanford University, Hauptman-Woodward Medical Research Institute (HWI), and additional collaborators from universities and national laboratories in the US and around the world. The distributed nature of the Center creates challenges in coordinating and managing activities and in providing a uniform educational experience. BioXFEL provides an education program that is supplemental to students' coursework and is intended to fill knowledge and skill gaps in XFEL science by offering additional scientific training and professional development that are not available in traditional undergraduate or graduate programs. This is accomplished through upskilling and training in cutting-edge techniques delivered through the initiatives described here.

We provide here specific examples of customized education initiatives and activities that are part of a comprehensive model designed for BioXFEL undergraduate and graduate students, postdoctoral associates, and young scientists, along with evidence that supports claims of effectiveness. Additionally, BioXFEL routinely leads outreach activities for primary and high school students. The main educational goal of the Center, at every level, is to ensure that our scholars are academically and professionally prepared for their current responsibilities and future career aspirations. However, we recognize that empowering a diverse cohort of future XFEL science leaders benefits not only our Center but also the broader scientific community through workforce development. By addressing specific challenges related to designing and implementing our educational programing model, we offer a roadmap and general best practices for large, center-scale education programs and individual laboratories that are interested in designing similar training and professional development initiatives.

Paramount to accomplishing the BioXFEL mission are the contributions of young BioXFEL scholars. Graduate and undergraduate students whose interest span a range of STEM fields are involved in Center efforts. With this in mind, Center education components have been designed to be flexible, unobtrusive, and tailored to the specific needs of scholars at different stages in their development. Center educational and professional development activities enhance scholars' opportunities and preparation for scientific careers, while supporting their full participation in Center research endeavors.

In order to develop an effective Center-level education program, it is necessary to integrate education and diversity efforts with the Center's research and scientific aims. While collaboration across Center entities is necessary to implement an effective and cohesive education program, central leadership is essential for planning, coordinating, and monitoring Center-level education programs. Assigning responsibility for education to an individual or committee demonstrates commitment to education programing that is commensurate with the Center's focus on scientific research. Further, developing and enacting structures and processes for monitoring and assessing education programing facilitate responsiveness to students' emerging needs, improve the effectiveness of programing, and contribute to more positive outcomes over time.

Since its inception in 2013, BioXFEL has developed a customized education program designed to meet the specific needs of our scholars, prepare them for careers in related fields, and address obstacles inherent to distributed, interdisciplinary Centers. Centers experience both challenges and opportunities in designing education and professional development programs for their students. Educational and professional development opportunities afforded by participation in a large research Center must be balanced by the recognition of the students' important role in advancing the Center's scientific endeavors. To develop responsive programing, it was essential to elicit participant feedback and incorporate suggestions that improved the experiences and outcomes of the initiatives (Fig. 1). Initial reactions and comments on initiative effectiveness and relevance were gathered through post-event surveys and informal inquiry. Surveys not only solicit feedback on events but also collect data about participants' prior experiences and their recommendations for future events and topics. This feedback is then presented to our Education and Diversity Committee, which consists of key stakeholders in Center leadership, and student and postdoctoral representatives. Initiatives are then optimized to enhance participation, positive experiences, and specific desired outcomes. Evaluation of the BioXFEL education program is performed primarily by an experienced external evaluator, through the use of confidential surveys, in person interviews, observations, and informal conversations. External evaluators are a key component of the initiative development and modification process as they provide honest, unbiased reports that are used to make improvements to Center activities. All of this information is then applied in an iterative process of modifying our education program or developing new initiatives as needed.

While this article presents a number of unique education initiatives, two key elements form the foundation of our education programs—mentoring and inclusivity. Our approaches to these elements reflect our desire to not only retain students but also to empower all students to achieve maximum productivity.

In addition to scholars' institutional faculty advisors, who are often Center PIs or co-PIs, the Center provides opportunities for scholars to receive holistic individualized mentoring.^9,10 According to a survey of nearly 6000 doctoral students, mentorship contributed more to respondents' overall satisfaction with their PhD programs than did any other factor; specifically, guidance from, and recognition by, the mentor was the primary determinant of student satisfaction.¹¹ BioXFEL scholars' opportunities for mentoring extend beyond the boundaries of their home labs and departments. All student and postdoctoral scholars are required to utilize our Individual Development Plan (IDP) mentoring system²⁸ in order to receive travel support from the Center. Each scholar is given the opportunity to identify a second Center investigator as a mentor, in addition to their assigned institutional mentor. More than 70% of Center graduate students reported that they had more than one mentor, and more than 60% of these students reported that they had at least one mentor from outside of their primary discipline.^12,13 Additionally, networking opportunities provided by Center events (i.e., annual conference, workshops) facilitate scholars' connections to a broad range of peers and senior researchers who can provide different types of support.

Each scholar also receives support from the Education and Diversity Director, which includes sponsorship activities. By providing sufficient sponsorship to student and postdoc participants, our Center has significantly increased the likelihood of trainees' long-term success and our contributions to workforce development in the field of XFEL science. Sponsorship takes many forms but is broadly defined as advocating for a person's career advancement, for example, by removing barriers to entry into the next phase of their careers.¹⁴ There are many examples of sponsorship in academics, including providing assistance in writing applications and proposals, supplying letters of recommendation, supporting pursuit of professional development, and engaging in objective discussions about career options. Beyond this, dedicated mentors also assist in funding critical activities, such as supplementing students' travel and registration costs, including trainees in grant proposals, publications, and actively promoting networking and career advancement. Continuation of sponsorship after scholars have exited the Center is critical to ensure they remain engaged and progress toward successful careers that contribute to the field.

Given the significance of mentoring in preparing scholars for future success,^15,16 our Center evaluator collects and reports data on students' perceptions of Center-level mentoring that provides information on site- and program-level strengths and challenges to mentoring that can be discussed and addressed by the Education and Diversity Committee and Center leadership. The effectiveness of our approach to mentoring is supported by evaluation findings that suggest graduate students' satisfaction with, expectations of, and extent to which their needs are met by Center mentoring are consistently high across all years. Longitudinal analysis of data found that scholars reported more positively on several aspects of mentoring they received over time, including better access to mentors and more career guidance.¹⁷ These data emphasize that mentoring programs must be flexible and responsive to students' needs as they progress through their graduate programs.

Individualized, holistic mentoring also includes attention to student characteristics that may influence the mentoring they receive. Findings of many national surveys of graduate students suggest that women and graduate students from underrepresented minority groups (URM) were consistently less positive about their mentoring experiences than were their white male counterparts.^18,19 While Center scholars' satisfaction is similarly high for all groups of students regardless of gender or race/ethnicity, our evaluation monitors differences among groups in the types of mentoring they receive.

A second overarching approach to our education program is attention to inclusivity for multiple forms of diversity. Participation of women and individuals from underrepresented groups in BioXFEL has increased significantly from Center inception. Our percentage composition of women has increased overall by nearly 20%, while the percentage of participants who are members of groups underrepresented in XFEL-related sciences has increased by more than 30%.¹² The participation of women graduate students in BioXFEL is 43%, slightly higher than the national average in XFEL-related science fields (i.e., 35% nationally for biological sciences, physical sciences, mathematics, engineering), while the participation of individuals from groups underrepresented in the sciences (i.e., African American/Black, Hispanic/Latino(a), American Indian/Alaska Native) reached a high of 37.5% in 2019, significantly higher than the national average of 20% at the graduate student level for XFEL fields.^20,21

Our Center uses both formal and informal strategies for monitoring the inclusivity of education programing. Interviews and surveys conducted by the external evaluator elicit students' perspectives on issues related to diversity and inclusivity. Findings indicate that BioXFEL scholars believe that the Center actively supports diversity and provides an inclusive environment for all. Scholars largely agreed that the Center had established a positive working environment for women and URM scholars. BioXFEL women and URM scholars have consistently reported more positive perceptions of relationships and inclusiveness of the Center environment than did non-URM and men scholars. Additionally, the Director of the Education and Diversity Committee plays a major role in designing and leading initiatives that increase the Centers' diversity and in monitoring education programing to ensure the needs of diverse students are met.^12,17

The BioXFEL education program consists of a set of progressive, customized initiatives aimed at educating and developing the next generation of researchers in XFEL science. This comprehensive set of initiatives has gone through several rounds of refinement and is summarized in Fig. 2. These are displayed along a timeline that represents a typical scholar's progression through the education program with a delineation made between continuous and intermittent activities.

Our most highly subscribed initiative is our undergraduate summer internship program, which serves as the gateway to our Center recruitment and training pipeline. While it is standard practice to incorporate a summer internship program into a Center, several factors should be considered to ensure the program is aligned with Center needs. The first step is recruiting a diverse pool of qualified applicants with career interest aligned to the program. This can typically be achieved through free, online advertising via relevant scientific societies and associations. Most notably, the Institute for Broadening Participation (IBP)²⁸ offers a comprehensive list of opportunities available to underserved groups. Using IBP's free internship posting service, as well as those provided by the American Chemical Society (ACS), the Society for Physics Students (SPS) among others, over the first seven years, BioXFEL achieved an average of 172 applicants per year, 32% of whom were from underrepresented minority groups (URM) and 52% were female. In total, this resulted in 124 summer interns, including 68 URM (55%) and 66 female (53%) interns, well above the national average for similar programs. The 10-week summer experience includes individualized research projects, hands-on laboratory and safety training, a scientific communication workshop, an introductory science seminar series, and a career development seminar series and concludes with a live research presentation to the Center and broader scientific community. Interns are typically distributed across four of our sites and receive customized projects with a hands-on training component in the laboratory that involves individualized holistic mentoring for each student.

Interns' responses to follow-up evaluation surveys consistently demonstrate the program's effectiveness in producing large gains in several areas related to students' general research behaviors, knowledge, skills, and dispositions, including the improved ability to analyze data and interpret results, to integrate theory and practice, learning new laboratory techniques, developing readiness for more demanding research, understanding the research process and how scientists construct knowledge in their fields, learning to work independently, and improved tolerance for obstacles faced in the research process. Across seven years of the internship program, interns reported on learning outcomes specifically related to XFEL science (e.g., academic preparation necessary for XFEL science) and indicated that the summer program made them more excited about studying science in college and interested in learning more about XFEL science.¹² Of the summer interns who have graduated, 12 have joined graduate programs within the Center and 19 completed a second internship with BioXFEL, typically in a different laboratory. Many others have transitioned to graduate STEM programs external to the Center (19) or MD/PhD programs (5).

While the COVID-19 pandemic necessitated a shift to an all-remote program in 2020, the Center managed to retain key components of the internship program. Interns were given research projects that were more focused on data analysis and algorithm development, while wet laboratory experience was transferred to the bioinformatic analysis of coronavirus targets with seminars and video demonstrations of common lab techniques. All interns are invited to attend and present posters at the annual international BioXFEL conference alongside graduate students and postdoctoral associates of the Center.

Our annual international conference brings together more than 150 researchers from around the world to discuss recent advances in the field of XFEL science. This event provides an ideal opportunity for our younger researchers to present their results and network, interact with the community, and share ideas. Inclusion of all groups is facilitated through the award of conference travel scholarships, which support the attendance of students from minority-serving institutions (MSIs). Every conference also includes professional development (PD) events to provide a common training experience for all scholars and social activities to promote collaboration and communication. Past PD events have included grant writing seminars, nontraditional career panels, lab management workshops, and several scientific workshops that facilitate cross-training in disciplines of the Center and bring together scholars from all of sites.

Findings from post-conference surveys indicate that students view the conference as a significant professional development opportunity that enhances their knowledge, skills, and confidence as novice researchers. A key benefit of conference attendance noted by a majority of students was networking with researchers and other students. Interviewed students frequently reported that their conference attendance had catalyzed a range of positive professional experiences for them.²¹ 

BioXFEL has hosted a number of in-person scientific workshops that deliver fundamental training and promote skill development in areas critical to our research. The Center has sponsored over 30 workshops in seven years, covering a broad range of topics, including protein production and purification, biomolecule crystallization, sample delivery techniques, and XFEL data analysis. The majority of these workshops have been developed based upon requests from our participants and deliver a common experience that educates the scientific community on research topics essential to our field. These specialized workshops cover cutting-edge techniques employed by our researchers and include skill development via hands-on training that is typically not found in standard coursework. On an annual basis, BioXFEL offers at least one workshop for researchers on the University of Puerto Rico (UPR) campuses. Post-event surveys often yield valuable feedback on suggested topics for future workshops, changes in participants' levels of expertise, and the extent to which expected learning outcomes were met, as well as opinions on meeting venues and delivery formats. Overall workshop participants report that the events increase their knowledge in the subject area and their interest in learning more about Center research and education.²² 

Further cross-training opportunities are made available through our cross-training scholarship and our external conference scholarship. The cross-training scholarship is available to all scholars to support their training at a partner institution or government laboratory and is awarded through a competitive application process. The Center funds travel and other costs necessary to receive training in a new skill or technique related to the student's primary research project that will advance their research or professional marketability. Our external conference scholarships are used by scholars to leverage training, including workshops, conferences, or courses offered by external institutions. Together, these scholarships, along with BioXFEL-sponsored workshops, provide ample opportunity to develop scholars' skills and enhance productivity.

BioXFEL further supports its scholars through a competitive postdoctoral fellowship that funds a unique project for a postdoctoral scholar within the Center. Following application and review processes similar to NSF's submission and review criteria introduces postdoctoral scholars to the rigors of proposal writing. Reviewers provide extensive, individualized feedback, beyond what is typically provided by grant referees, in order to help scholars improve their proposal writing skills. Fellowships are awarded based on the merit of the proposed project and its alignment with the scientific goals of the Center. At the conclusion of the award period, awardees give an oral presentation of their results at our annual conference, providing additional opportunity to showcase their work with other researchers who may be key to the next step of their careers. Evidence of the impact of our postdoctoral fellowship program is shown in the outcomes of awardees who have obtained faculty or industry positions (i.e., University of California, Merced, Kimball Physics) following completion of their projects.

The COVID-19 pandemic has made apparent the importance of having free and accessible online educational materials. Fortunately, BioXFEL was well-prepared for this paradigm shift as the Center had already made a considerable investment in the development and publication of online materials. BioXFEL has collected recordings of lectures, seminars, workshops, and journal clubs and provides a customized playlist of educational scientific videos hosted by the Journal of Visualized Experiments (JoVE). All of these materials are available through our website (www.bioxfel.org) and YouTube channel (www.youtube.com/bioxfel). To date, we have over 100 YouTube videos with more than 100 000 total views, a collection representing a body of knowledge that will exist beyond the life of the Center through continued support of participating institutions.

As outlined in Fig. 2, the BioXFEL Education model strategy is to engage larger audiences at younger ages to reach as many early scientists as possible. During outreach events, students are recruited by presenting opportunities to engage with our Center. Those who are interested will apply and may continue on through the BioXFEL programing. Outreach events consist of a combination of live, on-site events and online activities and have included participation in the National US Science and Engineering Festival, and many local events at our partner institutions. Outreach to the general public has been achieved through museum displays, open house events and tours, and general interest articles in popular magazines (i.e., publication in Scientific American²³). We also created two educational software tools to teach the fundamentals of X-ray diffraction and SFX to younger audiences. X-rayView,²⁴ developed by BioXFEL partners at Rice University, is an interactive module that examines diffraction theory as it relates to traditional and serial crystallography. The second piece of software, XFEL Crystal Blaster, is an educational game available through iTunes and Google Play. Developed by a summer undergraduate intern at UB, this application provides basic information about SFX experiments and protein structure in a fun, interactive game. These free software programs provide a unique, interactive platform to engage students in XFEL science and have been distributed to thousands of students.

While the sponsors and scientific aims of large scientific groups vary, students' needs for financial, professional development, and educational support are universal.^25–27 The education program described here represents a model system that can be applied to other Centers, institutions, or individual laboratories. While the specific examples provided were designed to meet the needs of BioXFEL members and Center goals, these initiatives may be adapted to new groups, as outlined in the Designing Effective Initiatives section. It should be noted that it is often more cost effective, and expected by funding agencies, to leverage existing resources and collaborations when creating new opportunities for student support. As examples, making use of university resources (such as student training and support services, fellowship and scholarship offices) and collaborating with other groups funded by the same agencies are effective strategies to achieve greater impact. An increase in remote learning during the COVID-19 pandemic has resulted in a massive expansion in online education and training materials, as well as a societal shift in the ability of institutions to provide resources for accessing content remotely. Professional associations and organizations with interest aligned with the scientific aims of the Center should be explored as potential sources of online education materials, live online training events, and professional development resources.

We would like to thank the BioXFEL scholars for their assistance in developing and refining this education program and its initiatives. Special thanks are due to Thomas Grant, Emily Ryan, and Erin Uppington for helpful review of this editorial and to the BioXFEL Education & Diversity Committee: George Phillips, Chair (Rice University), Edward Snell (HWI/UB), William Bauer (HWI), John Spence (ASU), Russell Fung (UWM), Thomas Grant (UB), Sabine Botha (ASU), Brenda Hogue (ASU), Darya Marchany (UPR Mayagüez/ASU), Abbas Ourmazd (UWM), Sharah Yasharahla (Howard University), Sarah Woodruff (Miami University), and Eaton Lattman (UB/HWI emeritus) for their valued input. This work was supported by funding from the National Science Foundation through a Science and Technology Center Grant No. 1231306.

Data that support the findings of this study are available from the corresponding author upon reasonable request.