Historical Study in the U.S.: Assessing the impact of Tuning within a professional disciplinary society

Daniel J. McInerney[*]

doi: http://dx.doi.org/10.18543/tjhe-6(1)-2018pp21-67

Received: 03.04.2018
Accepted: 12.11.2018

Abstract: The U.S.-based American Historical Association (AHA), the largest – and most influential – professional organization for historians, was the first disciplinary society in the world to lead a Tuning project, launching its work in 2012. This essay analyzes a survey distributed to historians on campuses that have taken part in the AHA Tuning project. The purpose is to understand, after six years of work on the project, what practical difference Tuning has made for historians, students, courses, curricula, and departments. Survey data indicate that, under the disciplinary society’s guidance and encouragement, historians have created meaningful learning outcomes, implemented the objectives in courses and curricula, and begun work in the measurement of student learning. Not surprisingly, the project has faced limits and obstacles, particularly with leadership of the work, faculty buy-in, administrative support, follow-up assistance, enrollment concerns, student engagement, and outreach to stakeholders. However, after half a dozen years of activity, U.S. historians have made marked progress not only in articulating disciplinary learning outcomes (as have colleagues in other parts of the world) but also in implementing and assessing those objectives. While precise readings of “impact” remain elusive, a Tuning project under the direction of a disciplinary society has helped generate significant pedagogical, curricular, and cultural changes in the field of history.

Keywords: Tuning; history; American Historical Association; AHA Tuning; disciplinary society; learning outcomes; assessment; US higher education; impact study

I. Introduction: Tuning the discipline of history around the world – and in the U.S.

The October 2017 special issue of Arts and Humanities in Higher Education examined “Tuning History.” Editors Alan Booth and David Ludvigsson gathered articles from colleagues in Europe, Latin America, and the U.S. reflecting on the work of Tuning in the discipline of history since 2000. Overall, the contributors offered a mixed report, one that was promising but cautious. Cooperative work on the initiative has proceeded at a steady pace – but still confronts confusion over the Tuning process and resistance to the reform effort. Disciplinary specialists have come together to clarify shared learning outcomes – yet they recognize significant differences in traditional national approaches to the subject area. The movement toward student-centered learning has gained support – though often without a systematic reappraisal of pedagogy. And faculty have crafted compelling declarations on disciplinary competences – however the statements remain, in some areas, quite removed from teaching practice. The process, Booth and Ludvigsson concluded, has been “both stimulating and challenging.”[1]

Whatever limits and constraints have operated in Tuning the field of history, the work continues to expand and become more refined. Colleagues have recently outlined in even greater detail guidelines for developing competencies and evaluating learning in five key subject areas, including history.[2] When considering the key figures in both past and continuing work on Tuning, who has played a central role? Ann Katherine Isaacs of the University of Pisa reminds us that “historians, working together, created, led, and lead Tuning,” demonstrating that the knowledge, skills, and abilities of the discipline “are useful in fields far broader than those of traditional historical teaching and research.”[3]

Reporting on the experience of French historians in the AHHE special issue, Jean-Luc Lamboley discussed the “unfavourable context” in which Tuning unfolded. Encountering more “indifference” than active resistance, the project has operated within an academic culture characterized by content-based approaches to learning, research-focused careers of professors, independent-minded aversion to business links, and time-constrained complaints of initiative fatigue. Despite these challenges, a promising, competence-based framework for history education has emerged.[4] Marco Velázquez Albo’s description of the two-phased Latin American Tuning project (2004-2007, 2011-2013) pointed to difficulties with “initial resistance” that grew out of national economic disparities, diverse methodological approaches, generational differences, limited attention to pedagogy, and deep concerns over “globalization.” Yet faculty participants displayed a strong sense of cooperation over shared competences and a greater awareness of “the importance given to teaching and learning history.”[5]

The special issue also included my report on Tuning history in the U.S., an overview that explained how effective engagement in the process has been accompanied by “confusion, hesitancy, and resistance.”[6] The present article’s survey of historians’ Tuning activity will show that disciplinary colleagues have made considerable progress by moving Tuning from stages of articulation to implementation to assessment. Yet, as in France and Latin America, work in the States has proceeded in “fits and starts.” Most U.S. history faculty came to the project with little formal pedagogical training and a lack of familiarity with outcomes-based approaches to learning. The core language of the project remained mysterious to many historians. Few had experience with discussions across multiple campuses – or in conversations with various “stakeholders.” Some were deeply concerned about intrusions on academic freedom and the seemingly “instrumentalist” framework of Tuning. And the prospect that the initiative might renew debates over “history standards” (bringing back memories of an intense dispute in the 1990s) left numerous colleagues uneasy.

As Pablo Beneitone and Edurne Bartolomé have noted in other regions,[7] these “cultural” concerns often combine with local educational structures to shape – and at times complicate – the work of Tuning. Such is the case in the U.S. which operates in a highly decentralized environment with no single, national administrative office for post-secondary education to guide projects such as Tuning. Among other distinctive national characteristics, a third of all undergraduates in the States attend one of 1600 two-year campuses that provide a strikingly diverse student body with both academic and vocational programs. Much of the teaching at two- and four-year institutions focuses on “General Education” programs that introduce U.S. students (regardless of their major areas of study) to a breadth of study across multiple disciplines. Two-thirds of the instructors that students meet in classes work as contract, “on-tenure track” faculty, colleagues who often find themselves removed from institutional discussions of pedagogy, learning, and curricula. And in this dispersed and varied world of higher education, the funding for pedagogical and administrative initiatives commonly comes from private foundations rather than public sources. Philanthropic agencies have been responsive, generous, and patient with the higher education community – although their “strategic plans” and objectives differ widely (and can change rather quickly) contributing, at times, to shifting or overlapping agendas for academic reform.[8]

One such agency, the Lumina Foundation, took the lead in sponsoring research and funding for Tuning in the U.S. Informed by the late Dr. Clifford Adelman’s studies on the Bologna Process and Tuning,[9] Lumina (along with the William and Flora Hewitt Foundation), initiated “Tuning USA” in December 2008.[10] Work began on the project in Spring 2009 in three state systems of public higher education. Lumina expanded the work to include additional states and a regional consortium of campuses in the nation’s midsection.[11]

The foundation then moved in a unique direction. Recognizing Tuning as a discipline-based, faculty-led initiative suited to all types of institutions and all levels of student learning, Lumina funded a project under the leadership of a fitting organization: a nationwide, faculty-focused, professional disciplinary society, the American Historical Association. The AHA’s members occupy positions on nearly all U.S. campuses; historians have maintained a strong presence in “Gen Ed” programs; and the organization has a continuing interest in questions of teaching and learning and in the needs of non-tenure-track instructors. In February 2012, Lumina officially announced its new “AHA Tuning” project.[12]

I.1. The American Historical Association Tuning Project

Founded in 1884, the American Historical Association is the largest, most prominent and prestigious disciplinary organization for historians in the United States. Representing nearly 13,000 members drawn from all areas of specialization and faculty ranks, the AHA produces the field’s most respected journal (the American Historical Review), holds an annual conference attracting 4,000-5,000 attendees, and organizes regional meetings held across the nation throughout the year. The AHA clarifies the key reference points and “best practices” for the discipline on issues tied to research, teaching, hiring, and ethical expectations while also performing the important work of public advocacy on controversial issues.[13] The society has helped shape curriculum at all educational levels, recognized excellence in teaching, guided the preparation and selection of history textbooks, and reported on pedagogical approaches, learning outcomes, digital history, liberal education, and career opportunities for graduates.[14]

The AHA’s Tuning work began in two stages. Initially, a small “project leadership core” of faculty, AHA officers, and consultants gathered at a retreat to discuss shared commitments to the discipline.[15] The resulting “Discipline Core” of historical study described “the skills, knowledge, and habits of mind that students develop in history courses and degree programs.” Embracing Tuning’s commitment to diversity and autonomy, the Core served as “a reference point,” not as a set of fixed requirements for history departments and curricula. The group distributed the document to AHA members for comment and revision.[16]

In the second stage of the project, the AHA invited historians from around the nation (and a range of institutions) to take part in Tuning workshops. The meetings introduced participants to a range of topics: reviewing Tuning basics; forming common goals; “aligning” courses and curricula with the “Discipline Core”; “scaling” learning outcomes to different course levels and degrees; assessing student learning; and clarifying the role of history in lifelong learning, careers, and active citizenship. The AHA expected participants to carry the Tuning project back to their home campuses in discussions with faculty, students, staff, administrators, and stakeholders. Dr. James Grossman, AHA executive director, explained Tuning as “part of the AHA’s emphasis on facilitating communication among historians and between historians and the general public.” The goal was to “generate curricular frameworks that combine common themes and practices with the flexibility appropriate for institutions with different missions and circumstances.”[17]

The project began with 60 participants (out of 120 applicants) who formed the first “AHA Tuning Cohort.” Their work in 2012 and 2013 proved so successful – and generated such wide interest – that the AHA organized a second “cohort” in 2014. Altogether, the two groups of Tuning participants totaled nearly 160 historians on over 120 campuses.[18]

Discussions of Tuning continued outside the cohorts in the AHA’s annual meetings and regional conferences, essays in the society’s Perspectives on History magazine, blog posts, discussion groups, and journal articles.[19] As the work expanded, so did engagement in issues closely tied to Tuning: the pedagogical and career training of doctoral students; the work of faculty with non-majors and history General Education courses; the common interests of college faculty and high school history teachers; and the connections between outcomes-based projects in the discipline and similar reforms across entire degrees.[20] Almost immediately, the work of Tuning opened up a wider field of activity than anyone had first anticipated, building a coherent, convergent, integrated “network” of affiliated reform initiatives in teaching and learning.

II. The Survey

A series of reports examining the organization and effects of Tuning in the U.S. (and in the American Historical Association) have appeared since 2009. All of the state and regional higher education systems involved in Tuning– as well as the American Historical Association –submitted periodic “grantee” reviews to the Lumina Foundation detailing the progress of their work.[21] The AHA also collected reports from members of its two Tuning cohorts about the work of the project on their campuses.[22] Colleagues with the National Institute for Learning Outcomes Assessment also released a Tuning Impact Study that draws from interviews, reports, observations, conference presentations, and journal articles.[23]

Yet practical questions about the way Tuning operates “on the ground” remain missing. How have “rank-and-file” faculty responded to Tuning – especially, the campus instructors who were not in Tuning leadership or the intensive “cohorts,” who learned about Tuning at second- or third-hand, but who are in classes day-to-day with heavy teaching responsibilities? To what extent has Tuning changed course design, teaching techniques, class assignments, curricular arrangements, assessment strategies, transfer policies, accreditation reviews, and engagement in the scholarship of teaching and learning? How have the majority of historians considered, applied, questioned, or ignored the core elements of the project? And how have history departments – the core administrative unit for the discipline in most U.S. institutions – responded to the AHA project? After six years of work, what practical difference has Tuning made in historical study? In a word, as Booth and Ludvigsson have also concluded, the work has been “both stimulating and challenging,” demonstrating a considerable (though circumscribed) advance in disciplinary activity on outcomes-based, student-centered teaching and learning.

II.1. Structure of the survey

The online AHA Tuning survey contained 42 questions with an estimated completion time of 12-15 minutes. The intended audience was college-level history instructors of all ranks teaching at institutions that sent representatives to the AHA Tuning “cohorts.” The material, formatted in Qualtrics software, went out through email to 1500 faculty members on 70 different campuses (nearly 60% of the institutions with cohort participants).

The survey opened with a review of the AHA Tuning project and previewed the questions.

• Part I: personal and professional information about each respondent.

• Part II: activities with AHA Tuning and related projects in history departments

• Part III: curricular/course design changes/Tuning’s perceived level of influence on changes

• Part IV: additional question on changes in teaching and learning.

• Part V: Tuning’s perceived influence over assessment of student learning.

• Part VI: Tuning’s effects on course enrollments and numbers of majors.

• Part VII: benefits and problems participants discovered with AHA Tuning.

• Part VIII: assistance that the AHA should offer respondents and their colleagues.

• “Conclusion”: allowed participants to write additional comments and queries.

II.2. Goals – and limits – of the survey

I came to this project as a novice in survey design and analysis – and as an interested participant. Serving as an adviser to the AHA Tuning project, my intention was to develop a survey that could help the AHA understand how its members apply Tuning in their daily work – and help those outside the U.S. to consider the contributions of professional disciplinary societies to Tuning projects. In other words, I developed the project as an advocate for AHA Tuning, not as a disinterested and detached observer. The utility of the survey has always been at the center of the research. I hope the survey serves as a starting point for the AHA to gather more detailed and systematic information about the directions in which Tuning has guided the discipline.

The survey results fell short of expectations in three important ways:

• Percentage of responses: Nearly 300 historians responded, but the response rate was 20%. The percentage was lower than anticipated.[24]

• “Non-tenure track” faculty: This group represents the majority of instructors on post-secondary campuses. But only 20% of responses came from this group.[25]

• Faculty at two-year institutions: Only 9% of respondents worked on these campuses.

Despite these limits, survey results did meet other expectations:

• Instructors who were not part of the AHA Tuning cohorts: Of those who replied, 22% were members of a cohort; but 78% had not participated in the Tuning workshops.

• Faculty with limited knowledge of Tuning: Over 46% of replies came from those who described themselves as “quite” or “fairly” knowledgeable about Tuning. Yet 53% of the responses came from those identified as Tuning beginners or unfamiliar with the work.

• Educators with teaching assignments: Nearly 80% of the responses were from those who identified as instructors. Fewer than 20% of replies came from colleagues who identified themselves in terms of an administrative office.

III. The Influence of AHA Tuning on activities in Teaching and Learning

What did the survey reveal about teaching within history departments? The survey asked participants about the types of activities they and their department colleagues have worked on over the past five years in relation to curricula, course design, pedagogy, assessment, and enrollments. In addition, the survey asked participants to consider the perceived level of influence that AHA Tuning had on reforms they had discussed or enacted.

Survey results: Initial questions asked if Tuning discussions took place in a history department, what issues came up for discussion, and who participated in the conversations.

Analysis: The responses point to three areas of success for the AHA Tuning project:

• Two-thirds of responses indicated that meetings on Tuning took place one or more times (although meetings occurred more commonly on 4-year rather than 2-year campuses).[26]

• The issues discussed in meetings matched the central points stressed by the AHA. Five major issues dominated most Tuning meetings, with fairly even participation on both 2- and 4-year campuses.[27]

• A range of faculty leaders and participants engaged in the conversations.

Three groups participated most often in discussions: tenure-track faculty; “cohort” members; and department heads. Other participants (in 15-39% of responses) included contract faculty and students. Two institutional contrasts stand out. No 2-year campus reported the presence of instructors with renewable contracts; and few 4-year campuses worked with adjunct faculty.

While recognizing some Tuning successes, three issues deserve comment:

First, two-thirds of respondents said that Tuning discussions had occurred; but a full third stated that “we have not had discussions on Tuning.” Recall that surveys went out only to institutions with cohort representatives, the colleagues asked by the AHA to carry the conversation back to their campuses. Yet, on 4-year institutions, 33% of replies reported that there were no meetings; on 2-year campuses, 41% of responses indicated an absence of meetings.

In subsequent parts of the survey, however, there were multiple questions, focused on a range of issues, that allowed respondents to address the presence or absence of Tuning discussions. While 33% of respondents in the broad, opening question stated that their department had no discussion of AHA Tuning, twelve additional questions with the same (or similar) option revealed a range of responses, with 22-36% indicating the absence of Tuning meetings. One might think respondents knew that Tuning discussions either had or had not occurred. Some of the difference may rest with slight shifts in wording that were, regrettably, part of the survey questions. In addition, it is possible that the precise language of “Tuning” did not dominate meetings on some campuses – even though colleagues may have still engaged in discussions and practices directly tied to the issues that Tuning addresses.[28] Perhaps some respondents, growing weary of stating that Tuning had not occurred, turned to other types of responses to make their point. Whatever the possible explanations, approximately one-third of replies indicated that “Tuning,” in some capacity, to some degree, at some level, had not come up for discussion.

A second concern involves the range of educators who took part in Tuning discussions. Project leaders hoped to engage a large and varied group of educators and stakeholders in conversations. However, fewer than 15% of responses indicated that department Tuning meetings included post-docs, deans, academic advisors, librarians, career counselors, K-12 teachers, employers, community leaders, or policy makers.

A third concern involves limits on the types of issues addressed in the majority of departments. Most reported that conversations focused on learning outcomes, assessment, the design of introductory and capstone courses, and graduates’ career prospects, all of which are significant and valuable discussions. But a number of additional Tuning activities received less attention. Fewer than 1 in 7 responses indicated that their departments met with “stakeholders” (such as employers and policy makers), arranged meetings of faculty from two-year and four-year institutions, improved the process of transferring student course credits, or focused greater attention on non-majors.[29]

These three concerns do not indicate “failures” of AHA Tuning. Project leaders within the AHA recognize that the activity, in the words of Dr. David Pace, “requires new thinking and a willingness to break with the past,” work that requires considerable time, effort, and patience.[30] History faculty are off to a strong start in Tuning work. These concerns simply point to work that history colleagues and the AHA still need to address on U.S. campuses.

III.1. Effects of the AHA Tuning Project on curricula

The survey asked if faculty engaged in any of eight Tuning efforts that addressed programs of study. Participants could add “other” projects that did not appear on the list. For each activity, respondents also reflected (from a large number of options) on the level to which AHA Tuning may have influenced curricular work: if Tuning had major, some, minimal, or no influence; if Tuning integrated well with other reform projects to prompt changes; or if the department had not instituted the particular change. Survey results led to interesting sets of reflections.

Survey results: The four most commonly noted curricular changes were also those for which participants believed Tuning had the greatest influence.

Analysis: A clear majority of history faculty have engaged in a wide range of projects reshaping their programs’ curricula. In the five most frequently-noted initiatives, 41%-62% of respondents felt that Tuning some or a major influence over the change.

The results in the far-right column are puzzling, however, and raise a complicated question concerning Tuning’s “influence,” “impact,” or “effect.” We will return to the topic in Section VI, “Conclusions.” At this point, it is important to acknowledge that the terms themselves are vague and elastic. The survey did not generate a precise, quantifiable measurement of “influence” but, instead, focused on colleagues’ general perceptions of the way Tuning affected departmental discussions. Was there a level of awareness that the AHA’s program shaped, at some level, the issues and proposals that came up for faculty discussion? Did faculty see an identifiable “Tuning approach” to questions of teaching and learning? Or was “Tuning” an imprecise and amorphous term that remained unclear – or seemingly unrelated to department discussions?

As survey results indicate, the perceived level of Tuning’s “influence” varied from one curricular change to another. In responses to the most commonly mentioned curricular change – “identifying learning outcomes / objectives / goals” – an issue that was central to the AHA Tuning initiative, a full 56% felt that Tuning had some or a major role in shaping the project. However, on the same issue, 36% believed Tuning had nothing to do with the change. Perhaps Tuning’s skills-centered approach to learning was not sufficiently clarified on different campuses. Perhaps colleagues discussed Tuning in association with a lengthy (and confusing) list of other reform activities and acronyms. Or perhaps the core themes of Tuning have become so well-incorporated into other skills-based projects that the AHA initiative is now less “visible” and distinct. As one respondent noted in comments, “we do all of these [teaching and learning activities] regularly but not in the context of AHA’s Tuning project.” Another added, “we do these as part of normal department work; did not know they were part of tuning.”

Was there a particular body of faculty who most often noted the absence of any Tuning influence on curricular changes? One group was more skeptical of Tuning’s impact on departmental initiatives: associate and full professors, with 20+ years of experience, working at 4-year institutions, who were not part of a Tuning cohort, and who describe themselves as “beginners” in the project. In opening questions – and subsequent queries – these colleagues expressed more doubt than others about the influence and contributions of AHA Tuning.

Still, while recognizing the reservations and suspicions held by some, one point is clear: historians have engaged in a broad range of curricular reforms and felt that Tuning played a considerable role in promoting many of these activities.

III.2. Effects of the AHA Tuning Project on courses

The next section asked about alterations that faculty and their department made in courses and course design since the start of AHA Tuning. Participants could mark any of six major types of course changes tied to Tuning. The survey also allowed respondents to add “other” changes and offer an indication of the level of influence Tuning had on any course innovations.

Survey results: Participants focused their replies on the six course design changes listed.

Analysis: As with work on curricula, the survey displayed a varied range of work by historians as they reexamined disciplinary courses. Results indicate that 62%-78% of respondents engaged in five of the six course reform projects listed in the survey. Although participants had the opportunity to write in “other” types of course reforms, few provided examples beyond the half-dozen listed in the question.

Comparing the responses of faculty from 2- and 4-year campuses on questions of course design, the survey revealed that a higher percentage of colleagues at community colleges engaged in course changes than instructors at 4-year institutions.

The only activity in which 2-year schools participated at a low percentage was, understandably, in the revision of capstone courses (the type of upper-division, research-intensive classes most commonly taught in the last year of a 4-year program).

A large majority of the history faculty who took part in the survey belong to departments that have taken part in a diverse set of initiatives focused on courses and course design. Although the perceived influence of AHA Tuning on these reforms was modest, the level of disciplinary engagement in reframing classes was quite high.

III.3. Effects of the AHA Tuning Project on teaching

The fourth part of the survey asked participants to consider how AHA Tuning may have helped revise teaching practices that they or their department made. The question listed nine commonly-referenced changes that cohort members and AHA leaders have discussed and implemented in the past five years. As in other parts of the questionnaire, respondents could also indicate the perceived “level” of Tuning’s influence on classroom reforms.

Survey results: Responses indicate that historians have reexamined many pedagogical techniques over the past five years, though (as with course redesign) Tuning’s perceived impact on these changes appears modest.

Analysis: The majority of respondents have taken part in six different pedagogical reforms, pointing to a considerable reexamination of classroom techniques among faculty and history departments. But among those noting the changes, a minority (20-40%) felt that Tuning exercised “major” or “some” influence over the new approaches, suggesting that the impact of Tuning itself on these initiatives remained at a fairly low level.

Three other observations about teaching practices deserve mention. First, survey responses from 2- and 4-year campuses broke down in very similar ways on this question. Second, as with courses and course design, participants noted few additional forms of innovations. Third, the phrasing of the survey may have contributed to an apparent disjunction in responses. While nearly 60%-80% of responses indicated that faculty had engaged in six different pedagogical experiments, 20%-40% of participants noted that those same changes had not occurred. One possible reason for this discrepancy? One response available to participants stated “I / my department has not instituted this change.” I deliberately included this slight rephrasing of questions in order to see if results displayed any variation. My hope was to gain some sense of the way individuals responded to new teaching approaches. Those most likely to argue that teaching changes had not taken place were, once again, senior, tenured faculty on 4-year campuses who did not take part in Tuning cohorts. It remains unclear if these colleagues referred to themselves or to other faculty in their departments in their replies.

One final – and broader – point concerning Parts 1-3 of Section III sums up findings about the “impact” of Tuning. Stated simply, participants’ responses suggest that the influence of AHA Tuning has been most pronounced in the design of history curricula. But faculty see the project’s impact as less evident in the work of course revisions and teaching.

III.4. Effects of the AHA Tuning Project on assessment

The fifth section of the survey focused on assessments of student learning. While not emphasized at the start of AHA Tuning, the question of how educators recognize, capture, and report student learning has taken on an increasingly important role in the society’s project, culminating in a collection of journal articles on the subject in early 2016.[31] The survey asked about faculty engagement with different types of assessment exercises promoted by the AHA and other U.S.-based educational organizations. Respondents could also reflect on the level of influence AHA Tuning held over the work.

Survey results: Of the fifteen assessment techniques listed, the majority of respondents noted work on half a dozen. While results indicted a significant level of engagement with several types of assessment, participants felt that AHA Tuning had – at best – only a moderate level of influence on the work of measuring student learning.

Historians have widely adopted rubrics in the evaluation of student exercises. Eighty percent of participants noted their use. Of those, well over a third believed that AHA Tuning had some or a major influence on the adoption of rubrics. The second most common assessment technique (in 71% of responses) involved evaluation of a “major research project within a capstone course.” While the practice is widespread in departments, only 23% of surveys suggested that Tuning had some – or a major – influence on this form of assessment. In fact, the majority of responses stated that Tuning had no influence. Understandably, colleagues at 2-year campuses, working with students just starting historical studies, rarely reported the practice.

Among other popular approaches, three-fifths of surveys indicated that historians used the assessment tools available in course management software programs, and a slim majority reported on revised course evaluations and surveys of alumni. But fewer than 20% of respondents perceived some or a major influence from the Tuning project.

Among the remaining assessment approaches offered in the survey, 23-49% of responses indicated work departments had done with various techniques: student self-assessments, pre- and post-tests, entry and exit interviews with students, electronic portfolios of student work, student journals, employer surveys, and the use of inter-rater reliability techniques. The least common assessment practice reported on the survey – at 21% – involved the use of standardized exams

Analysis: While survey respondents generally categorized Tuning’s “influence” on assessment practice as limited, three points are worth noting. First, leaders of AHA Tuning began to place more emphasis on measurements of student learning after the first years of the project. Second, respondents pointed to an impressive variety of tools and techniques used within the discipline. And third – perhaps most importantly –the meaningful engagement of history departments in assessment work has taken root after a long period of avoidance and aversion. A former Vice-President of the AHA’s Teaching Division, Dr. Elizabeth Lehfeldt, noted (only half-jokingly) her colleagues’ deep-seated reluctance to tackle assessment. In April 2013 she wrote, “Walk into a meeting of your average college or university history department and utter the simple word ‘assessment.’ The reactions will probably range from eye-rolling to resigned sighs to blank stares. Some might even run screaming from the room.”[32]

Survey results suggest that a significant – and rapid – turnaround in historians’ practices has occurred since the start of AHA Tuning. What accounts for the shift? The work may be a response to institutionally-based assessment projects; 3-11% of responses indicated that Tuning integrated with other campus initiatives. The work also seems tied to purposeful, “pro-active” measures (to ensure that disciplinary experts maintain control over disciplinary assessment). In addition, colleagues are genuinely concerned about student success. And, in part, the work also appears to have come from the encouragement of the AHA and its recommended “best practices” for the discipline – including a recognition that thoughtful assessment allows historians both to comply with institutional reporting and improve educational practice.[33]

III.5. Effects of the AHA Tuning Project on enrollments

While survey participants felt that AHA Tuning has had a modest-to-significant effect on teaching and learning within the discipline, the project has remained fairly ineffective in reversing or halting declining enrollments in history courses and numbers of history majors.

Survey results:

Analysis: Only one in five respondents reported that enrollments in history classes had increased in the past five years. Replies from both two- and four-year campuses were the same. Fewer than one in ten respondents believed that AHA Tuning had helped enrollments rise. None of those expressing that optimistic view, however, were faculty at two-year institutions. Reports of enrollment declines came from half of respondents; a perception that enrollments remained steady came from one-fifth. On this pair of observations, there was little appreciable difference between faculty at two- and four-year schools. Had Tuning “contributed to a decline in enrollments in history classes”? No participants believed that to be the case. The most common reply, across the board (at 82%) was that “Tuning had no appreciable effect on enrollments.”

Participants offered similar views on a second question tied to numbers of history majors. While one in five reported increases in numbers of history majors over the past five years, 17% believed that figures remained stable, while nearly three-fifths claimed that numbers of majors had declined. Considering the effect of Tuning on these figures, only 1 in 8 surveys stated that the project helped increase the number of majors; fewer than 1 in 10 felt that Tuning helped keep the numbers steady; and nearly four-fifths thought that Tuning had no appreciable effect on the number of majors. Responses from two- and four-year campuses were, again, quite similar on the issue of Tuning’s influence.

However, in replies about the numbers of history majors, 60% of reports about declines came from four-year campuses. Only 15% of respondents from two-year colleges made the same claim. It is important to remember that the number of survey results from 2-year faculty was quite small; and some of the colleges do not specifically offer a “history” major. Still, the figures show a significant difference between the two types of post-secondary institutions.

One other interesting part of the responses dealing with enrollments and majors came in the additional comments written by many participants. When explaining the reasons why Tuning helped numbers grow or remain steady, colleagues offered several suggestions. Respondents noted that Tuning: provided “clarity for the students in terms of the curriculum, expectations, and outcomes”; “created an esprit de corps among our majors”; and helped faculty “revise [their] approach to survey classes,” “promote the major,” “increase the diversity of offerings,” and “communicate the many benefits of studying history.” The comments offered a concise overview of key AHA objectives, providing a small indicator of the way Tuning has even affected casual “conversations” about the history discipline.

IV. Benefits Historians identified in the AHA Tuning Project

In addition to questions about AHA Tuning activities, the survey asked participants to reflect on “particular benefits you have observed in the AHA Tuning process.” The question listed 12 answers that colleagues have frequently expressed to the AHA since 2012 – and allowed participants to “measure” the level of benefit they perceived (“major,” “modest,” “small,” “fairly inconsequential,” or “does not seem to be a part of the Tuning project”). The survey also offered participants open-ended space to add “other” benefits – and the option of indicating there were “no significant benefits observed.”

Survey results: This section of the survey revealed noticeably skewed results. On one hand, participants indicated that several of the available choices represented important benefits of AHA Tuning. Yet a considerable percentage of other respondents came to a strikingly different conclusion, stating that the very same points “[do] not seem to be a part of the Tuning project.”

The top four choices identified most frequently as “major” or “modest” benefits of Tuning display a contrasting pattern of responses (with fairly high negative rankings). In all four cases, 55-58% of responses were quite favorable, while one-fifth to one-quarter of respondents believed the goals were not part of Tuning’s objectives. A cross-tabulation of the responses in all four choices showed that slightly more favorable responses came from a distinctive group: younger faculty, working on all types of campuses, who were not members of the AHA Tuning cohorts, and who described themselves as “quite” or “fairly” knowledgeable about Tuning. Slightly less favorable responses came from another distinctive group, noted earlier: senior faculty (20+ years of experience), working at public four-year institutions, not connected with the AHA Tuning cohorts, who had little to no familiarity with Tuning. (The remaining responses ranked the choice as a “small” or “fairly inconsequential” benefit.) Two other choices (dealing with student skills and accountability) also received a favorable response from the majority of respondents. There, too, more than a quarter of respondents categorized the issues as outside the realm of Tuning.

Participants also had the opportunity to declare that there were “no significant benefits observed” in Tuning. These replies captured some of the least enthusiastic responses to the project. Only 15% of participants replied in this way. While their responses were not “representative,” the presence indicates, again, that a small group of colleagues harbor considerable skepticism or suspicion about the Tuning project.

Finally, 5% of participants noted “other” benefits from Tuning (in addition to the available choices). Half suggested that the project had either stalled or had just begun in their departments.” Others felt that Tuning had not adequately addressed “under employment” among historians – or under enrollment among students.

Analysis: The question on Tuning benefits demonstrates that the AHA’s project has succeeded in clarifying the advantages of conversations, collaboration, curricular reform, critical self-reflection, and careful attention to student needs and opportunities. These are the very changes that the leadership of the American Historical Association has tried to encourage in historical study. In 2013, the vice president of the AHA’s Teaching Division, Dr. Elaine Carey, found that “Tuning has helped us coordinate our goals to build skills and knowledge important to history and transferable to other disciplines.” Dr. Kenneth Pomeranz, then president of the AHA, praised “Tuning’s intensive structured dialogue . . . from historian colleagues, from other disciplines, and from both friendly and hostile members of the general public.” Encouraging “close collaborations with our colleagues,” AHA President, Dr. Vicki L. Ruiz, applauded Tuning in 2015 as a project that “forges strategic partnerships within and across departments,” cultivating “philosophical conversations and teaching resources, as well as markers of student competencies.”[34] The key objectives laid out by the AHA for the Tuning project correspond to the “benefits” that history faculty have endorsed.

One other point becomes clear when examining both “benefits” and the earlier question on “activities.” The qualities of Tuning most highly admired by historians are also the aspects of the project most commonly implemented by faculty and departments. In other words, survey results point not only to Tuning’s aspirational goals, but also to real achievements on the ground

Yet there remains a striking division of opinion between those who perceived “benefits” with regard to the key aspects of Tuning and those who did not even consider the themes part of the initiative. Part of the explanation may involve gaps in the way colleagues have come to understand Tuning. Perhaps the distinguishing qualities of Tuning may remain unclear to some. Perhaps the question itself was open to different interpretations: some might have felt that faculty discussions, explicit outcomes, curricular redesign, and clear reference points “[do] not seem to be a part of the Tuning project” at their institution. Still, even if these possibilities are valid, the survey points to a layer of skepticism, suspicion, or misunderstanding about Tuning among some history faculty.

The result for one other type of “benefit” also deserves scrutiny from Tuning advocates. The AHA regards the work of two-year institutions as critical in any discussion of reform in U.S. higher education. The organization worked continuously to see that “community colleges” had a key role in the project, serving in the original “leadership core,” participating in the two “cohorts” of history Tuners, and guiding regional conferences sponsored by the disciplinary organization. The survey asked about work that “encourages meetings between 2- and 4-year faculty.” Over half of responses stated that 2-year/4-year collaboration “does not seem to be a part of the Tuning project.” If we add in those who believed that the collaboration was “fairly inconsequential,” and the percentage rises to nearly 70%. None of the other eleven possible responses to the “benefits” question received a similar type of skeptical response.

Which historians felt that collaboration efforts were slight? Recognizing that there were a low number of responses from community colleges, over a third of faculty at 2-year campuses believed cross-campus work was simply not part of the Tuning project. Perhaps more surprisingly, nearly 56% of faculty at 4-year institutions responded the same way. If we add in those identifying the work as “fairly inconsequential,” and the percentages rise to 45% and 72% respectively. Those who worked in the AHA’s two “cohorts” replied in a similar fashion: 63% of cohort members felt collaborative meetings were “inconsequential,” and 71% of those outside the cohorts agreed. Whatever the level of knowledge about Tuning that respondents claimed, the majority in all categories felt that 2-year / 4-year collaboration was inconsequential or not a part of Tuning. While the AHA did much nationally to bring different types of campuses together, little seems to have come of the effort locally. AHA Tuning leaders need to place greater emphasis on the work between institutions offering associate and bachelor degrees.

V. Problems Historians identified in the AHA Tuning project

One key purpose of the survey was to help the AHA identify problems that historians have encountered with Tuning. Four questions addressed this issue in different ways, asking participants about difficulties they experienced, listing a range of pre-defined criticisms they could check, and providing open-ended responses they could add.

Survey results: The main inquiry asked, “Are there problems you have observed in the AHA Tuning process?” The question offered participants nine commonly-expressed sources of difficulty, allowing respondents to check all that applied – at different levels of concern – while also offering the option of replying “no” or writing in “other” problems.

Analysis: Of the many choices offered, respondents identified only two issues as significant:

• 52% felt that Tuning’s “overemphasis on job and career prospects of students” was a major, modest, or minor problem;

• 52% saw Tuning’s neglect of “course content” as a major, modest, or minor problem.

The replies are worth noting because of the way AHA leaders have presented the subjects of employment and course material in documents and workshops.

Tuning discussions in the AHA have consistently examined the students’ “next steps,” particularly the professional and career opportunities that historical study might open for them. No Tuning materials make this subject pre-eminent. But it appears that a significant number of historians remain concerned that Tuning overemphasizes career questions. Similarly, Tuning’s advocates – along with historians connected with the “Scholarship of Teaching and Learning” (SOTL)[35] – have addressed the importance of developing key skill sets and modes of inquiry in history courses rather than giving exclusive or primary attention to names, dates, places, and events. The reform projects have emphasized analyzing – rather than merely covering – content.[36] Yet, to judge from the survey, a considerable number of historians remain uneasy about pedagogical approaches that steer attention away from traditional subject matter.

These two “problems” are interesting for another reason. The concerns raised about careers and content cut across different variables. Respondents who were part of the AHA’s two cohorts – and those who were not in the workshop groups – responded in similar ways to the pair of issues. The same holds true for respondents working at either two-year or four-year institutions. In other words, the concerns of historians about the handling of careers and content were similar regardless of their familiarity with Tuning – and regardless of their home institution.

On four other topics, participants expressed concern, although in none of the cases did a majority of respondents point to serious problems with the Tuning initiative.

One issue, tied to the way Tuning was originally presented, held particular interest. The first group of U.S. Tuners in 2009 experienced a measure of difficulty with the terminology of Tuning. The words that E.U. colleagues used to describe the components and operation of the Bologna Project and Tuning – including terms such as “learning outcomes,” “rubrics,” “competences,” “cycles,” and “qualification frameworks” – were new to some participants whose disciplines had long been removed from discussions of teaching, learning, and assessment.[37] I anticipated that colleagues eight years later would have the same level of difficulty “decoding” the language of Tuning. But the survey suggested something quite different. When asked if the “vocabulary used in the project” posed a problem, 44% agreed (with most classifying the problem as modest or minor). However, 56% did not view the language of Tuning as an obstacle or difficulty, suggesting yet another important shift in the vocabulary and expectations that guide historians.

Similarly, the majority of participants did not detect problems with “standardization,” excessive “corporate / foundation influence,” or “challenges to academic freedom.”

Three topics remained strikingly un-problematic for participants. On the “appropriateness” of Tuning, most believed that the initiative examines questions that are pressing and important to post-secondary institutions. On the “faculty-led” nature of Tuning, most see the project as one shaped and controlled by disciplinary experts. Regarding the time allotted for Tuning work, 70% believed this was not a problem, suggesting that the project did not seem rushed or constricted.

The question that addressed “problems” with Tuning also left room for open-ended responses. Fewer than 10 participants contributed suggestions, but the points they raised are worth noting. More than one colleague mentioned a lack of administrative “buy-in” and the absence of “incentives” to encourage faculty participation in the Tuning project. Others expressed concern that Tuning at their institution was too rigid (allowing too little flexibility in introductory courses) or too inattentive to assessment issues. Another respondent worried that the informing “culture” of the department was too settled and unchanging to make Tuning work. A particularly interesting observation came from a participant who wondered if Tuning had “done enough to figure out what students want/think about the goals of their degree,” suggesting that the “faculty-centered” nature of the project needed to become more “student-centered.”

Overall, the difficulties respondents noted with Tuning tended to fall into the “minor” range. There are still significant points of dissatisfaction with the project, particularly in areas of student opportunities and course content, that remain sources of concern for many in the survey. While the AHA has repeatedly addressed these two considerations, it appears that a fair number of colleagues have not yet been convinced by the disciplinary society’s arguments.

V.1. Respondents’ suggestions to the AHA

An additional survey question provided an indirect measurement of “problems” encountered by history faculty: “Do you have any suggestions or requests to make to the American Historical Association for further assistance that the organization can offer?” The question presented ten concerns that AHA Tuning leaders have frequently heard from project participants.

Survey results: Only half of survey participants responded to this question. Those who replied could make multiple selections. The results below break down responses by type of campus and by cohort experience with Tuning.

Analysis: Bear in mind that half of the survey participants did not respond to the question. With the limited number of responders, there was still only one request that came from a majority of participants. It appears that a sizable number of historians want to know more about the actual experiences of colleagues trying to integrate Tuning measures into courses, curricula, department life, and discussions with stakeholders.

The second most frequent request, involving Tuning and enrollments, addresses a national concern about declining numbers of students declaring a history major and enrolling in history courses. All hope that Tuning’s attention to student-centered needs and innovative pedagogical practices might help clarify strategies for dealing with this worrisome trend.[38]

The third most requested form of assistance from the AHA shifts attention to faculty engagement. Recognition of Tuning work in tenure and promotion procedures highlights the importance many historians place on reward systems that can encourage faculty participation.

A fourth request, on introductory Tuning workshops, revealed an interesting set of replies. The group most likely to call for the sessions were cohort members, those who have already participated in introductions to Tuning. Perhaps these responders see a need for periodic “refresher” courses in the work – or perhaps they know from experience that the workshops would be helpful for their colleagues.

Among less popular selections, it is curious to see that few participants from 2-year schools perceive a need to address “naysayers” or critics of Tuning. Perhaps their responses indicate the comparative success of Tuning in community colleges. Two-year faculty were particularly interested in the AHA providing forms of recognition for Tuning work – and for encouraging conversations between 2- and 4-year campuses. Four-year campus faculty were far less likely to call for guidance on the latter issue.

Historians displayed the least interest in “revising the AHA Disciplinary Core.” Only 3% of replies called for this change, suggesting that the organization has done a good job listening to and communicating with members about the informing principles and goals of historical study.

VI. Conclusions

The survey was designed to examine the “impact” AHA Tuning. The results call attention both to the work of historians and to the complex nature of “impact” itself.[39]

One formal way of determining AHA Tuning’s impact is to consider the “deliverables” outlined in the original agreement. The Lumina Foundation asked the AHA to work with a large national group of historians, define core competencies in the field, clarify the value of historical study, and build the foundation for meaningful assessment of student learning. In turn, the AHA asked members of its “cohorts” to attend Tuning workshops, work with their campus colleagues to clarify core learning goals, identify key curricular revisions, and explore the career prospects of students.[40] In both cases, the work has been achieved in a thoughtful and engaged fashion

A second approach to the question of impact is to observe the effects of Tuning on the American Historical Association itself. The organization’s annual meeting looks different from conferences a decade ago with a greater range of panels focused on teaching, learning, and career opportunities. The institutional affiliations of the AHA’s Council and committee members has become more diverse with greater representation from teaching-focused institutions. And the AHA offers more regional meetings focused on questions tied to course instruction.[41]

A third way to address “impact,” following in-formal conversations the author had with Lumina leaders, is to focus on changes in faculty practice. Tuning has not remained an abstract topic of discussion and debate. Participants have not simply focused on declarations of objectives and intentions. Rather Tuning has generated observable shifts in the conduct of coursework, curricular design, assessment, and mentoring, pointing to another achievement of project goals.

A fourth approach to “impact” addresses changes in the “culture” of the discipline.[42] As noted in Section III: 4, historians have long sidestepped calls to examine the actual learning that students have achieved. Colleagues who experienced frustration with older practices of assessment and accreditation have often pulled back from projects that called for critical self-evaluation of disciplinary work and course structure.[43] As Anne Hyde, faculty chair of AHA Tuning, commented, “historians are used to thinking very individualistically and proprietarily about their classrooms, content, and students. . . . What we do has become obvious and natural,” seemingly requiring no explanation.[44] Survey results indicate a considerable turnaround in the habits, outlook, and practices of historians. Colleagues have engaged in much more collective, intentional, and transparent work designed to demystify historical study and make implicit assumptions about the discipline explicit for a wider audience.

A fifth consideration of impact addresses how well Tuning has integrated with additional post-secondary reforms in teaching and learning that also focus on outcomes-based, skills-focused, student-centered approaches. “Many colleagues have worked on AHA Tuning in conjunction with other academic projects on teaching and learning,” one question stated. “Have any of the following projects formed a part of the discussions and planning in your department?” The survey listed initiatives with fairly wide adoption on U.S. campuses. A follow-up question asked if participants sensed a close connection between Tuning and the other projects.

The survey shows that only a minority of respondents indicated the participation of their department in additional teaching and learning projects. But of those who did engage in other reforms, nearly a third perceived a clear sense of connection with their Tuning work. No respondents believed Tuning was “in conflict” with any of the projects cited. The outcomes-, skills-, and student-centered nature of these additional initiatives likely reinforces a sense that the projects “converge” rather than conflict with one another. At the same time, when asking faculty to determine the “influence” of AHA Tuning, we should bear in mind that it may be difficult for to distinguish Tuning-related projects from Tuning-caused activities – and come up with a clear notion of Tuning’s specific “impact.”

Although survey responses provided only a general sense of colleagues’ perceptions, many history departments have extended their engagement with teaching, learning, and assessment through a diverse range of projects. And the historians working on these initiatives tend to see conceptual and procedural links among the reform efforts. Tuning’s exact “impact” on the field of history may be difficult to measure precisely, but a good deal of evidence points to the formal, organizational, structural, and cultural changes that the project has promoted in the discipline.

VI.1. Questions to consider

Survey results that demonstrate Tuning’s effectiveness also shed light on some concerns and problems in the project. Five points hold particular importance as the AHA continues its work:

• Engagement with students: A core concern of all Tuning work revolves around building a “student-centered” project. One way to address the needs and interests of students precedes the work of altering courses, assignments, and curricula, and involves bringing students into the broader discussion of Tuning itself. Survey results raise questions about the extent to which historians have drawn students into the conversation.

Question 22 asked, “Did your department or academic unit consult with student representatives in any of your discussions on Tuning or related initiatives tied to teaching and learning?” Surveys showed limited work in this area. Nearly half the respondents (from both two- and four-year campuses) stated that “discussions only involved faculty.” One in five noted work with students in surveys or focus groups – though these activities were only reported at four-year institutions. And only 5% said that students “attended some of our meetings.”

A half dozen respondents reported work with graduate students, suggesting an effort to bring pedagogical discussions into master’s or doctoral level work. Others commented on focused discussions of Tuning with undergraduates in methodological and capstone research classes. But the survey revealed few accomplishments in continuous, systematic integration of faculty and student work on Tuning. This is one issue for the AHA to consider in the future.

• Outreach: Another major theme in Tuning emphasizes the importance of broadening discussions of academic learning with diverse groups, bringing in a range of educators, transcending departmental and institutional boundaries, and reaching out to employers, policymakers, and the general public. Survey results indicate, however, that this type of work has been limited and inward-looking. Non-tenure track faculty are left out of most conversations. Cooperative work with other campuses remains limited. And community “stakeholders” in higher education have not commonly take part in the discussions. It may prove useful in future AHA Tuning work to stress the value of hearing from a wide range of voices and interests.[46]

• Incentives: As the AHA tries to engage faculty in Tuning, it is reasonable for educators to ask what they might gain from the work in terms of tangible benefits, advantages, or recognitions: “What’s in it for me?” The data yielded few clear answers. Responses revealed the decidedly limited state of incentives. It seems appropriate for the AHA to review the types of rewards and encouragements that can best inspire colleagues to continue their valuable work.

• Enrollment: Material presented in Section III, 5 makes another point quite clear: despite initial hopes, Tuning has not helped turn around falling course enrollments and declining numbers of declared majors. It may be worthwhile to revisit this issue with fresh eyes to see if department heads and faculty members can propose new ways to use the tools and techniques of Tuning to reverse this unsettling trend.

• The missing third: Finally, it is worth recalling two puzzling points raised in Section III: that roughly one-third of respondents believed Tuning discussions had not occurred in their departments; and that roughly one-third also felt Tuning had no influence over some key curricular reforms. I have struggled to understand how these colleagues conceive of Tuning – and how others may have presented the project to them. Perhaps one way to respond to these curious results is for the AHA to consider a request from respondents: to “provide ‘introductory’ [or refresher] workshops on AHA Tuning at annual conference and regional meetings.”

VI.2. Successes in AHA Tuning

Survey results point to four key areas of success in the AHA Tuning project:

• The qualities of Tuning most highly emphasized by the AHA – continuous, collective discussion, thoughtful self-reflection, clarification of disciplinary goals and value, intentional curricular and course revision, invigorating pedagogical experimentation, attention to student learning – are also the aspects of the project most commonly implemented by faculty. Historians have clearly engaged with outcomes-based, student-centered, discipline-focused, faculty-led initiatives in teaching and learning, not simply by outlining aspirational goals but in actually reframing the day-to-day work they perform with students.

• Work in Tuning has expanded into a wide range of projects in teaching and learning, building not simply a “collection” of initiatives but a group of thoughtfully-integrated approaches to the intellectual, career, and civic interests of students.

• Lingering concerns with the Tuning project, once fairly common, remain quite muted. Colleagues voice few strenuous objections – and detect few major problems – with the initiative.

• Tuning has helped illuminate significant changes within the disciplinary “culture” of history. The focus and expectations of faculty continue to shift from an emphasis on “my course” to the contributions of “our curriculum,” from the “private” nature of teaching to the collaborative efforts in education, and from the “burdens” of accountability and assessment to the knowledge and guidance that the work generates. James Grossman and Emily Swafford, the AHA’s executive director and manager of academic affairs, frame all of this as an evolving process in matters of outlook, habit, and values, posing a “radical notion”: that “we begin not with what we want to teach but rather with what we want our students to learn.”[47]

Tuning through a disciplinary society has sparked tangible, transformative changes among historians – and has helped “scale” reforms in teaching and learning across the U.S.

Bibliography

Adelman, Cliff, Peter T. Ewell, Paul Gaston, and Carol Geary Schneider. The Degree Qualifications Profile. Indianapolis: Lumina Foundation, 2011. http://degreeprofile.org/wp-content/uploads/2017/03/DQP-web-download-reference-points-FINAL.pdf.

Adelman, Clifford. Learning Accountability from Bologna: A Higher Education Policy Primer. Washington, D.C: Institute for Higher Education Policy, 2008. http://www.ihep.org/sites/default/files/uploads/docs/pubs/learning_accountability_from_bologna.pdf.

   . The Bologna Club: What U.S. Higher Education Can Learn from a Decade of European Reconstruction. Washington D.C.: Institute for Higher Education Policy, 2008. https://files.eric.ed.gov/fulltext/ED501332.pdf.

   . The Bologna Process for U.S. Eyes: Re-Learning Higher Education in the Age of Convergence. Washington, DC: Institute for Higher Education Policy, 2009. https://files.eric.ed.gov/fulltext/ED504904.pdf.

“AHA History Tuning Project: 2016 History Discipline.” American Historical Association, n.d. https://www.historians.org/teaching-and-learning/tuning-the-history-discipline/2016-history-discipline-core.

AHA Tuning Project. “AHA History Tuning Project: 2013 History Discipline Core.” American Historical Association, n.d. https://www.historians.org/teaching-and-learning/tuning-the-history-discipline/2013-history-discipline-core.

   . “AHA History Tuning Project: 2016 History Discipline Core.” American Historical Association, 2016. https://www.historians.org/teaching-and-learning/tuning-the-history-discipline/2016-history-discipline-core.

   . “American Historical Association History Tuning Project: Call for Applications.” American Historical Association, 2014. historians.org/Documents/Tuning/AHA%20Tuning%20application.pdf.

   . “Tuning Participants.” American Historical Association,https://www.historians.org/teaching-and-learning/tuning-the-history-discipline/tuning-participants.

American Historical Association, “About AHA & Membership,” n.d., https://www.historians.org/about-aha-and-membership.

   . “Call for Applications,” 2014. http://www.historians.org/projects/tuning/.

   . “Career Diversity for Historians,” n.d. https://www.historians.org/jobs-and-professional-development/career-diversity-for-historians.

   . “Resources for Tuning the History Discipline,” n.d. https://www.historians.org/teaching-and-learning/teaching-resources-for-historians/resources-for-tuning-the-history-discipline.

Association of American Colleges & Universities. “AAC&U Programs,” n.d. https://www.aacu.org/aacu-programs.

   . An Introduction to LEAP, Liberal Education America’s Promise: Excellence for Everyone as a Nation Goes to College. Washington, DC: Association of American Colleges and Universities, 2015. https://www.aacu.org/sites/default/files/files/LEAP/IntroToLEAP2015.pdf.

Barnard College. “Reacting to the Past,” n.d. https://www.historians.org/teaching-and-learning/teaching-resources-for-historians/reacting-to-the-past.

Becher, Tony, and Paul Trowler. Academic Tribes and Territories: Intellectual Enquiry and the Culture of Disciplines, 2nd ed. Philadelphia: Open University Press, 2001.

Beneitone, Pablo, and Maria Yarosh. “Tuning Impact in Latin America: Is There Implementation Beyond Design?” Tuning Journal for Higher Education 3, no. 1 (December 23, 2015): 187–216. https://doi.org/10.18543/tjhe-3(1)-2015pp187-216.

Birtwistle, Tim, Courtney Brown, and Robert Wagenaar. “A Long Way to Go . . .: A Study on the Implementation of the Learning-Outcomes Based Approach in the EU.” Tuning Journal for Higher Education 3, no. 2 (May 2016): 429–63. https://doi.org/doi: 10.18543/tjhe-3(2)-2016pp429-463.

Brookins, Julia. “AHA Tuning Project Holds Events around the Country.” Perspectives on History 53, no. 9 (December 2015). https://www.historians.org/publications-and-directories/perspectives-on-history/december-2015/aha-tuning-project-holds-events-around-the-country.

   . “Enrollment Declines Continue: AHA Survey Again Shows Fewer Undergraduates in History Courses.” Perspectives on History 56, no. 2 (February 2018).

   . “History Faculty Begin Nationwide Tuning Project.” AHA Today (blog), February 13, 2012. http://blog.historians.org/2012/02/history-faculty-begin-nationwide-tuning-project/.

Calder, Lendol. “Uncoverage: Toward a Signature Pedagogy for the History Survey.” The Journal of American History 92, no. 4 (2006): 1358–70. https://doi.org/10.2307/4485896.

Carey, Elaine, Tracey Anne Cooper, Elizabeth Herbin-Triant, Philip Misevich, and Alejandro Quintana. “Tuning the Core: History, Assessment, and the St. John’s University Core Curriculum | AHA.” Perspectives on History 51, no. 4 (April 2013). https://www.historians.org/publications-and-directories/perspectives-on-history/april-2013/tuning-and-recharging/tuning-the-core.

Dillman, Don A., Jolene D. Smyth, and Leah Melani Christian. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th ed. Hoboken, NJ: Wiley, 2014.

Donà dalle Rose, Luigi F. “Impact of Erasmus Mobility for Study on the Development of Tuning Europe Generic Competences, As Assessed by Outgoing and Incoming Students in a Large University: A Revealing Analysis.” Tuning Journal for Higher Education 3, no. 1 (2015): 57–98. https://doi.org/10.18543/tjhe-3(1)-2015pp57-98.

Doyle, Christopher. “On Historians, Tuning, and Markets.” Perspectives on History 50, no. 6 (September 2012). http://www.historians.org/publications-and-directories/perspectives-on-history/september-2012/letter-to-the-editor-on-historians-tuning-and-markets.

Galgano, Michael J., and Patrick Manning. “Liberal Learning and the History Major (2007).” American Historical Association, 2007. https://www.historians.org/about-aha-and-membership/governance/reports-of-committees-and-divisions/liberal-learning-and-the-history-major-(2007).

“Greater Expectations: A New Vision for Learning as a Nation Goes to College.” https://www.aacu.org/sites/default/files/files/publications/GreaterExpectations.pdf.

Grossman, James. “A Time for Renewal: The AHA and the Path Forward.” Perspectives on History 55, no. 6 (September 2017). https://www.historians.org/publications-and-directories/perspectives-on-history/september-2017/a-time-
for-renewal-the-aha-and-the-path-forward.

   . “History, Historians, and ‘the Current Moment.’” Perspectives on History 55, no. 8 (November 2017). https://www.historians.org/publications-and-directories/perspectives-on-history/november-2017/history-historians-and-the-current-moment.

   . “Response to On Historians, Tuning, and Markets.” Perspectives on History 50, no. 6 (September 2012). http://www.historians.org/publications-and-directories/perspectives-on-history/september-2012/letter-to-the-editor-response-to-on-historians-tuning-and-markets.

   . “To Be a Historian Is to Be a Teacher.” Perspectives on History 53, no. 8 (n.d.). https://www.historians.org/publications-and-directories/perspectives-on-history/november-2015/to-be-a-historian-is-to-be-a-teacher.

   . “Tuning in to the History Major.” Perspectives on History 50, no. 6 (April 2012). https://www.historians.org/publications-and-directories/perspectives-on-history/april-2012/tuning-in-to-the-history-major.

Grossman, James, and Julia Brookins. “Assessment Is What We Make of It.” Journal of American History 102, no. 4 (March 2016): 1132–37. https://doi.org/10.1093/jahist/jav824.

Grossman, James, and Emily Swafford. “Graduate Education Reconsidered.” Perspectives on History 54, no. 4 (April 2016). https://www.historians.org/publications-and-directories/perspectives-on-history/april-2016/graduate-education-reconsidered.

Hauss, Miriam E. “White Paper Presented on the Role of the History Major in Liberal Education.” Perspectives on History 46, no. 9 (December 2008). https://www.historians.org/publications-and-directories/perspectives-on-history/december-2008/white-paper-presented-on-the-role-of-the-history-major-in-liberal-education.

“Hewlett Foundation.” Hewlett Foundation. https://www.hewlett.org/.

Hewson, Claire, Carl M. Vogel, and Dianna Laurent. Internet Research Methods. Second edition. Los Angeles: SAGE, 2016.

Hyde, Anne. “Five Reasons History Professors Suck at Assessment.” Journal of American History 102, no. 4 (March 2016): 1104–7. https://doi.org/10.1093/jahist/jav791.

Ilieva, Janet, Steve Baron, and Nigel M. Healey. “Online Surveys in Marketing Research: Pros and Cons.” International Journal of Market Research 44, no. 3 (Q 2002): 361–76.

Indiana University. “History Learning Project,” n.d. http://www.iub.edu/~hlp/.

Isaacs, Ann Katherine Isaacs. “Tuning and History: A Personal Overview.” Arts and Humanities in Higher Education 16, no. 4 (October 2017): 403-409, https://doi.org/10.1177/1474022216686506.

Isaacs, Ann Katherine, Guðmundur Hálfdánarson, and Carla Salvaterra, eds. TUNING Guidelines and Reference Points for the Design and Delivery of Degree Programmes in History, Edition 2018. Groningen, Netherlands: University of Groningen, 2018. https://www.calohee.eu/wp-content/uploads/2018/11/1.3-Guidelines-and-Reference-Points-for-the-Design-and-Delivery-of-Degree-Programmes-in-History-FINAL-v2.pdf.

Jankowski, Natasha, and David W. Marshall. Degrees That Matter: Moving Higher Education to a Learning Systems Paradigm. Sterling, VA: Stylus Publishing, 2017.

Jones, Jacqueline. “The Malleable PhD Mini-Conference.” Perspectives on History 51, no. 2 (February 2013).

Jones, Norm. “LEAPing in Utah: Lessons Learned along the Way.” Peer Review 13, no. 2 (Spring 2011): 16–18.

Kuh, George D., and Pat Hutchings. “Assessment and Initiative Fatigue: Keeping the Focus on Learning.” In Using Evidence of Student Learning to Improve Higher Education, n.d.

Lefever, Samúel, Michael Dal, and Ásrún Matthíasdóttir. “Online Data Collection in Academic Research: Advantages and Limitations.” British Journal of Educational Technology 38, no. 4 (July 2007): 574–82. https://doi.org/10.1111/j.1467-8535.2006.00638.x.

Lehfeldt, Elizabeth. “Tuning and Recharging: Tales from Twelve Months In.” Perspectives on History 51, no. 4 (April 2013).

Lehfeldt, Elizabeth A. “Addressing the Issue of Declining Enrollments.” Perspectives on History 54, no. 7 (October 2016). https://www.historians.org/publications-and-directories/perspectives-on-history/october-2016/addressing-the-issue-of-declining-enrollments.

“Lumina Foundation.” https://www.luminafoundation.org/.

Lumina Foundation. “Tuning USA: Lumina Foundation Launches Faculty-Led Process That Will Involve Students and Employers in Linking College Degrees to Workplace Relevance and Students’ Mastery of Agreed-Upon Learning Objectives,” April 8, 2009. https://www.luminafoundation.org/news-and-views/tuning-usa-lumina-foundation-launches-faculty-led-process-that-will-involve-students-and-employers-in-linking-college-degrees-to-workplace-relevance-and-students-mastery-of-agreed-upon-learning-objectives.

Marshall, David W., Natasha A. Jankowski, and Terry Vaughan. Tuning Impact Study: Developing Faculty Consensus to Strengthen Student Learning. Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment, 2017.

McInerney, Daniel J. “A Network of Curricular Connections: Lessons from Cultivating History in the State of Utah.” Journal of American History 104, no. 4 (March 1, 2018): 956–71. https://doi.org/10.1093/jahist/jax432.

   . “Eight Years On: Early – and Continuing – Lessons from the Tuning Project.” NILOA Guest Viewpoints, March 18, 2017. https://blogs.illinois.edu/view/915/489501.

   . “Tuning and Degree Profiles: U.S. Projects in Departments, Universities, States, Regions, and Disciplinary Societies, 2009-Present.” NIER (National Institute for Educational Policy Research [Japan]) Research Bulletin, no. No. 144 (March 31, 2015): 27–50. https://www.nier.go.jp/kankou_kiyou/144-103.pdf

   . “Tuning History in Utah: Winning Friends and Influencing Policy Makers.” Perspectives on History 52, no. 4 (April 2014): 14–15. https://www.historians.org/publications-and-directories/perspectives-on-history/april-2014/tuning-history-in-utah.

   . “Tuning the Discipline of History in the United States: Harmony (and Dissonance) in Teaching and Learning.” Arts and Humanities in Higher Education 16, no. 4 (October 2017). https://doi.org/10.1177/1474022216686523.

Nash, Gary B., Charlotte A. Crabtree, and Ross E. Dunn. History on Trial: Culture Wars and the Teaching of the Past. New York: Vintage Books, 2000.

National Center for Education Statistics. “Educational Institutions.” https://nces.ed.gov/fastfacts/display.asp?id=84.

National Council for the Social Studies. College, Career, and Civic Life (C3) Framework for Social Studies State Standards. Silver Spring, MD: National Council for the Social Studies, n.d. https://www.socialstudies.org/sites/default/files/2017/Jun/c3-framework-for-social-studies-rev0617.pdf.

Pace, David. “The History Classroom in an Era of Crisis: A Change of Course Is Needed.” Perspectives on History 55, no. 5 (May 2017). https://www.historians.org/publications-and-directories/perspectives-on-history/may-2017/the-history-classroom-in-an-era-of-crisis-a-change-of-course-is-needed.

Pomeranz, Kenneth. “Why History? A Note on the AHA Tuning Project.” AHA Today (blog), April 30, 2013. http://blog.historians.org/2013/04/why-history-a-note-on-the-aha-tuning-project/.

Quam-Wickham, Nancy. “Reimagining the Introductory U.S. History Course.” The History Teacher 49, no. 4 (August 2016): 519–47.

Ruiz, Vicki L. “A Quest for Balance.” Perspectives on History 53, no. 1 (January 2015). https://www.historians.org/publications-and-directories/perspectives-on-history/january-2015/a-quest-for-balance.

Safman, Phyllis, William Evenson, and Norman Jones. “Lumina Foundation Grantee Final Narrative Report.” Utah Tuning: Defining Learning Outcomes for Post-Secondary Degrees, 2014. http://utahtuning.weebly.com/uploads/1/4/6/9/14699846/report.7.15.14.pdf.

Serbati, Anna. “Implementation of Competence-Based Learning Approach: Stories of Practices and the Tuning Contribution to Academic Innovation.” Tuning Journal for Higher Education 3, no. 1 (December 23, 2015): 19–56.

Shurts, Sarah. “History in Harmony: The AHA ‘Tuning’ Project in the Community College and the Closing of the Transfer Gap.” The History Teacher 49, no. 4 (August 2016): 503–17.

Southern Association of Colleges and Schools. “Quality Enhancement Plans,” n.d. http://www.sacscoc.org/QEPSummaries.asp.

Spiegel, Gabriel M. “A Triple ‘A’ Threat: Accountability, Assessment, Accreditation.” Perspectives on History 46, no. 3 (March 2008). https://www.historians.org/publications-and-directories/perspectives-on-history/march-2008/a-triple-a-
threat-accountability-assessment-accreditation.

Stein, Robert, and Leah Reinert. “Transparent Pathways, Clear Outcomes: Using Disciplinary Tuning to Improve Teaching, Learning, and Student Success.” Minneapolis, MN: Midwestern Higher Education Compact, 2014. https://pdfs.semanticscholar.org/55c7/912b88114a179ac80a422df0dbbe0f5d4ebc.pdf.

The Higher Learning Commission. “AQIP Pathway Overview,” n.d. https://www.hlcommission.org/Accreditation/aqip-overview.html.

Townsend, Robert. “Data Show a Decline in History Majors.” Perspectives on History 51, no. 4 (n.d.). https://www.historians.org/publications-and-directories/perspectives-on-history/april-2013/data-show-a-decline-in-history-majors.

Weldmeskel, Fisseha Mikre, and Dreyer Johann Michael. “The Impact of Formative Assessment on Self-Regulating Learning in University Classrooms.” Tuning Journal for Higher Education 4, no. 1 (2016): 99–118.

---