Testing Framework: Scientific Evaluation of Moral and Legal Core Competencies

The development of academic quality standards and the shift towards core competencies in the academic level examination is one of the two highlights of the “Compulsory Education Moral and Legal Curriculum Standards (2022 Edition)” (hereinafter referred to as the “New Curriculum Standards”). Among them, the academic quality standards are the basic basis for the examination questions guided by core competencies, with the former answering the question of “what to test” under the guidance of core competencies, and the latter answering the question of “how to test core competencies”. However, both the “academic quality standards” and the “requirements for academic level examination questions” rely on the core competency testing framework (hereinafter referred to as the “core competency testing framework”). The core competency testing framework integrates “what to test” and “how to test” to achieve a scientific and objective assessment of moral and legal core competencies. To deeply understand the academic quality standards and the requirements for academic level examination questions, one must fully understand this framework.

From the implementation of the new curriculum standards for high school ideological and political education, existing interpretations and studies at the frontline have not addressed the core competency testing framework. Without understanding this foundational testing framework, it is difficult to accurately grasp and implement the relevant content of the new curriculum standards. The two authors, while participating in the work of the core competency testing group for compulsory education in moral and legal education, were specifically responsible for the development of the testing framework and have a relatively in-depth and systematic consideration of how to scientifically evaluate moral and legal core competencies. In this article, based on the experience from this testing work, we specifically introduce the theoretical foundation and constituent elements of the core competency testing framework, hoping this work will assist frontline ideological and political teachers in better implementing the new curriculum standards, enhancing their question-setting abilities, and thereby promoting the shift towards core competencies in teaching.

Constructing the core competency testing framework aims to explore how to assess moral and legal core competencies in paper-and-pencil tests. To construct the testing framework, two theoretical questions must first be answered: how to operationally define core competencies? How to understand the relationship between core competencies and various assessment elements?

(1) Operational Definition of Moral and Legal Core Competencies

The new curriculum standards state, “Core competencies are a concentrated embodiment of the educational value of the curriculum, formed gradually through students’ learning in the curriculum, encompassing correct values, essential qualities, and key abilities.” This definition, while revealing the disciplinary essence and basic components of core competencies, is not an operational definition and is difficult to apply in assessments. Therefore, we must operationally transform the essential definition of core competencies in examination settings. After repeated research and extensive literature review, the testing group provided the following operational definition for moral and legal core competencies:

Moral and legal core competencies refer to the comprehensive qualities exhibited by individuals when facing simple or even novel real-life situations, utilizing the knowledge and skills, thoughts and methods cultivated through this course to discover, analyze, and solve problems. This encompasses a comprehensive quality formed by correct values, key abilities, and essential qualities. It integrates interdisciplinary aspects from ethics, psychology, and law while also embodying the comprehensive nature of knowledge and skills, thinking and methodology, and emotional attitudes and values.

This operational definition transforms moral and legal core competencies from an unmeasurable internal essence into observable behaviors in real task situations, highlighting three key elements of core competency assessment: curriculum content, real situations, and tasks. Moral and legal core competencies and their levels can only be inferred “based on specific disciplinary contexts and observable performance characteristics in completing specific tasks.” Based on this operational definition, the construction of the testing framework must further clarify the relationship between core competencies and these three testing elements, thereby integrating them into an inseparable whole directed towards core competencies.

(2) Relationship Between Moral and Legal Core Competencies and Curriculum Content, Situations, Tasks

Based on the above analysis, we can clarify the relationship between moral and legal core competencies and curriculum content, situations, and tasks from the following three aspects.

First, focusing on the comprehensiveness of curriculum content, grasping the elements of competencies as a whole. The content of the moral and legal curriculum includes disciplinary knowledge and skills, thoughts and methods, which are not the objects of assessment but rather the basis for judging the level of students’ core competency development. To effectively utilize the tools of curriculum knowledge, one must grasp its comprehensive characteristics. As mentioned earlier, moral and legal core competencies possess interdisciplinary characteristics from ethics, psychology, and law. Therefore, students undergo interdisciplinary learning, where comprehensive curriculum content is not only the foundation for forming and cultivating core competencies but also an important dimension for evaluating the development level of students’ competencies. As question setters, it should be recognized that the core competencies of the moral and legal curriculum result from the learning of interdisciplinary, comprehensive content included in this course, and there is no one-to-one correspondence between a specific competency element and a specific content theme; each content theme is indispensable for cultivating each competency element.

Second, focusing on the value of competencies in real situations, organically integrating curriculum content. Real situations are the carriers for cultivating moral and legal core competencies. Curriculum content can only truly reflect its competency value when integrated with specific situations. Only by applying the learned curriculum content to address the problems and challenges in real situations can test-takers demonstrate their values, abilities, and qualities. Therefore, the setting of specific situations is indispensable in constructing the core competency testing framework. As question setters, it should be recognized that the relationship between the manifestation of moral and legal core competencies and real situations is complex; the same situation can exhibit different competencies, and the same competency can be expressed through different situations.

Finally, focusing on the mediating value of task activities, obtaining evidence of competency development. As previously mentioned, core competencies should be inferred through observable, explicit behaviors and their result characteristics, which we refer to as “behavioral performance characteristics.” The test should utilize specific tasks (both cognitive and non-cognitive) to observe the behavioral performance characteristics of the test-taker, thereby confirming whether they possess the competency to transfer curriculum content to real problem situations. Thus, task activities play a mediating role in evaluating the development level of core competencies and are another important dimension for testing moral and legal core competencies. As question setters, it should be recognized that the manifestation of moral and legal core competencies and task activities do not have a one-to-one correspondence; the same task activity can reflect multiple competency elements, and the same competency element can also be reflected through different task activities.

In summary, moral and legal core competencies are the starting point and goal of examination question setting and evaluation; content is the basis for verifying the level of competency development; situations are the carriers of competency development; and tasks integrate content and situations, serving as the stage for demonstrating core competencies. In short, the examination question setting for moral and legal core competencies should be “task-centric, integrating curriculum content and situations to achieve assessments directed towards core competencies.” As question setters, it should be recognized that different competency elements impose certain constraints on the selection of content, situations, and tasks, and that content, situations, and tasks also mutually constrain each other.

The testing framework is constructed based on clarifying the theoretical issues mentioned above and serves as the practical work framework for setting examination questions on core competencies. It represents a fundamental shift from traditional question setting based on the personal experience of the question setter to one based on a rigorous framework and scientific process.

A complete core competency testing framework should include elements such as testing objectives, testing structure, specific requirements for question setting, specific processes for question setting, and a test item information submission form. These elements collectively constitute the necessary conditions to ensure and improve the quality of question setting, enabling scientific assessment of core competencies. Large-scale question setting should prioritize the framework, with strict adherence to the framework as a basic requirement for question setting work. The enhancement of teachers’ question-setting abilities, both pre-service and in-service, should include learning to develop the core competency testing framework as a necessary component.

In the core competency testing framework, the testing objectives and specific processes for question setting are entirely determined by the specific work. For example, the specific testing objectives of the moral and legal core competency testing group include: exploring testing technologies and methods guided by moral and legal core competencies, ensuring the scientificity, precision, and realism of academic quality descriptions, and cultivating a group of core competency evaluation backbone personnel in moral and legal education. Clearly, different question-setting tasks have different objectives. The specific processes for question setting should clearly describe when to submit test items, how and when to screen and refine test items, and how and when to finalize and compile the exam, among other specific steps. Obviously, different question-setting tasks have different specific processes.

Relatively speaking, the testing structure, specific requirements for question setting, and test item information submission form possess a higher degree of content universality and generality, constituting the core part of the core competency testing framework.

(1) Testing Structure and Its Dimensions

The testing structure refers to a multi-dimensional two-way detailed table. Based on the aforementioned theoretical analysis, the moral and legal core competency testing group constructs a “four-dimensional two-way detailed table,” which specifically includes: characteristics and structure of moral and legal core competencies, structure of curriculum content, structure of situations, and structure of tasks. The so-called structure refers to the scoring proportions of each element in this test, which can be distributed according to actual circumstances. The testing structure is the most core and important content in the testing framework, specifically stipulating “what to test and how to test” in this examination.

The characteristics of core competencies describe the specific manifestation characteristics of each moral and legal core competency in this examination question setting. Core competencies are the internal comprehensive qualities of individuals, which are not directly observable. To ensure the reliability, objectivity, and scientificity of the test results, question setters must transform core competencies into observable behavioral performances. In the work of the moral and legal core competency testing group, this task is nearly equivalent to the actual workload of question setting.

The curriculum content should be guided by the big ideas of the discipline. To ensure that question setting is based on the comprehensiveness of the curriculum content, question setters should extract two categories of big ideas based on actual testing requirements: interdisciplinary big ideas and discipline-specific big ideas. From the development trends of academic level examination questions, the use of interdisciplinary big ideas is becoming increasingly important, such as applying knowledge from law and psychology to solve practical problems.

Situations are another very important aspect in constructing the testing structure. According to the design ideas of the new curriculum standards, the moral and legal core competency testing group categorizes situations into four basic types: personal life, human-nature relationships, human-society relationships, and human-state relationships. Question setters must consider not only the distribution of situation types but also list several typical situations within each type as references and inspirations for question setting.

The new curriculum standards state, “Tasks should exhibit diversity, such as describing characteristics, arguing, analyzing, evaluating, and formulating plans.” The moral and legal curriculum has the characteristic of unifying knowledge and action, but academic level examination questions are only suitable for testing cognitive operational behaviors. According to the new curriculum standards, the core competency testing group categorizes cognitive operations into three major types: identification and judgment, explanation and argumentation, and exploration and construction. Identification and judgment refer to recognizing and judging the nature of specific issues and objects in the context of moral and legal education and the construction of socialism with Chinese characteristics, emphasizing the questions of “what is it, what should it be”; explanation and argumentation involve analyzing causes of real-life issues using disciplinary skills and methods, exploring relationships between different variables, and logically and scientifically justifying and testing conclusions using empirical materials or theoretical principles, focusing on the question of “why”; exploration and construction refer to proposing ideas, strategies, and methods for solving problems based on specific situations and logical reasoning, emphasizing the question of “how to do it”.

(2) Necessary Principles for Question Setting Requirements

The specific requirements for question setting include the following necessary principles: normativity (i.e., strictly adhering to the core competency testing framework for question setting), guidance (focusing on correct value guidance), scientificity, fairness (including gender fairness, regional fairness, etc.), and appropriateness (mainly referring to the authenticity and typicality of the created situations). Of course, new principles can be added based on actual question-setting needs, but the above five principles are indispensable.

(3) Sample Test Item Information Submission Form

The test item information submission form is a significant technical achievement obtained by the testing group under the guidance of relevant departments and experts from the Ministry of Education, playing an important role in improving the quality of item setting and scoring. Here, the author provides the information submission form used by the testing group for frontline teachers’ reference. Teachers can modify this form according to their actual needs, and learning to fill out this form will help improve teachers’ question-setting abilities.

In the process of developing the new curriculum standards, the work logic of the standards group was to first form a draft of the academic quality standards, then empirically validate and modify the academic quality standards (i.e., the behavioral performance characteristics of core competencies) based on the core competency testing framework, and extract the planning for academic level examination questions. Although the officially promulgated new curriculum standards do not include the core competency testing framework, this omitted framework serves as a technical guarantee for constructing the new curriculum standards, enabling the measurement and evaluation of core competencies. With the promulgation of the new moral and legal curriculum standards, the ability to set examination questions and evaluate based on core competencies will become a key competency for every moral and legal teacher. Mastering the construction of the core competency testing framework is of significant importance for enhancing teachers’ professional qualities.

For more details, see the Renmin University of China Reprint Data “Quality Education” Issue 11, 2022