Chapter 3 “Evaluation: A Hot Topic in High School Education”
(Sorry for writing a long article again!)

“Nabetan Journal” is a series of articles that shows you what the UTokyo FFP classes (conducted every other week) are like. I’m sorry for the extremely belated post since the previous article (DAY 2) due to personal reasons. I’m eager to catch up now. Let’s go!

DAY 3 was conducted on May 6th and 7th (right after the Golden Week Holidays…). The goals, objectives, and agenda of the class are as follows.

[Goals]
To obtain basic knowledge in evaluating student learning, to understand the significance and features of evaluation, and to be able to apply evaluation to student learning.

[Objectives]
1. To be able to explain the significance of evaluation.
2. To be able to contrast formative evaluation and summative evaluation.
3. To be able to explain any given evaluation method based on the features of evaluation.
4. To be able to create a rubric.
5. To be able to express one’s thoughts on the merits and demerits of rubrics.

[Agenda]
0_Feedback from participants on the previous class (message)
1_Review of two topics (work)
2_Evaluation (lecture)
3_Consultation about evaluation (work)
4_Rubrics (lecture)
5_Exercises in creating rubrics and evaluating with them (work)
6_The merits and demerits of rubrics (work)
7_Today’s class design (i.e., how the instructor designed today’s class) (message)

With respect to the whole structure, the proportion of participants’ activities (as shown as “work” on the agenda) was larger than that of DAY 1 and 2.

As mentioned in the section “The Significance of Active Learning” of “DAY 2 Class Design,” the important thing I would like you to learn is to make sure to output what you learned (i.e., to give it a try) in addition to input. The objective for DAY 3 is to understand “what evaluation is” and what “a rubric,” an important tool for evaluation, is through creating and using it. From now on, the instructor will deliberately reduce the support for learners (i.e., “scaffolding”) little by little and leave more time to them, which will be explained in the next class, “DAY 4 Course Design.”

As mentioned earlier, taking this course itself equals experiencing the knowledge you acquire through the course.

 

The Significance of Evaluation

“Evaluation” is a hot topic in my field recently.

New curriculum guidelines (PDF) will come into effect at high school, where I was engaged in education for a long time, from the next academic year. The guidelines are revised every 10 years, and the next revision is expected to have a large impact on the way of education according to the “Arguing Points” (Central Council for Education) (PDF). The revision will also impact evaluation methods, and I am discussing the issue with teachers working at high schools. Our discussion included the topic “the significance of evaluation,” which appeared on DAY 3.

“The significance of evaluation” is explained from the following three perspectives in UTokyo FFP.

　◯ Significance for students ① Grasp one’s level, ② Support one’s learning
　◯ Significance for instructors ③ Check and support students’ comprehension, ④ Reform one’s classes
　◯ Significance for institutions (schools/boards of education) ⑤ Assure its quality, ⑥ Accountability

　*Evaluation is not only for grading students.
　*Evaluation is not a goal but a starting line.

High school teachers are provided with “The Handbook on the Method of Learning Evaluation: Upper Secondary School Edition” (National Institute for Educational Policy Research, Ministry of Education, Culture, Sports, Science and Technology) as the new curriculum guidelines are implemented. The following sentence is clearly written in the Handbook.

“Teachers should grasp learning outcomes (i.e., what kind of skills students acquired) accurately, improve their guidance, and let students proceed to the next learning by reflecting on their own learning.”

The subject of “grasp,” “improve,” and “let” is “teachers,” so let’s first look at this sentence through the lens of “significance for instructors.”

Significance of Evaluation ③ “Check and support students’ comprehension”
・(Instructors should) grasp learning outcomes (i.e., what kind of skills students acquired) accurately. (check students’ comprehension)
・(Instructors should) let students proceed to the next learning. (support)
Significance of Evaluation ④ “Reform one’s classes”
・Instructors should grasp learning outcomes and improve their guidance.

Next, let’s extract the section where the subject is “students” from this sentence.
・reflecting on their own learning
・students proceed to the next learning
To realize these things, the evaluation must be meaningful to students as described as follows.

Significance of Evaluation ① “Grasp one’s level”
・Students can grasp by themselves what and how much they were able/unable to accomplish.
Significance of Evaluation ② “Support one’s learning”
・Students can identify by themselves how they should improve their next learning by grasping the level of accomplishment.

When applying “evaluation methods” indicated in the Handbook to high school classroom settings, it is important to look at “the significance of evaluation,” the topic of DAY 3, from both perspectives of students and instructors to appreciate the values shown in the sentence: “Evaluation is not only for grading students.” (The significance of evaluation for institutions (⑤&⑥) is also controversial in high school, so I would like to take up the issue in another article.)

Now, let’s get back to UTokyo FFP classes from high school education. “The significance of evaluation” is also connected to “the ADDIE model” we learned on DAY 2.

The fifth stage of this model is “evaluation.” The evaluation here means “the evaluation for class reform” conducted by the instructors in various spans ranging from every class to every unit or every semester. When a series of various activities (Analysis/Design/Development/Implementation) comes to an end, you need to reflect on the respective ADDI activities, and connect the reflection to the next activities, as described as “Close the Loop!” It exactly means that “Evaluation is not a goal but a starting line.” By the way, you need to continually go back and forth between “A-D-D” and reflect on the activities before reaching I (=Implementation), or you need to conduct a “formative evaluation,” as described in the next section.

 

Summative Evaluation and Formative Evaluation

Dr. Kurita says, “You should come up with these two at the same time when you hear the word ‘evaluation.'” Then, how do these two differ from each other? Answering this question is exactly the second item of the objectives: “To be able to contrast formative evaluation and summative evaluation.”

　Summative evaluation: “For measuring the learning outcomes or deciding whether to pass or fail a learner”
　　・It can be used for deciding whether to pass or fail a learner and is conducted after learning.
　Formative evaluation: “For improving the learning process of learners and providing feedback to instructors for small-scale improvement of learning activities”
　　・It mainly functions as feedback which helps learners modify their learning activities one by one and is conducted during learning.

What Dr. Kurita emphasized is as follows: “You should keep in mind that actual evaluation is conducted by combining these two elements. The most important thing for the instructor is to keep thinking about how these two evaluations should be combined to help smoothen students’ learning process.”

I used to distinguish these two by regarding summative evaluation as evaluation for grading, and formative evaluation as evaluation for feedback during classes, and thought that they were incompatible with each other, but Dr. Kurita says, “It’s not that you can only choose either one. Some types of summative evaluation have formative aspects and vice versa.”

Evaluation can of course be used to measure accomplishment, and you can decide whether to pass or fail a learner or grade him/her based on that evaluation, but you can also provide feedback to improve his/her learning for the next step by clarifying what he/she needs. We should not view these two types of evaluation as a binary system, but we should regard them from the perspective of “the purpose of evaluation.” And here, again, the subject of the sentences describing these two types of evaluation may not only be instructors but also students.

What to Evaluate

We started with how to view “evaluation.” Now, considering the specific procedure for conducting evaluation, we are forced to answer the following question: “What should we evaluate?” It became another hot topic in the discussion with the high school teachers.

UTokyo FFP listed what to evaluate as follows: “Knowledge/Comprehension, Thinking/Decision-making,” “Skills/Expression,” and “Interest/Motivation/Attitude.” These are almost perfectly aligned with the evaluation categories indicated in the present curriculum guidelines for elementary and secondary education. These categories are rooted in Bloom’s Taxonomy (as described on DAY 2): “Knowledge (cognitive domain),” “Skills (psychomotor domain),” and “Attitudes (affective domain).”

In the new curriculum guidelines, the evaluation categories on “what to evaluate” have changed into “Knowledge/Skills,” “Thinking/Decision-making/Expression,” and “Attitude for learning actively”; please refer to the “Arguing Points” given at the beginning of this article for background information. “Attitude for learning actively,” in particular, largely overlaps with Fink’s Taxonomy of Significant Learning as described as follows.

【Foundational Knowledge】
　　　Understanding and remembering:  ・Information　・Ideas
【Application】
　　　・Skills　・Thinking: Critical, creative, & practical thinking　・Managing projects
【Integration】
　　　Connecting: ・Ideas　・People　・Realms of life
【Human Dimension】
　　　Learning about: ・Oneself　・Others
【Caring】
　　　Developing new　・Feelings　・Interests　・Values
【Learning How to Learn】
　　　・Becoming a better student　・Inquiring about a subject　・Self-directing learners

The “Period for Integrated Studies” and the “Period for Inquiry-Based Cross-Disciplinary Study” are conducted respectively at elementary schools and junior high schools. Carefully look at the new curriculum guidelines, and you will find that these “Periods” should be closely connected to the educational goals of schools, which means that they are placed to have different roles from the traditional learning of subjects. It is also remarkable that what these “Periods” aim aligns with much of the Taxonomy of Significant Learning.

When it comes to practices such as classes, it is necessary to address specific “objectives” as well as “evaluation categories,” a large framework, as shown above.

Even if instructors are the only ones who evaluate the learners, they should identify “what specific objectives” the learners should work on and “what exactly will be evaluated” in the phase of class design (i.e., Design in the ADDIE model). In addition, learners are those who evaluate themselves, so it is necessary for instructors to share “what exactly will be evaluated (and evaluation methods)” as well as “specific objectives” with the learners in an explicit way. This is a requirement stated in elementary and secondary education as “integration of objectives and evaluation”; the topic will be taken up in the next session “DAY 4 Syllabus/Course Design.” It was also referred to in the session of DAY 2. There was a thorough explanation and group activity on “objectives,” saying, “Objectives should be shown with observable verbs (i.e., expressions used for outputting), and will become evaluation categories.”

[By the way…] I recently read a book called, “Your Biology (Hill, S.) (*Japanese translation supervised by Matsuda, R., and Okamoto, T. Hakusuisha Publishing).” It is a textbook for 13–14-year-olds in the Netherlands and has “Objectives” at the end of every Unit as a “Wrap-up,” followed by a “Test.” As an ex-teacher of biology, I am interested in the content of the textbook itself, but what also intrigues me is that it displays “objectives” and “measurement of knowledge comprehension” as a set from the perspective of “evaluation.”

Dr. Kurita explained the effects of the cycle of practice and feedback, which means combining “practice” (activities for knowledge input) based on objectives and accurate “feedback” (information given to the learner based on the outcome of practice that works as a guide to the next step), in enhancing learning quality while delivering specific classes. Chapter 5 of How Learning Works (Japanese translation published by Tamagawa University Press) elaborates on this topic, which I consider important for teaching in the classroom.

 

How to Evaluate

Evaluation methods and learning objectives are two sides of the same coin. It is necessary to adopt evaluation methods that align with the preset learning objectives.

DAY 3 focused on “Exercises in evaluating evaluation” and “Exercises in creating rubrics” as shown below and skipped explaining specific evaluation methods. If you are interested in the details of such methods, please refer to “Learning Assessment Techniques: A Handbook for College Faculty” (A Japanese translation published by the University of Tokyo Press in 2020). Diverse evaluation methods are given based on “Fink’s Taxonomy of Significant Learning.” I’m sure this book will be greatly helpful to teachers when the “Period for Integrated Studies” and the “Period for Inquiry-Based Cross-Disciplinary Study” are about to be positioned in the center of school educational activities. It also shows a lot of rubric examples.

Let’s get back to “Exercises in evaluating evaluation (Consultation about evaluation).”

In this exercise, participants were asked to advise someone saying, “I am in charge of XXX course. I give students assignments like this, evaluate them in this way, and I am worried about XXX.” They discussed ideas from the viewpoints they learned in the lecture: evaluation methods, evaluators, and evaluation of evaluation (shown below), and proposed their ideas in a Google Form.

　<Evaluation of evaluation>
　Reliability: To what degree you could obtain the same results no matter how many times you conduct the same examination with the same group (Reproducibility of outcomes and accuracy of tests)
　Validity: Whether the evaluation method you have adopted can really measure the skills and behaviors you are focusing on (Appropriateness of the evaluation method)
　Efficiency: Whether it is easy to conduct and grade (The practicality of the evaluation method in terms of time and economy)

Participants should identify which aspect of the above three was problematic and propose ideas on what the client should do.

After each participant filled in the Google Form, their responses were shared in the classroom. The activity of advising on a clear problem was relatively simple; change evaluation methods, and you can improve reliability, validity, and efficiency. Therefore, participants were all capable of developing appropriate responses to the problem.

And this is what Dr. Kurita said to everyone:
“It’s easy to advise someone else, but once you become the one who delivers classes, many people are likely to overlook these aspects, so please do not leave the basic knowledge you learned here as some kind of trivia, but make sure that you can apply it to your own classes.”
Hearing her words, I broke out in a cold sweat as an ex-teacher at high school, haha!

 

What Is a Rubric?

One hour and 15 minutes out of the class time of three hours and a half were spared for lecture and activities regarding “evaluation,” followed by a break and stretching exercises time, and the remaining two hours were allotted to this topic. Quite a long time was spared for this activity, but as Dr. Kurita says, “Creating something is far from just knowing it.” To spare time for creating rubrics as much as possible, participants learned about the basics of rubrics before the session with videos. (This strategy is called a “flipped classroom.”)

　<Video clips on rubrics>

The “Period for Inquiry-Based Cross-Disciplinary Study” has made rubrics a hot topic in high school, and there is an increasing demand for creating rubrics, so here I would like to elaborate on what I think is important in that context.

“Rubrics divide an assignment into its component and provide a detailed description of what constitutes acceptable or unacceptable levels of performance for each of those parts” (Stevens & Levi, 2013). Dr. Kurita says, “This is where the values of rubrics lie,” and cited the following sentence. It describes the first step when using a rubric.

“The first step in constructing or adapting any rubric is quite simply a time of reflection, of putting into words basic assumptions and beliefs about teaching, assessment, and scholarship.”

Rubrics are for learners, but before that, they are for instructors. The value of rubrics lies in the point that they enable instructors to ask themselves what assumptions and beliefs they have when asking learners to work on an assignment and what they are trying to evaluate by putting them into words by creating a rubric.

Rubrics align with every evaluation method except multiple-choice questions which have distinct right or wrong answers. They can also improve all three aspects used for evaluating evaluation: reliability, validity, and efficiency.

Here’s the typical way of using rubrics:

1. An instructor creates a rubric.
2. The instructor hands out an assignment and a rubric to students.
3. The students complete the assignment while referring to the rubric as a guiding principle.
4. The students submit the assignment. (They can also attach the rubric with which they self-evaluated their assignment.)
5. The instructor or another person grades the submitted assignments. (The rubric enables people besides the instructor to grade the assignments.)
6. The instructor returns the assignments to the students with the rubric attached.

Students can also join the process of creating a rubric. (In that case, creating a rubric itself becomes a learning process.) They can also peer-evaluate the assignments with each other. (Evaluation using a rubric deepens their understanding of the learning material.)

Rubrics are not a tool secretly possessed by an instructor as his/her evaluation criteria. The instructor should present rubrics to the students and involve them in learning activities by using rubrics. I believe that would further enrich student learning in various ways. And of course, that cannot be successful without building a good relationship between the instructors and students, and among students themselves (i.e., creating a good learning environment).

In relation to that, when used amid learning activities, rubrics can become a tool for formative evaluation by putting into words the situation around student learning. On the other hand, they can become a measurement method for summative evaluation, if they are used for assessing the final outcome. Here again, it is important to see things from two perspectives: formative evaluation and summative evaluation.

 

Creating a Rubric and the Merits and Demerits of Using Rubrics

Following the explanation of the points of rubrics, participants created a rubric by following the procedure. For details of the procedure, please refer to the lecture materials available on OCW (PDF).

The activity is not just for creating a rubric, but is also structured as follows:

1. Assign marks to four writing assignments on a scale of 1–10 by briefly considering what dimensions they have. (Google Form)
2. Create a rubric for this assignment in groups.
(Participants work in breakout rooms and create a rubric on Google Slides. They bring the dimensions they came up with individually and narrow down and fix them through discussion.)
3. Walk around (gallery walk) and explore other groups’ rubrics.
4. Refine the rubric in groups.
5. Assign marks to the four writing assignments on a scale of 1–10 by using the final rubric. (Google Form)

Creating and using a rubric and contrasting the evaluation processes between those with or without rubrics helped participants effectively gain materials (i.e., experience) for considering the advantages and disadvantages of rubrics.

Finally, participants discussed the advantages and disadvantages of rubrics in groups and shared their outcomes by filling in Google Forms. Here, again, they examined the significance of evaluation for both “students” and “instructors” and put them into words. This process helped them realize that the subject of the sentences for describing evaluation activities can be both instructors and students.

 

That’s all for today. Sorry for writing another long article, but I didn’t want to separate it into two parts.

A slide called “Design” was presented at the end of the session, as usual, showing how Dr. Kurita designed the session. There were four points:

・Flipped Class
・・Participants learned about the basics of rubrics before the session by watching videos.
・Works based on cases
・・The instructor provided a situation of giving advice on evaluation and evaluating evaluation that allowed the participants to apply highly abstract knowledge to specific cases.
・・The participants created and examined a rubric by assigning marks on particular writing assignments.
・Gallery Walk (variation)
・・The participants were able to explore what other groups worked on online.
・Various ways of sharing ideas
・・The participants experienced various ways of exchanging their ideas online.

Participation in this session as a learner itself becomes an opportunity for him/her to learn as an instructor, so I think this slide is highly meaningful to the participants.

 

To be continued in the next article on the session “DAY 4 Course Design (Syllabus).”
(By the way, we have already finished DAY 7 “Microteaching session (FINAL)”…)

See you next time!

 

Here are the recommended websites related to UTokyo FFP. For more details on the course materials and AY2020 course schedule, please click the following links!
(Official) UTokyo FFP Website
UTokyo OCW “Teaching Development in Higher Education” (UTokyo FFP AY2020)　　Interactive Teaching (Video Clips)
Osami Nabeta
Research Support Staff
Center for Research and Development of Higher Education

Chapter 2-2 “We Passed the Turnaround Point of DAY 2, Filled with Essential Topics for Class Design. Here’s the Second Half!”

Let’s review Part 1 before moving on to the next topics!
We should always repeat important things, and that makes it easier for us to acquire them (as described in the last session.)

Part 1 consisted of the following topics:
(1) Reflection on making research presentations
(2) Goals and objectives / Where are we?
(3) Work: “Students don’t like statistics?” Let’s give advice to the professor.
(4) Premise
(5) A model of motivation “expectancy/value/environment”
(6) Significance of class design
(7) The ADDIE model (a model for instructional design)
(8) The first step to creating a class design sheet: work on “setting objectives”
As listed above, Part 1 was filled with information related to premises and frameworks necessary for class design, and these topics were not just conveyed by Dr. Kurita to the participants one-sidedly but were also provided through time for the participants to discuss and use them. So that’s “active learning,” which the participants already “experienced” on DAY 1 and DAY 2. This was the main topic in Part 2.

(9) Definition of active learning
Dr. Kurita started with the following question: “If you were asked what active learning is, how would you answer?” The participants answered the question via Sli.do (a highly anonymous Q&A app), and their replies were shared with everyone. (This activity is also categorized as active learning.) Receiving diverse answers from the participants, Dr. Kurita said, “In fact, many people define the term in many different ways,” and showed three definitions. (Slide 51)
[Just talking to myself] The term “Active Learning” was specified in the reform of higher education in Japan, but it rapidly became a trend after the issue of “(Consultation) What the Standards for Curriculum in Elementary and Secondary Education Should Be (November 20th, 2014)” in elementary and secondary education. The term was stated as “Active and deep learning through dialogue” in “Curriculum Guidelines (Revised in 2017/2018)” which was developed through discussion based on the Consultation.
In the definition (2012) by the Ministry of Education, Culture, Sports, Science and Technology, “educators and learners” is the subject of the following: (a) communicate with each other, (b) compete with each other, (c) stimulate each other, (d) develop intellectually, and (e) create a space. “Learners” is the subject of the following: (f) actively discover problems and (g) find solutions. I used to think that the subject of (b), (c), and (d) only referred to students, and (e) was for teachers, based on experiences in my job. Eventually, through trial and error, I realized that I, a teacher, was also the subject of (b), (c), and (d) along with students. If I had reexamined my classes when I first encountered this definition, I could have shortened the trial-and-error period, so I feel sorry for my students in those days.

(10) The point of active learning
“Strategies to encourage active learning are not a goal but a means.”
Aside from things in education, it often happens that the means is replaced by the goal… As described in “Part 1” and the past article in Nabetan Journal, “Goals and Objectives” are crucial, and replacing them with means may deteriorate the value of the learning activity itself.
Dr. Kurita provided the participants with the following three important perspectives (or questions):
・Do they correspond to the goals and objectives of your class?
　・Do they contribute to student learning?
　・Do they consider students’ perspectives? (motivation/academic level/relevance)
I believe these three questions are important not only in terms of active learning but also in every aspect of creating a class from design to implementation, and I think teachers should always keep them in mind.

(11) Effectiveness of active learning
The pre-post test scores of the student groups who took either two-way or one-way classes, each divided in terms of their scores (high/intermediate/low) were shown with graphs on Slide 53. Dr. Kurita asked what the graphs implied. The participants again replied to her question with Sli.do. (I really recommend this tool!)
The graphs show only the test scores, but it is remarkable that two-way classes have a huge positive impact on learners compared to one-way classes in various ways, such as a decrease in dropouts and an increase in their motivation, as shown by many other studies.

(12) Risk in active learning
While there are outcomes and effectiveness, there is also a risk in active learning. What matters is whether it realizes the points of active learning as listed in (10).
[Just talking to myself] When I was teaching as a school teacher, I had many opportunities to observe classes at various schools that they called “conducted in an active learning style.” It also applies to my own classes, but I often felt the risk of active learning in such classes.
They happened to overlook “class design” as described in Part 1. Active learning or “active and deep learning through dialogue” is just a method, and “goals and objectives” should come first. In addition, it is necessary to “analyze” the situation of learners and their learning environment. One elementary school teacher referred to it as “a problem that comes before adopting an active learning style,” and that was exactly to the point. With our knowledge of the “premise” in Part 1 and the ADDIE model, we can now verbalize what the problem is by pointing out the lack of Analysis and Design. 

(13) Active learning strategies
Based on the “premise” of active learning explained so far, the session moved on to introducing specific active learning strategies and letting the participants experience them.
There are various active learning strategies for different group sizes: minute paper, self-evaluation, peer-reviewing, brainstorming, jigsaw method, case studies, PBL, TBL, poster tour, etc. The following three were explained thoroughly in the session.
(A) Asking questions: Ask students questions.
(B) Think-Pair-Share: Let students think about the topic individually and then share their ideas in pairs.
(C) Peer Instruction: Following a brief lecture or students’ pre-class learning, the instructor poses a multiple-choice question (ConcepTest). Students discuss with their peers, and then the instructor gives an explanation.
For more details, please refer to Slides 56–66.
Dr. Kurita said that the following are the points in implementing any method, which overlap with the points listed in (10).
1- Don’t let the means become an end. Don’t set participation in activities as the goal.
2- Clarify the goals and objectives of activities.
3- Explain the merits of adopting AL. Cooperation enhances learning as demonstrated in research. (Johnson & Johnson, 2009)
4- Give specific instructions. Clarify the instructions by telling the students “what” and “how” they should do and “how long” they should get involved in the activities. It is also recommended to include “why” they should get involved in the activities.
We should always repeat important things. That’s the principle!
[Just talking to myself] I often hear at elementary and secondary schools that students who are assertive and active become the only ones to say something and the rest of the students just remain listening to them in group activities, but I think it is important to design classes in accordance with the learners. For example, adopting methods such as (A) Asking questions and (B) Think-Pair-Share in small steps will help you create a learning environment where students can express their ideas without worries.
At the same time, even in the case of an active learning style for a small number of students, teachers need to design and prepare classes carefully and flexibly change the style, especially at the beginning of a course. As the students get the feel of active learning, teachers need to decrease their support (i.e., removing the scaffold), but at first, they need to support them carefully by grasping their situation (i.e., scaffolding). And this is also advice to myself. 

(14) Creating a class design sheet
As mentioned in Part 1, participants moved on to designing “a 6-min class on a topic in your field for novice students.” The design will be applied to the microteaching sessions on DAY 6 and DAY 7. They designed a class that realizes the objectives they set in the first half of the session by using a design sheet. They were provided with plenty of information so far, but now it was time for them to actually use it. The first 15 minutes were for creating a design sheet individually, and then they shared their sheet (still unfinished) with another participant in the breakout rooms. They explained their own sheet in six minutes per person, organized their ideas, and found problems and new insights through discussion.
Finishing the class design sheet is an assignment to be submitted by May 5th, but the participants were able to share their ideas and discuss them in pairs, which makes the difference compared to just working on it alone as homework. Therefore, even if they couldn’t finish their sheet within a limited time, these 15 minutes and 12 minutes became precious time for them.

(15) Class design of DAY 2
What you learn in UTokyo FFP is expressed in the class design of the program itself, so participants can learn by doing. To recognize that, the “explanation of the class design of today’s session” is important. As described in the past article in Nabetan Journal, you can also call it the class structure. Today, the following points were explained:
・This class was conducted by using a strategy called “Flipped Class.” The assignment of DAY 1 worked as preparation for DAY 2.
・Structure: an introductory section, main section, and concluding section It also applies to DAY 2.
・Various issues and learning contents are related to the final assignment (i.e., class design sheet).
・Peer-reviewing (as an assignment): It is frequently used in online courses.
We have finished only two sessions so far, but the various topics we have learned are already related to practice.

That’s all for DAY 2. Part 2 was also filled with important topics. The participants should experience what they learned as much as they can to prepare for the following sessions. As I mentioned before, everything we learned will be connected to future topics.

And let me conclude the article with another [Just talking to myself].
Reflecting on my own teaching experience at school, I frankly think that I could have shortened the trial and error if I had known the topics of DAY 2. I regard the significance of UTokyo FFP as letting those who aim to become faculty, which doesn’t need a teaching license, learn about instructional methods, but they were not in the curriculum for obtaining a teaching license for elementary and secondary education either, at least in my generation. Many of my colleagues in the same generation and I myself learned about such findings in instructional methods through trial and error, from experiences through interacting with pupils and students. I think many of the mistakes I made shouldn’t have happened in the first place, but that was the reality. But how are the teachers today? With new Curriculum Guidelines and the need for a new type of education, I truly hope that future/present teachers can have an opportunity to learn about such findings before becoming a teacher or while working as a teacher at school. Knowledge must always be practiced, I know, but there is a tragedy that comes from the lack of knowledge in any field. When I finished DAY 2 in the last semester, I thought from the bottom of my heart that many things can be resolved by just knowing them.

See you next time!

For more details on UTokyo FFP, please click the following links! (Let me repeat this again and again for we should always repeat important things!)
(Official) UTokyo FFP Website
UTokyo OCW “Teaching Development in Higher Education” (UTokyo FFP AY2020)
Interactive Teaching (Video Clips)

Osami Nabeta
Research Support Staff (FFP)
Center for Research and Development of Higher Education

Chapter 2-1 “DAY 2 Is Filled with Important Topics! I Had to Divide the Long Article into Two…”
I was supposed “to shorten the length from the next article on DAY 2” as I wrote in the last post, but DAY 2 was extremely rich in content as the instructor Dr. Kurita herself said. I tried to summarize the points briefly, but “extremely rich content” literally means that there are no points to be omitted…so I decided to divide the article into two parts!

(1) Reflection on making research presentations
The class began with participants sharing their thoughts (in groups of three) on the assignment of DAY 1 (i.e., filming and submitting a 1-min video on their research presentation). This activity is to let the participants recall what they learned in the last session and helps them output their thoughts online; it must have been not so difficult to put into words their own experiences. In that sense, it worked as an icebreaker for DAY 2, but “giving a research presentation” itself is an important component that is linked to the next topic. I’d like to talk about that later on.

(2) Goals and objectives / Where are we?
These are shown in every class as the key components of the “class structure.” Please refer to the past article in “Nabetan Journal” for more details on what “class structure” is.

(3) Work: “Students don’t like statistics?” Let’s give advice to the professor.
Participants were given an assignment in the last session to advise an imaginary professor who was in trouble. They watched a video “Motivation: Expectancy-Value Theory” and thought of one or more pieces of advice from three viewpoints: expectancy, value, and environment. They wrote down their individual thoughts on sticky notes on a Google Slide shared by each group, followed by a group activity to organize the members’ ideas and see if there were any other points to be added.
This activity also worked for participants to experience a “Flipped Class,” which appeared at the end of the session. It aligned with the “class structure” that enabled learners to experience the essence of a topic first before learning about that topic in a lecture format. This activity also worked as an introduction to “motivation,” which was one of the major themes for DAY 2.

(4) Premise
The lecture slide said, “Encouraging and maintaining students’ motivation are essential to their active learning.”
[Just talking to myself] Through my involvement with the environment of education until today, I encountered a lot of situations that made me think that whether you are really taking the above sentence seriously as a premise makes all the difference in becoming a good teacher. I’d like to talk about this topic with teachers at elementary/secondary schools.

(5) A model of motivation “expectancy/value/environment”
Following the above activities and showing the “premise,” “expectancy/value/environment” was explained as one of the models of motivation. (Link 1/Link 2) The lecture was conducted in a limited time, but the participants were already knowledgeable enough to receive the information thanks to their preparation before the class. As described in the last slide of the section titled “What educators can do,” “Think of what would hold high value for students,” “Figure out some ways to enhance students’ expectancies,” and “Create a cooperative environment” are the three points based on the premise and will work as a basis for designing lectures for microteaching sessions.
There are various models of motivation, and another one is called the ARCS model. Here is a video clip you can refer to.
[Just talking to myself] I always kept the learners’ motivation in mind as a teacher, but I did not use such a specific list of items for describing motivation then, so what I did at school must have been somewhat vague. And without such a list, I had no criteria to reflect on my classes, so I think I could not work on my class reform from the perspective of motivation. If I have a chance to teach a class once again, I would like to clarify the items related to motivation. Teachers at elementary/secondary schools, how about you?

(6) Significance of class design
A class refers to every session, and a course refers to a series of classes as a whole. Speaking of this course, UTokyo FFP (with a total of eight classes) is considered a course, and DAY 1, 2… are classes.
The significance of class design was described as follows:
・It allows efficient use of limited time.　・It enables you to choose instructional methods suitable for goals and objectives.　・It makes it easier to improve your classes.　・It makes it easier to share your ideas on class design with others.　・It motivates students.
[Just talking to myself] When I discussed with the teachers at elementary/secondary schools what the significance of class design is and who it is significant for, the following four perspectives came out: 
　a- An efficient use of class time (for learners/teachers)
　b- Planned utilization of instructional methods (for learners/teachers)
　c- Sharing of knowledge and skills (among teachers)
　d- Effective identification of the target when working on class reform (among teachers)
The fact that “a” and “b” are considered to be significant for learners seems to reflect the current need for teachers “to emphasize classes as a place where learners learn rather than a place where they teach.” It simply states the obvious, but most classes I have seen before were those where the teacher only knows the class design and the learners just try to catch up with what the teacher presents, and it also applies to my own classes in the past…
Our discussion further developed, and one of the elementary school teachers proposed the following idea:
“Speaking of ‘d,’ in light of the fact that classes are a learning space for the learners, identifying the target of class reform should be done through an exchange between the learners and teachers (who designed the classes). Under the trend of emphasizing learning from each other, it also must be done among learners themselves in terms of learning how to learn.”
Therefore, I would like to revise the item “d” into the following expression:
　d- Effective identification of the target when working on class reform (among teachers, between teachers and learners, and among learners)
I regularly joined the class observation at the elementary school where the proponent belonged. When the teacher proposed the unit design to the students, I saw them expressing their ideas such as, “Why are you planning to allot three classes for that activity? Deciding from the classes so far, I think two are enough.” “Learner-centered” has now become a word that frequently appears everywhere, but I believe it is such an exchange between the teachers and learners that truly realizes the word in terms of creating the classes together through their mutual respect.

(7) The ADDIE model (a model for instructional design)
The next point is, “What should we specifically do for designing a class?” The ADDIE model was shown as one of the models for class design.
It proceeds in the order of A (analysis), D (design), D (development), I (implementation), and E (evaluation), and as described as “CLOSE THE LOOP!”, E is linked with A again and the cycle is repeated like ADDIEADDIEA… It is sort of a growing/changing spiral model rather than a repetitive cycle. For more details, please refer to the video by clicking the above link. Another point is that this is not a one-way direction. It requires going back and forth constantly between each phase, and E (evaluation) is conducted not only at the end of a class or a course but also in each phase of ADDI as necessary. (Self-evaluations, student ratings, and external reviews were shown as examples of evaluation in the session, but we will further delve into “Evaluation” as the main topic in DAY 4.)
[Just talking to myself] I introduced this ADDIE model during the above-mentioned discussion with the teachers at elementary/secondary schools on class design, and almost everyone mentioned as follows:
“Speaking of ‘Development,’ we call it ‘Material Development’ and spare quite a long time for that, and regarding ‘Implementation,’ we call it ‘Instructional Technique” and take it up as a topic in teacher training. We realized that we already have taken good care of these two aspects, but we were not taking time to work on ‘Analysis’ and ‘Design.'”
“If ‘Analysis’ and ‘Design’ are insufficient, it is likely to end up just delivering a self-satisfied class no matter how much you take good care of ‘Development’ and ‘Implementation.'”
“‘Evaluation’ tends to focus on whether the learners’ outcome is good or bad, and in the latter case, we do review ‘Development’ and ‘Implementation,’ but that’s all. I think that’s because ‘Analysis’ and ‘Design’ are insufficient in the first place.”
They said that they would like to reflect on their own class design from the perspective of the ADDIE model, especially from ‘Analysis’ and ‘Design.’
Therefore, I am holding an informal meeting to introduce the ADDIE model to school teachers and discuss the topic with them. If you are interested, please feel free to contact me. The ADDIE model in fact has already become one of the topics for in-school training at the above-mentioned elementary school. I’m excited to hear the outcomes!

(8) The first step to creating a class design sheet: work on “setting objectives”
The next activity was “setting objectives” as the first step to designing a class for the “microteaching session” (DAY 6 and Day 7). By the way, the topic of the microteaching session is “A 6-min class on a topic in your field.” The participants must bring up the “research description” they wrote in the application document again here, after refining it through self-introduction and creating a video. According to Dr. Kurita, “six minutes for explaining one topic” is the minimum unit that includes the components of a class. There is a wide variety of microteaching styles and research on microteaching, but that amount of time and content seem to be the best option that fits the course design of UTokyo FFP (in terms of the learning environment with a limited amount of time and people).
There was a lecture on “Goals and objectives,” which are important in designing a class (and in the ADDIE model) prior to this activity. I also explained these topics in the past article in Nabetan Journal. Please also refer to Bloom’s taxonomy and Fink’s taxonomy of “significant learning” which appeared in the lecture as related topics.
Furthermore, Gagné’s nine events of instruction and systematization of knowledge through presenting a graphic syllabus and actively associating one’s knowledge were shown as tips for structuring a class. By the way, speaking of the active association of one’s knowledge, the topics in DAY 1 and 2 were linked to one another in this UTokyo FFP course, and all the topics will further develop and connect with one another in DAY 3 and following sessions. Stay tuned!

DAY 2 then moved on to the next topic “Active Learning,” but that’s all for this long article (perhaps because of too many “talking to myself”!). To be continued in Part 2.

See you next time! (I’m fully determined to post Part 2 before DAY 3!)

For more details on the course materials and AY2020 course schedule, please click the following links!
(Official) UTokyo FFP Website
UTokyo OCW “Teaching Development in Higher Education” (UTokyo FFP AY2020)
Interactive Teaching (Video Clips)

Osami Nabeta
Research Support Staff (FFP)
Center for Research and Development of Higher Education