The cognitive process of complete problem solving

Diagnosing students’ learning difficulties plays an important role in educational settings currently, but it is not easy. A typical diagnostic method usually involves testing, which uses a pen-and-paper test. Specifically, students receive a test paper, and they are asked to finish it. After students answer the question on the test paper, the teacher will assess their learning qualities based on their answer results. In other words, the teacher diagnoses students’ learning difficulties based on problem-solving results.

Problem-solving refers to common tasks encountered during the learning process. The aim of problem-solving is to find unknown information based on known information, and the known information is usually presented in texts and attached figures. If students successfully find the answer, it means that the problem is correctly addressed.

Problem-solving can test the problem-solvers’ ability to integrate relative knowledge and then explore their learning situations. Therefore, problem-solving plays an important role in diagnosing learning. PISA also lists problem-solving competence as one of the key indicators of the assessment of teenagers' literacy.

In light of the importance of problem-solving, researchers explore the learner’s problem-solving process, diagnose their learning difficulties, and then assist their learning effectively. However, as aforementioned, the traditional diagnostic method is a pen-and-paper test, which merely presents final content of students’ problem-solving. It is difficult for this kind of method to record students’ complete problem-solving process and help teachers observe it. Therefore, to fill the gap in recording and diagnosing students’ problem-solving processes, we use eye tracking and tablet devices to attempt to record their complete problem-solving processes.Researchers can record solvers’ visual attention with the eye tracker(Rayner, 1998). In particular, the unobtrusive manner of eye tracking facilitates objective observations of attention-related evidence for the problem-solving process. Studies suggested eye-tracking was promising in directly examine the underlying cognitive processes (Johnson & Mayer, 2012). For example, research evaluated the spatial abilities, which is essential for mathematics learning, by using eye-tracking devices (Wang et al., 2014). Eye tracking can be integrated with instructional technology, such as computer games, to investigate how does participants’ attention change (Alkan & Cagiltay, 2007). Zheng and Cook (2012) suggested eye-tracking can reveal a change in several constructs of cognitive load in solving complex problems. The fixation duration can be considered as the time required to interpret visual information (Rayner, 1998). Additionally, before analyzing the eye movement, the Area of Interests (AOIs), which include the important information in the frame, were set. Three eye movement measures are promising for differentiating between successful and unsuccessful solvers (John J. H. Lin & Lin, 2014): (1) Dwell time (DT): Summation of the duration across all fixations within an AOI. (2) Fixation count (FC): Total fixations falling in the AOI, and (3) Run count (RC): Number of times that an AOI is entered and left.

In addition to tracking eye movements, recording handwriting is promising to observing problem solving as well, as solutions/answers reflect the reasoning process, which is fundamental to identifying the difficulties that solvers encounter. Typical geometry problem solving involves reading (input phase) and writing (output phase). Previous studies emphasized the “input phase” ; the “output phase” was simplified by providing multiple-choice options via keyboard responses (Madsen et al., 2012) or verbal protocols. An improved design, which introduced handwriting as an input device, enables solvers to express thoughts more intuitively, and makes it feasible to observe the process of complete problem solving (CPS). The term was used by John J. H. Lin and Lin (2014) to stress problem-solving with both input and output phases. The integrated technique can monitor the allocation of visual attention, record the handwriting/drawing, and specifically capture the consecutive signal switching between eye and hand. J. J. H. Lin and Lin (2018) used this design to analyze students’ difficulties when they are solving geometric problems (similar triangles).