3.1 Scaffolding

There are many definitions of the term ‘scaffolding’ and there is also a lack of precision on its definition because many researchers have used this term to denote different instructional methods (Belland, 2017). Scaffolding is a process that enables a novice learner to solve a problem or carry out a task that is outside of their current skills or ‘beyond his unassisted efforts’ (Wood et al., 1976). Scaffolding has been successfully implemented in different mobile learning applications (Zydney & Warner, 2016). For instance, Hung et al. (2013) confirmed that scaffolding in their m-learning application helped students to enhance their inquiry skills in ecology. According to Santos et al. (2015), in mobile assessment, the mobile device can be transformed into a ‘more capable peer’ when scaffolding strategies are used to assist learners during the assessment activity. Thus, one of the factors that might ensure success in a learning activity is the use of scaffolding strategies. Kim and Frick (2011) pointed out that the learning activities should be in the zone of proximal development so that the activities can be done with the support and guidance of the teacher and other educational resources.

A broader perspective on the research on scaffolding was provided by Belland et al. (2015), who conducted a meta-analysis on computer-based scaffolding and found that scaffolding has a positive influence on learning with an effect size of 0.53. Scaffolding has been used as a strategy to promote the development of higher-order thinking skills and content knowledge. However, to the best of our knowledge, none of the studies has explored the effect of scaffolding in mobile-based assessment systems during formative assessment activities.

Consequently, according to the literature, it seems that scaffolding has a positive effect on students' learning outcomes. Hence, the following hypothesis is proposed in which scaffolding is the independent variable and learning outcomes is the dependent variable:

3.2 Time on-task

Time on-task measures have been used as an accurate estimation of students' learning (Kovanović et al., 2015). Learners who practice and engage with a task have better learning outcomes than those who practice less or engage less with a task (Landers & Landers, 2014; Louw et al., 2008). Landers and Landers (2014) found that time on-task strongly predicted student performance. In their study, Rawson et al. (2017) conducted a correlational study and found that there is a positive correlation between the amount of time that students spend doing the homework and their learning outcomes.

Taylor et al. (2010) concluded that students spent more time using the assessment system in a mobile application compared to the time spent on a personal computer. Moreover, students spent more time per question on a mobile device than on the personal computer. Larabee et al. (2014) found that students spend more time on reading tasks when using an iPad compared to traditional learning materials. Together these studies suggest that time on-task positively influences student performance when technology is used to present the content or mediate the learning process. Together these results provide evidence of the positive effect that time on-task has on student performance. Other researchers have concluded that time on-task influences learning performance when other variables are considered together (i.e., the interaction between other variables and time on-task; Nonis & Hudson, 2006). In that regard, some studies in the literature have reported a positive effect of scaffolding on students' time on task (Ibanez et al., 2016). This means that time on-task might be a mediating variable that mediates the relationship between scaffolding and student performance. Thus, the following hypotheses are suggested:

3.3 Motivation

Motivation is a psychological construct that explains why people make an effort to pursue a goal and why people actively work to attain that goal (Keller, 2010). Motivation is a complex psychological construct that can be studied from a wide variety of dimensions. Consequently, we focused on some specific dimensions of student motivation that are more representative in test taking such as effort, importance, test anxiety, and pressure/tension.

3.4 Feedback

Feedback is defined ‘as information provided by an agent (e.g., teacher, peer, book, parent, self, experience) regarding aspects of one's performance or understanding’ (Hattie & Timperley, 2007, p. 81). Feedback is considered to be one of the most critical factors that positively influence students' learning outcomes (Abdurrahman et al., 2018; Hattie & Timperley, 2007). Feedback is a crucial aspect of formative assessment (Gedye, 2010) which in turn improves student performance (Hwang & Chang, 2011). Feedback provided in ongoing assessment significantly improves student performance and increases motivation which at the same time improves their cognitive processes (Abdurrahman et al., 2018). Moreover, feedback helps students to recognize the gap between their desired performance (competencies) and the actual achievement (Abdurrahman et al., 2018). However, research on assessment feedback and its impact on student performance is still in its infancy (Bikanga, 2018).

Feedback helps students to be aware of their current learning status and allows them to adjust their learning strategies to achieve the learning goals. Thus, students can examine their learning behaviour (Chen & Chen, 2009). Providing individual feedback before the final exams is essential to address students' needs and help students to acquire the desired competencies (Förster et al., 2018). According to Chen and Chen (2009) and Syarifudin and Suharjito (2020), feedback from formative assessment helps students to improve their performance, and therefore feedback predicts student performance. Feedback should emphasize the effort to achieve the learning objectives (Meyer et al., 2014). In general, Förster et al. (2018) and van der Kleij et al. (2012) call for more research on the effect that some feedback mechanisms might have on students' learning outcomes. Consequently, there are still some open issues in this field. Thus, the following hypothesis is proposed in which the provision of feedback is the independent variable, and student motivation is the dependent variable:

3.5 Acceptance of mobile-based assessment systems

Recent research has identified some of the factors that influence the acceptance of mobile-based assessment systems (Nikou & Economides, 2017b) and defined the mobile-based assessment acceptance model (MBAAM). Nikou and Economides (2017b) and Alrfooh and Lakulu (2020) confirmed that some sub-constructs of the technology acceptance model (TAM), such as perceived ease of use and perceived usefulness, have a positive effect on the behavioural intention to use mobile-based assessment systems. Other researchers concluded that the acceptance of e-learning has a positive and significant effect on the students' learning outcomes (Pham & Tran, 2020). In the same vein, Larmuseau et al. (2018) found that perceived usefulness (one of the constructs of the TAM) has a positive effect on the actual use of an e-learning platform and has a positive effect on the students learning performance. Likewise, Tabak and Nguyen (2013) pointed out that students who perceive that a system is easy to use are more likely to believe that the system will improve their learning performance. Chan et al. (2015) confirmed these results by including the satisfaction factor and found that satisfaction with the system also influences students' learning outcomes. Ghosh (2016) concludes that perceived usefulness has a positive effect on the usage of an e-learning platform and usage has a positive impact on students' learning outcomes. Based on the evidence in the literature, we hypothesize that the acceptance of e-learning artefacts positively affects student learning performance. However, there is still a lack of research on the attitudes and acceptance towards the use of mobile-based assessment (Al-Emran & Salloum, 2017). One question that remains unanswered is how students' acceptance of mobile-based assessment systems might have a positive influence on student performance in the context of mobile-based formative assessment activities. Thus, in this study, some sub-constructs of the MBAAM model were considered to determine how the behavioural intention to use a mobile-based assessment system might affect student performance, and consequently, we suggest the following hypotheses (based on the aforementioned model):

The structural model based on all the hypotheses in this research is depicted in Figure 1.

4.1 Participants

To determine the number of participants (sample size), we followed the minimum R-squared method as suggested by Hair et al. (2017). In our structural model, we have nine arrows as the maximum number of arrows pointing at the learning performance variable; we defined a 5% probability error with a statistical power of 90% for detecting R² values of at least 0.25. The results show that our study would require a minimum of 88 participants.

Participants will be invited from EFL courses that correspond to the A1 and A2 levels of the CEFR (Common European Framework of Reference for Languages) at university level. Participation in the research will be voluntary. Students are going to use a mobile-based assessment application developed by the authors and called K-English in the context of a formative assessment activity for a period of 4 weeks to prepare students for the Cambridge KET official exam. The mobile-based assessment application was developed to enable students to practice vocabulary, grammar, and listening from a corpus of 305 questions/items included in the mobile application. At the same time, students practiced some skills for answering six different types of questions/items that usually appear in the Cambridge KET official exam. Further details of the application are provided in Section 4.2. Almost all of the students that are going to participate in the study may belong to the undergraduate programme in psychology, which is the largest higher education programme in the university where this study will be conducted.

4.2 The mobile-based assessment application

For the educational intervention, a mobile-based assessment application called K-English was developed. The application was developed with the support of EFL teachers, and the primary purpose was to provide an educational tool for a formative assessment activity in which students practice some of the types of questions (items) from the Cambridge KET (Key English Test) official exam. The app K-English enables students to practice vocabulary, grammar, and listening through six different types of questions in preparation for the Cambridge KET exam. In particular, the mobile-based assessment application covers the types of questions for practicing for parts 1, 2, 3, and 6 of the reading and writing sections of the Cambridge KET as well as parts 2 and 3 of the listening section (Cambridge., 2016). For the purposes of this research, a new module was developed for the K-Enlish application. This module uses the GPS (Global Positioning System) to identify the student's location to determine if the students are at the university, at home or moving in public transport. The student's location is used to display tests that have more related questions to the specific locations. For instance, questions that contain vocabulary about household items when students are at home or vocabulary about public transport when they are moving to or from the university. In some cases, when the GPS is not available, the application shows a pop-up asking students about their current location. Regarding data privacy, students will be informed about recording their location only for research purposes while the application is being used when they are asked to participate in this research. In total, the application has 305 questions for all of the six types of questions described earlier. These questions were reproduced from the following well-known books for learning English as a foreign language: Gray and O'sullivan (2000), Capel and Ireland (2004), and Andrew Betsis (2014) after obtaining the corresponding written permission from the publishers for this research study. More information about the publishers is provided in the acknowledgements. Each question was tagged to be adapted to the student's location.

4.3 Formative assessment activity with the application

As part of the EFL course, students will be invited to participate in a formative assessment activity designed together with EFL teachers in which students have to use the mobile-based assessment application for a period of 4 weeks in preparation for the Cambridge KET exam. Due to the COVID-19 pandemic, students are going to use this application as a support for their learning process at home. This activity was in line with the curriculum and the learning objectives. The application provided corrective feedback on the questions so that students could be aware of the topics they needed to improve. The scaffolding mechanism helped them complete the assessment activity as described in Section 4.2. In this study, the students' learning outcomes in the mobile-assessment application were considered for identifying the factors that might influence them during their use of the application as part of the formative assessment activity. We did not consider other learning outcomes that come from the course exams or the results of the Cambridge Exam.

4.4 Instruments

Data will be collected from two sources: an automatic monitoring mechanism integrated into the mobile-based assessment application and a self-reported instrument. The self-reported instrument consists of the following five scales adapted from the MBAAM (Nikou & Economides, 2017b): Perceived ease of use, perceived usefulness, behavioural intention to use, feedback, and user interface to collect information about the factors that influence the acceptance of mobile-based assessment systems and determine how the acceptance influence the student learning performance. To collect data about motivation, the Student Opinion Scale questionnaire (Thelk et al., 2009) is adapted, which is a well-known questionnaire for collecting data about student motivation in low-stakes tests. Both scales from this questionnaire were used: Importance and Reported Effort.

According to Looi et al. (2015), there are many studies in mobile learning research that have mainly used only self-reported measures. However, self-report measures have some disadvantages. For instance, some instruments do not provide the level of detail needed by the researcher to interpret the results. In other cases, some experiences are subconscious and cannot be fully expressed by people using a questionnaire (Barker et al., 2002). Thus, this study is different from previous research because in this study, we are going to supplement the self-reported measures with an automatic non-intrusive monitoring module developed in the mobile learning application to automatically collect data about how students interact with the application.

4.5 Research procedure

The educational intervention is planned for 4 weeks. Students are going to use the mobile-based assessment application for that period of time. The intervention will not be cross-sectional in order to reduce the novelty of the technology effect or Hawthorne effect (Looi et al., 2009) in the results of this study. By following the recommendations defined by Looi et al. (2015), we integrated the mobile-based assessment application to the curriculum and aligned the use of the application with the learning objectives of the EFL courses. The research procedure is depicted in Figure 6.

4.6 Isolating the assessment system and the learning intervention

In this research, the mobile-based assessment application is used to support a formative assessment activity (intervention) in EFL courses. The formative assessment activity implies using the application for a period of 4 weeks, as described in Section 4.3. However, the data collected for validating the structural model is gathered using the self-reported instrument in which the questions refer to the students' experience using the application only and are not related to any other activity in the formative assessment activity or any other experience in the EFL course. In that regard, the information collected to validate the structural model is not affected or biased by the learning intervention itself and allows an accurate result of the predictors of learning performance when using the mobile-based assessment application. Moreover, the monitoring module integrated into the mobile-based assessment application collects information about students' use of the mobile-based assessment application and does not collect any other information about the formative assessment activity.

The hypotheses that can be validated from the data collected with the monitoring module are H_1a, H1_b, H₂ because the variables of time on-task and scaffolding are automatically measured by the mobile-based assessment application. Moreover, the rest of the hypotheses were validated with data collected from the self-reported instrument.

5.1 Evaluation of the formative measurement model

The formative measurement model consists of the following exogenous variables with its corresponding indicators (see Figure 1): Scaffolding, time on-task, effort, importance, user interface, feedback, perceived ease of use, perceived usefulness, behavioural intention to use. To evaluate this measurement model, the redundancy analysis will be applied for evaluating the convergent validity, analysis of collinearity (VIF), and analysis of the outer loadings (Hair et al., 2017). The recommended thresholds that will be used to evaluate the measurement model according to Hair et al. (2017) are outer loadings (>0.5), Convergent validity (>0.7), Collinearity (VIF <5), Composite Reliability (>0.7), and Average Variance Extracted (AVE; >0.5).

5.2 Evaluation of the structural model

To evaluate the structural model, the bootstrapping method will be applied in SmartPLS as recommended by Hair et al. (2017). The evaluation of the structural model will provide the amount of variance explained in the student learning performance by the variables in the model. The values of R², R² adjusted, total effects, and indirect effects of the structural model will be calculated and provided in this section. Moreover, the effect size (f²) will be calculated in SmartPLS with the p-values and t-values for each path in the structural model.

5.3 Predictive relevance of the structural model

To evaluate the predictive relevance the Stone-Geisser's Q² measure will be used as recommended by Hair et al. (2017). This measure gives an idea of the extent to which the model is able to predict new values. To obtain the Stone-Geisser's Q² measure, the Blindfolding method with an omission distance of seven and using the cross-validated redundancy approach can be applied in SmartPLS. The purpose of this evaluation is to obtain an adequate predictive power and effect sizes in the model proposed.