Assessing Course Outcomes and Predicting Grades in an Undergraduate Level Program: Insights from Performance Analysis and Factor Regression Modeling

As part of an educational program, course outcome assessments play a significant role in evaluating students’ understanding of the course content they are learning and their ability to apply it in a real time situation. The quality of these assessments also influences student motivation, which affects their engagement in the classroom as well as their ability to learn effectively. Students can become active, engaged, and motivated learners when assessments are meaningful, aligned with learning objectives, and supportive of student growth. As a key indicator of quality in education, this research aims to evaluate the effectiveness of the courses in achieving the desired learning outcomes. This study employs an exploratory data analysis (EDA) to analyze the intricate relationships among variables and their impact on student performance. We use descriptive statistics to summarize the data and to give a clear picture of the achievement levels for the different course outcomes based on the data. In addition, factor analysis and regression modeling are employed in this study to analyze the relationship between course outcomes and grade predictions. We employ factor analysis to capture the essence of each course outcome to extract the most information from many variables. The results indicate variations in the proficiency levels among students in the course outcomes. Most of the course outcomes showed positive correlations, indicating that the higher performance in one outcome was consistently connected with a higher performance in other outcomes. In addition, the regression model developed predicts the grades with a reasonable degree of accuracy. The findings of this study will provide invaluable insights for educators and administrators seeking to better understand student performance and optimize educational outcomes for their students.

Keywords: Course outcome; Outcome assessment; Factor analysis, Regression model

Course outcomes are statements that describe what students are expected to accomplish and demonstrate at the end of a course because of acquiring the specific knowledge, skills, and abilities the course will provide. The breadth and depth of the content are developed based on the quality of the outcome defined. Providing students with clear and measurable course outcomes gives them direction and clarity and helps them understand the purpose and value of pursuing the course. Clear expectations facilitate students’ alignment with their efforts and help them understand how the course content will benefit their academic and career goals. To achieve the course objectives, students are required to apply their knowledge and skills in practical contexts that enhance their ability to analyze, evaluate, solve problems, communicate effectively, collaborate effectively, and demonstrate critical thinking. Preparing course outcomes requires an understanding of the overall program goals and the ability to achieve them within the course’s timeframe and scope. Tracking and assessing students’ progress is easier with a clearly defined course outcome.

Faculty members can also use course outcome assessments to evaluate their own teaching methods and the quality of the resources they use. It is possible for instructors to revisit their teaching methods, curriculum development, and instructional materials based on the extent to which students have achieved the desired learning outcomes. Feedback assists in identifying strengths and weaknesses, resulting in a more successful learning experience. It is also very important to see that the course outcome assessment can help the instructors identify the areas where students are struggling so that they can incorporate effective remedial steps accordingly into their curriculum design or teaching methods. Students benefit from feedback by gaining an understanding of their strengths and areas for improvement to advance their performance. The provision of clear assessment criteria and feedback facilitates self-directed learning and enables students to monitor their progress through an understanding of what they need to learn and develop. Providing appropriate assessments motivates and directs students towards reaching their goals.

The assessment of course outcomes would also assist educational institutions in meeting their educational objectives and standards. A demonstration of the quality and effectiveness of a program will enable institutions to make informed decisions concerning curriculum, teaching methods, and the allocation of resources. Furthermore, it improves course delivery by enabling instructors to provide targeted instruction, align the curriculum with the desired outcomes, provide meaningful feedback for improvement, and facilitate continuous improvement by offering meaningful feedback and remediation. A systematic way of assessing course outcomes can help instructors optimize their teaching practices and improve student learning outcomes.

In summary, assessment of course outcomes are relevant as they establish shared goals, enhance student learning, and guide instructional decisions. They provide a roadmap for student learning, specifying the knowledge, skills, and abilities students are expected to gain from a course, align teaching practices, and serve as steppingstones towards the achievement of program outcomes. By connecting course outcomes to program outcomes, educators ensure that the curriculum is designed to develop the necessary abilities, knowledge, and skills required in a particular field or discipline.

Exploratory data analysis (EDA) is being used in this research to investigate the relationship between the variables and how they affect student performance. Factor analysis and regression modelling is used to examine the relationship between the course results and grade predictions. Descriptive statistics are used to summarize the data.

The rest of the paper is organized as follows: Section 2 presents the literature work followed by the methodology in Section 3. Section 4 highlights the results and discussion. Conclusion and future are presented in Section 5 followed by References.

Raquel M. Crespo Garca et al. (2023) investigated outcomebased learning, in which learning outcomes (knowledge, skills, and competencies) attained by learners are at the center of the learning process. They offered an overview of the current state of outcome-based learning in Europe and proposed a unified conceptual model for outcome-based assessment, forming a theoretical framework for the integration of key concepts such as learning outcomes, assessment, and units of learning. Finally, an application case is provided to demonstrate how the model can be used. Mithaq et al. (2022) describe a neuro-fuzzy system for forecasting student achievement. Predicting student achievement is critical for educational organizations. It aids in the revision of plans and the enhancement of students’ achievements throughout their educational experience, and the findings reveal that an excellent accuracy of up to 99% was obtained, as well as the enhancement of college admission processes and future planning in educational institutions. Kittipong Theephoowiang and Ekawat Chaowicharat plan to develop an automatic system in 2022 that can estimate the difficulty level of mathematics problems in a manner like human judgement, reducing teacher workload in question bank construction and assisting students who want to practice problems with varying difficulty levels for self-learning. However, their solution began by directly extracting information from the mathematics problem and then utilizing machine-learning techniques to assess the difficulty level so that the target value is consistent with human experts’ estimates.

In 2021, Jelena Stojanovi et al. investigated the analysis of learners’ mathematical knowledge by an adaptive neural fuzzy inference system (ANFIS) following the adoption of a distance learning application. As many faculties and other institutions are utilizing e-learning, it can be argued that the Modular objectoriented dynamic learning environment (Moodle) learning management system (LMS) is most utilized. The findings indicate that past knowledge has the greatest influence on students’ performance. Prior knowledge is more effective when integrated with educational software in primary school mathematics lectures. Prior knowledge is more effective in secondary school when combined with the motivation to learn mathematics. In 2021, Abdul Aziz and M.N.A. Hashem used the fuzzy logic system to investigate the course learning outcome (CLO) and program learning outcome (PLO)-based student performance evaluation technique. As evaluation parameters, they examined semester final examinations (SFE) and continuous assessment (CA), consisting of class tests (CT), spot tests, home tasks, attendance, and so on. The course teachers and moderators create question papers and issue grades based on CLOs, and the course teachers keep track of the grades earned. The ratios to earned and assigned marks considered by the CLOs in the SFE and CA are then computed and fuzzified.

C.M. Vivek and P. Ramkumar (2020) conducted a methodical evaluation of the course outcomes (CO) of students learning utilizing various methods. According to the findings, the use of digital tools as instructional aids has a beneficial impact on CO achievement. However, other students found it difficult to adjust to various approaches, which resulted in a fall in CO attainment percentage levels. In 2020, Somsubhra Gupta and Pushan Kumar Dutta have presented work to identify the significance of each topic included in the curriculum and anticipate its association with previous topics and, finally, with expected learning outcomes. So, if Program Educational Objective (PEO) and Program Outcome (PO) are broader efforts, Course Objective and Course Outcome are intended to identify relevance during the knowledge transfer process under the framework of Outcome-Based Education (OBE).

Ngoc Le Chau et al. 2019 investigated the Taguchi technique (TM), Adaptive neuro-fuzzy inference system (ANFIS), and Teaching learning-based optimization (TLBO) as an effective integration for CNC turning optimization of S45C carbon steel. They used the analysis of variance to establish the importance of each factor’s contribution and, they determined the appropriate ANFIS structure to optimize the root mean square error. In comparison to those anticipated by other methods, the results demonstrate a relative decrease in the roughness of surfaces. Consequently, the recommended optimization technique is a trustworthy and practical instrument for engineering applications. The findings of Yongtao et al., (2017) study on the adaptive neuro-fuzzy inference system (ANFIS) technique for country-level sustainability assessment shows that the ANFIS method is useful for assessing a country’s sustainability performance when it is applied with appropriate training data. ANFIS can be improved by selecting training samples carefully from other data sources such as the World Bank, UNHabitat, or new datasets.

The several studies that have been presented demonstrate how much attention researchers have paid to course outcome assessments. These studies emphasis how crucial it is to analyse learning outcomes, investigate how effective courses are, and suggest different approaches like Kirkpatrick model, neuro-fuzzy systems, and linkage matrices in order to enhance the student performance and satisfaction.

Dataset

The course outcome evaluation is performed on the Calculus course, which is taught at the Diploma level. The dataset utilized in this study included 50 undergraduate students. This size was selected to balance the necessity for a detailed analysis of the data with the practical constraints of the research. We observed an average variation of 0.2357 for the twelve outcomes that we are analyzing based on our data. We were thus able to calculate the margin of error we could permit and maintain a 95% confidence level in our outcome. As a result, we decided on a margin of error of roughly 0.0654, and this decision produced a sample size of roughly 2 50 1.96 0.2357 49.89 0.0654. A range of evaluations, including quizzes, midterms, assignments, and final exams, are given to students to assess their knowledge of the content. Using the results of these assessment activities, it is possible to determine the degree to which the course outcomes have been achieved.

Method

The study methodology applied in this study consists of an exploratory data analysis (EDA) approach, which is intended to look at the intricate relationships existing among various variables as well as their total impact upon performance. This study examines the fundamental characteristics of the dataset through rigorous data preparation, descriptive statistics, and advanced visualization techniques. A factor analysis is performed to identify the latent variables that describe each course outcome, since there are many variables and a limited number of observations. In Factor Analysis, information about several variables can be reduced to a smaller number of variables by combining the information from a larger number of variables. This model assumes that there are several factors in a dataset and that each of the measured variables represents some part of at least one of those factors in the dataset. Mathematically, the factors can be obtained as follows: Here, we are trying to find three latent variables, which are obtained as follows:

Typically, factor analysis begins with the preparation of a correlation matrix between the observed variables and the factors. Matrix decomposition can be applied, and factors can be extracted by replacing the diagonal entries of the correlation matrix. The diagonal entries of the correlation matrix are the variances of the individual variables that make up the correlation matrix and are represented by the diagonal entries. In factor analysis, variables are decomposed into common and unique factors based on the number of factors that are common to each variable. The unique factors in this case represent the variation that does not have a direct connection with the common factors and cannot be explained by them.

Accordingly, to consider the unique factors, we modify the correlation matrix so that the diagonal entries are replaced with 1 minus the variance of that variable’s unique factor to take all the unique factors into account. It is assumed that there is a 1:1 correlation between a variable and its own value; because of this modification, the diagonal entries of the modified correlation matrix are not equal to 1. As a result of subtracting the variance of a specific factor, we can effectively account for the unique variation within each variable. As part of the matrix decomposition process, such as principal component analysis (PCA), common factors are extracted, and factor loadings are estimated using the modified correlation matrix. Using the matrix decomposition, we can identify and estimate the relationships between variables and the factors that explain the correlations. After the latent factors are identified, multiple regression models are used to develop a prediction model.

Table 1 offers an overview of the data acquired for this research, arranged, and presented in a descriptive manner. By reviewing the summary statistics, one can see any trends, patterns, or areas that need development in the learning outcomes, in addition to any areas where the courses are strong or could use some improvement. The various measures, like variance, kurtosis, skewness range, etc. give insights into the pattern of course outcomes scores. The summary statistics offer important insights into students’ accomplishments in addition to being a useful tool for comprehending their performance. With an average mean score of 0.6740, the mean scores also show that the students were able to obtain the highest scores in CO6, indicating that they were able to reach the greatest score in that outcome. This shows that students demonstrated a strong understanding and application of CO6-related knowledge.

Table 1: Descriptive Statistics Summary of marks of the students.

The standard deviation is a helpful tool for examining the dispersion within a certain course outcome. CO12 has a standard deviation of 0.3748, which sets it apart from the other findings. This suggests that compared to the scores for the other outcomes, the CO12 scores had a wider distribution. Assessing the lowest and highest value of each result will help us to better understand the range of scores that students have received. CO12 varied widely, with some performing remarkably well and others not meeting expectations. CO12, for instance, varies the most, spanning from a minimum value of 0 to a maximum score of 1. This suggests that students’ performance in CO12 varied widely, with some achieving very high standards and others finding it difficult to meet passing grades.

The correlation heatmap between the course outcomes is shown in Figure 1. The correlation matrix of the course reveals several interesting findings. A significant number of outcomes demonstrate a positive correlation with one another, implying that higher achievement in one area is typically correlated with better performance in other areas. The correlation coefficients above 0.5 for many outcomes strongly supports this observation. For instance, there appears to be a persistent relationship between CO1 and CO2, as indicated by their strong positive correlation of 0.725. Likewise, there is a significant correlation of 0.710 between The correlation heatmap between the course outcomes is shown in Figure 1. The correlation matrix of the course reveals several interesting findings. A significant number of outcomes demonstrate a positive correlation with one another, implying that higher achievement in one area is typically correlated with better performance in other areas. The correlation coefficients above 0.5 for many outcomes strongly supports this observation. For instance, there appears to be a persistent relationship between CO1 and CO2, as indicated by their strong positive correlation of 0.725. Likewise, there is a significant correlation of 0.710 between

The 5-number summary of every course outcome is displayed in Figure 2. The value of each course outcome varies from 0 to 1. The course outcomes, CO1, CO2, CO6, and CO10, have a median value above 0.65 which indicates that 50% of the students achieved a score of 0.65, or higher. It suggests that a significant portion of the students performed well and achieved scores close to the maximum possible value. The median scores for CO3, CO4, CO11, and CO12 are below 40%, indicating that approximately half of the students received scores that were close to the minimum value.

Table 2 presents the three latent variables that are derived from the given 12 course outcomes. Upon analyzing the observations from Table 2, we find that certain course outcomes exhibit higher factor loadings on Factor 1, indicating a strong relationship with this factor. The course outcomes CO1, CO2, CO3, CO4, CO5, and CO6 have higher loadings on Factor1 and to a lesser extent with CO11. They are particularly influenced by Factor 1 and can be considered more relevant to Factor 1. Similarly, there are course outcomes with higher loadings on Factor 2, such as CO7, CO8, and CO10. These course outcomes are more strongly associated with Factor 2 and can be considered more relevant to Factor 2. Lastly, there are course outcomes with higher loadings on Factor 3, such as CO9 and, to a lesser extent, CO12. These course outcomes demonstrate a stronger connection with Factor 3 and can be considered more relevant to Factor 3. Since Factor 1, Factor 2, and Factor 3 accommodate all the course outcomes, they can be used to predict the grade of the mark using regression.

Table 2: Factor Analysis.

The predictive model was built using various regression models, and the one that yielded the lowest Root Mean Square Error (RMSE) was selected as the final model. The dataset was divided into a training set (70% of the data) and a testing set (30% of the data) using a holdout method. The various regression models were trained using the training set, where the predictor variable and target variable were paired. After training the model, it was evaluated using the testing set. Figure 3 shows the error mean, RMSE, and error standard deviation of the test data. In this case, RMSE is a commonly used metric to assess the accuracy of regression models. It measures the average deviation between the predicted values and the actual values. In this case, the RMSE value of 0.06 indicates that, on average, the predicted values from the model deviate from the actual values by approximately 0.06 units. The mean error is calculated as the average of the errors, and in this case, it is -0.0166. A mean error close to zero suggests that, on average, the model predictions are close to the actual values. The standard deviation of the errors is 0.06459, which provides a measure of the spread or dispersion of the errors around the mean error. A smaller standard deviation indicates the errors are close to the mean error, suggesting consistent model performance.

Considering the context of the problem, where the maximum grade value is 10 and the minimum grade value is 1, an RMSE of 0.06 can be considered acceptable. The magnitude of the RMSE is negligible compared to the range of grades, indicating that the model’s predictions are reasonably accurate. Overall, the combination of a mean error close to zero, a small standard deviation, and an RMSE of 0.06 suggests that the predictive model performs well in predicting the grade variable.

Course outcome assessment facilitates instructors’ ability to provide targeted instruction, align curriculum with desired outcomes, provide meaningful feedback, and facilitate continuous improvement. Fifty undergraduate students took part in the study, completing a variety of tasks such as quizzes, self-study, midterms, and final exams. Descriptive statistics, including mean, standard deviation, minimum, maximum, and percentiles, were used to analyze the students’ performance in the different course outcomes. To further comprehend the interdependencies between the course outcomes, a correlation matrix was also created. The findings suggest that students had differing levels of proficiency in the course outcomes. The course outcomes with the greatest mean score, CO6, indicated a relatively strong performance. On the other hand, the course outcomes with the lowest mean score, CO11 and CO12, indicated the need for further progress in these outcomes.

Descriptive statistics and correlation analysis included in the study make it clear that the student struggled with some course outcomes while excelling in others. Higher performance in one result was consistently correlated with higher performance in other outcomes, as seen by most of the course outcomes showing positive correlations. It appears that students who excelled in certain areas were likely to perform well in the related examinations as well. Conversely, weaker correlations between specific outcomes highlight potential areas for further investigation and instructional adjustments. As a result of these findings, educators and researchers can gain valuable insights into students’ achievements, determine patterns, and identify potential areas for improvement.

In addition, latent variables describing each course outcome were identified using factor analysis. Factor 1, Factor 2, and Factor 3 were identified through the factor analysis conducted in this study as underlying dimensions that explained the patterns of correlation among the course outcomes. Factor 1 was strongly associated with a number of course outcomes, indicating that the factor is relevant to these outcomes. There was a similar relationship between Factor 2 and Factor 3 and specific outcomes, indicating that they have distinct dimensions. The factor analysis provides a deeper understanding of the relationship between the course outcomes and the various variables in the data, in addition to reducing the dimensionality of the data. A regression model based on these factors has also been developed that can predict grades with a reasonable degree of accuracy, with the final model showing a reasonable degree of accuracy. Educators and researchers can combine factor analysis and regression modeling to understand the factors driving student performance and improve instructional practices by using this information.

The work presented in the study is limited to a sample size of 50 participants. It is imperative that we consider extending the participant pool of future studies to gain a deeper understanding of student achievement on a more comprehensive level, by including a larger number of students. Moreover, studies with the goal of exploring various instructional strategies along with their impact on student learning should also be a focus of future research. This could involve comparing different teaching methods, incorporating technology-enhanced learning approaches, or implementing innovative pedagogical approaches to identify the most effective strategies for enhancing the learning experience and improving outcomes for undergraduate level students [1-6].

None.

The authors have no financial, personal, or professional conflicts that could have influenced the research or its outcomes.

1. AM Kadim, WR Saleh (2017) Morphological and Optical Properties of CdS Quantum Dots Synthesized with Different pH values. Iraqi Journal of Science 58(3A): 1207-1213.
2. Q Zhang, H Li (2007) MOEA/D: A Mult objective Evolutionary Algorithm based on Decomposition. IEEE Transactions on Evolutionary Computation 11(6): 712-731.
3. S Romano, J Baily, V Nguyen, K Verspoor (2014) Standardized Mutual Information for Clustering Comparisons: One Step Further in Adjustment for Chance. in Proceedings of the 31st International Conference on Machine Learning, Beijing PMLR 32(2): 1143-1151.
4. J Brownlee (2012) Clever Algorithms: Nature-Inspired Programming Recipes. Autralia: LuLu Enterprice pp. 436.
5. HOA (Procedures) (1986) Code of Practice for the Housing and Care of Animals Used in Scientific Procedures.
6. CoPE (COPE) (2011) Code of Conduct and Best-Practice Guidelines for Journal Editors.