Effectiveness and Efficiency Analysis of Education Systems in EU Countries Employing Secondary Big Data

On November 29 - December 1, 2018 10th International Workshop "Data Analysis Methods for Software Systems" (DAMSS 2018) was held in Druskininkai.

Program of Workshop and Plenary Speakers

Plenary speakers of education

ana camanho small           Assoc. Prof. Ana Camanho

Associate Professor at the Faculty of Engineering of the University of Porto, Portugal (FEUP). She is the Director of the Master of Science in Industrial Engineering and Management at FEUP. She has a Ph.D. in Industrial and Business Studies from Warwick Business School, University of Warwick, United Kingdom. Her research interests include the assessment of performance and evaluation of productivity change over time, focusing primarily on the use of the Data Envelopment Analysis technique. She has been involved in research projects, both at the national and international level, in the following sectors: banking, retailing, healthcare, education, fisheries, construction industry, regulation of electricity distribution companies, water utilities, mining industry, transportation systems and urban quality of life. She was vice-president of the Portuguese Operational Research Society and Pro-director of the Ph.D. program in Industrial Engineering and Management of the School of Engineering of the University of Porto. She has published more than 60 articles in international peer-reviewed journals. She has got more than 1300 citations to date. She has been a member of the programme and organizing committees of several national and international conferences.

Talk title: Benchmarking Secondary Schools Using Students’ Results at University

Abstract: In this research, we use data on university students’ first-year scores as a means to compare and benchmark secondary schools on their ability to lead students to university success. Data from two universities in Portugal, Portuguese Catholic University (with seven faculties) and the University of Porto (with 14 faculties) were used to compute a number of metrics, based on which secondary schools were compared. The performance of more than 10.000 students from 65 different degrees was explored for a three year period.

There are mainly two research questions addressed in this paper: (1) what are the determinants of success for first-year university students? and (2) what is the role of the secondary school of origin in determining success for first-year university students? The regression model used to test these hypothesis assumes that university success in the first year is a function of student characteristics (e.g., scores on entry to higher education), secondary school characteristics (e.g., private vs. public) and university/course characteristics (e.g., public vs private university and the main scientific area of the course). University success is measured through two measures: number of ECTS finished by first-year university students and the final grade at the end of the first year (this final grade is normalised per degree attended and cohort to allow comparability between different degrees and years). Conclusions from the preliminary analysis point to the importance of scores on entry (based on standardized national exams), but this importance is relatively small in explaining the success of students at the university (it explains to a certain degree the average scores obtained at the end of the first year but not the number of ECTS completed). The type of school appears as the second most important factor in determining student’s success, with public schools performing better than private.

A DEA model was constructed to compare secondary schools on the extent to which they prepare students to university success. For this purpose, a set of outputs relating to success in the first year of higher education was defined (e.g., percentage of students at the top percentile of the degree attended, average number of ECTS completed by the students, and average score of students at the end of the first year, normalised according to the degree attended). The input of the DEA model is the score on the entry of the students, normalised according to the degree attended by each student. This model attempts to evaluate secondary schools by the value added accomplished by their students when attending higher education. This model is unprecedented in the literature, as it evaluates secondary schools based on the performance of students in higher education, which is undisputedly a key objective of secondary school education.

kristof de witte           Asoc. Prof. Kristof De Witte

Kristof De Witte is a tenured associate professor at the Faculty of Economics and Business at KU Leuven, Belgium, and he holds the chair in ‘Effectiveness and Efficiency of Educational Innovations’ at Top Institute for Evidence Based Education Research (TIER) at Maastricht University, the Netherlands. At KU Leuven he is director of the research group ‘Leuven Economics of Education Research’. The research interests of Kristof De Witte comprise education economics, performance evaluation, and political economy. He published his work in many leading academic journals including ‘The Economic Journal’, ‘Journal of Urban Economics’, ‘European Journal of Operational Research’, ‘Economics of Education Research’, ‘European Journal of Political Economy’ and ‘Scientometrics’. Further information on

Talk title: The Laboratory of an Education Economist. Testing Cures for Disadvantaged Students

Abstract: Kristof De Witte overviews his research agenda on socio-economic segregation. In particular, he discusses various quasi-experimental evaluations of interventions to change the odds for disadvantaged students. On the one hand, he presents the effects of additional resources at school level on cognitive and non-cognitive outcomes and illustrates that that extra resources at municipality level only result in grade inflation. On the other hand, he argues that information shocks provided by making school quality information public changes the socio-economic composition of schools. He shows that in the longer run, these findings are alarming as the inability to break the vicious circle for low SES students leads to more school dropout, and that particularly those students have a lower return to education.

This annual scientific forum also hosted a section called "Education Data Modeling", in which reports were read

Trends of Secondary Analysis of International Student Achievement Studies

R. Želvys, R. Dukynaitė, A. Jakaitienė

International student achievement studies – PISA, TIMSS and PIRLS – are primarily focused on providing data on student achievement in comparative perspective. However, tje surveys also contain information about the socioeconomic status of students, school systems, management and evaluation, etc. The collected data provides good opportunities for secondary analysis. The amount of publication of that kind is increasing each year. The aim of our presentation is to review main trends of secondary analysis in related publications.

The Effect of Student-Oriented Teaching Practices in Explaining PISA Science Performance Across EU Countries

S. Raižienė, D. Stumbrienė. L. Ringienė, R. Dukynaitė, A. Jakaitienė

The acquisition of scientific principals and theories during school years increasingly gains greater value as science-related employment is expected to grow (Fayer, Lacey & Watson, 2017). However, the students' interest in science is declining (Potvin, Hasni, 2014). Therefore it is important to analyze what student-oriented teaching practices are the most effective for science disciplines in order to keep students' interest in science. Teaching practices vary across educational systems and their effectiveness depends on the composition of other factors (Kyriakides, 2008). This study investigates how differ and what is the differential effectiveness of student-oriented teaching practices (enquiry-based science teaching practices, adaption of instruction, teacher support and feedback) across EU countries. Data from PISA 2015 were used. 24 EU education systems were analyzed. Multiple linear regression analyses for each education system was performed with student's science performance as the dependent variable and four students' perceived student-oriented teaching practices (enquiry-based teaching, adaption of instruction, teacher support and feedback) as independent variables. Student economic, social and cultural status was entered as control variable. From the study we conclude that the prevalence (based on students' reports) of four analyzed student-oriented teaching practices is different across EU countries. Similar patterns of association for adaption of instruction and perceived feedback with science performance, and different patterns of association for teacher support and enquiry-based teaching with science performance are observed across EU countries. This confirms different effectiveness of student-oriented teaching practices in EU learning contexts.

Effectiveness and Efficiency of Education Systems: a Review of Research and a Way Forward

J. Vaitekaitis, D. Stumbrienė, A. Jakaitienė

The aim of our research (EFFECTAS: Effectiveness and Efficiency Analysis of Education Systems in EU Countries Employing Secondary Big Data) is to investigate the factors determining the effectiveness and afficiency of the EU education systems. The presentation will focus on the discussion about differences between the effectiveness and efficiency in education, practical examples of variables and models actually used in empirical studies. We will discuss distinct theoretical approaches between effectiveness (the levels of educational outputs achieved) and efficiency (the levels of educational inputs vs outputs) in education, and provide an overview of input and output variables, as well as different targeted education system levels and its classifications. Finally, we will present traditional frontier approaches (Data Envelopment Analysis) for analysing effectiveness and efficiency in a cross-country perspective.

  • 20181130_01d
  • 20181130_02d
  • 20181130_03d
  • 20181130_04d
  • 20181130_05d
  • 20181130_06d
  • 20181130_07d
  • 20181130_08d
  • 20181130_11d
  • 20181130_12d
  • 20181130_13d


This project has received funding from European Social Fund (project No DOTSUT-39 (09.3.3-LMT-K-712-01-0018) / LSS-250000-57) under grant agreement with the Research Council of Lithuania (LMTLT).