Psychometric properties and factor structure of the Early Development Instrument in a sample of Jordanian children

Abstract

Background

Investing in children’s early years can have a lasting positive effect, such as better academic outcomes throughout their school careers. In Jordan, investments have been made in early childhood development and early childhood care and education to improve children’s school readiness. School readiness comprises a range of abilities needed to succeed in school, including physical, emotional, social, and cognitive skills. To measure the impact of these investments on children’s school readiness, Jordan has been implementing the Early Development Instrument (EDI), a population-level, teacher-completed checklist of children’s school readiness, assessing children’s development in five main areas, referred to as domains.

Methods

The goal of the current study was to examine the psychometric properties of the Arabic version of the EDI, using data collected in 2018 on a sample of 5965 children in Jordan. The EDI was translated from the original English version to Arabic and adapted for use in Jordan. We conducted a categorical confirmatory factor analysis (CFA) for each of the five domains of the EDI and examined the reliability of the domains and subdomains using Cronbach’s alpha reliability coefficient.

Results

With few exceptions, the study results are in line with those of the analysis of the psychometric properties found with the original, Canadian English version of the EDI in a population of Canadian children. Results of CFAs demonstrated, for the most part, good model fits. Internal consistency indices of the domains ranged from 0.74 for physical health and well-being to 0.96 for social competence. For the subdomains, they ranged from 0.42 to 0.94.

Conclusions

Our results provide empirical support for the adaptation of the EDI for population monitoring of school readiness in Jordan. Validation of the Arabic adaptation opens up the possibility of assessing school readiness of young children in Jordan in comparison to the many other countries that have successfully adapted and applied the EDI.

Background

Investing in education from an early age has the potential to have a long-term positive impact on children. The early years of life pave the way for children’s future development and success, both in school and in life [1]. Early childhood experiences affect the growth and development of the brain [2], creating neural connections that provide the basis for a range of skills, both cognitive and non-cognitive. It is therefore important to offer children a rich, stimulating environment to help them reach their full potential and be ready for school. School readiness is a holistic term that represents a child’s transition from their preschool to their school years and encompasses a range of abilities needed to succeed in school, such as physical, emotional, social, and cognitive skills [3, 4].

In Jordan, national efforts have been made to improve children’s school readiness and subsequent educational outcomes, including investments in early childhood development (ECD) and early childhood care and education (ECCE). In 2003, Jordan launched a program called the Education Reform for Knowledge Economy (ERfKE) which was developed to help students gain the necessary knowledge and skills to be competitive once they enter the workforce [5]. This program included improving access to, and quality of, kindergarten classes [6], thus contributing to making this sector one of the most vital sectors in the educational system. In 2017, the Ministry of Education developed its strategic plan for the years 2018–2022, including a separate early education and early childhood development domain which related to the provision of quality programs for early education and childhood development in the second stage of kindergarten. This domain comprised two components: access and expansion, and quality, which included setting quality assurance criteria, and raising the proportion of kindergarten children who are ready to learn based on the Early Development Instrument (EDI) from 76% in 2017 to 80% in 2022 [7].

Jordan’s Modernization Vision for the year 2022 postulated that education, especially early childhood education, is one of the engines of growth. The vision identified a set of initiatives to advance the sector so that by 2033 all children in Jordan would have an integrated, equitable system centered on the child to develop health care and education in early childhood [8]. Furthermore, the National Human Resources Development Strategy was developed for the years 2016–2025 to ensure that all children would have access to quality early childhood learning and development experiences that promote primary school readiness, ensure healthy lives, and encourage their future well-being. In addition to that, the strategy aspired to ensure that children complete equitable and quality primary and secondary education, leading to relevant and effective learning outcomes. Moreover, it sought to increase the number of youth and adults who have relevant technical and vocational skills for employment, decent jobs, and entrepreneurship. In addition, the strategy works to guarantee fair access to affordable, relevant, and quality university education opportunities [9].

The Queen Rania Foundation conducted a study on the economic effects of investing in ECCE in Jordan. The results of the study suggested that ECCE services for three years for Jordanian children would achieve benefits totaling $23,881 per child when the child finishes school and enters the labor market. The study showed that from a social point of view, the benefits of providing these services outweighed the costs, with a ratio of 1:9, meaning that every dollar invested in providing ECCE services would produce returns of $9 [10].

As with many regions in the Middle East and North Africa, about half of schools in Jordan are privately-run and therefore not accessible to a large segment of the population [11]. The ECCE policies developed as part of the ERfKE included establishing state nurseries and kindergartens. The focus on the first five years of life was dictated by existing and growing knowledge of how crucial the first five years of a child's life are for optimal health and development [12].

In order to examine the impact of these policies on children’s school readiness, Jordan has turned to the EDI [13]. The EDI is a population-level, teacher-completed checklist covering five major developmental domains: physical health and well-being, social competence, emotional maturity, language and cognitive development, and communication skills and general knowledge. The EDI is a community-based instrument and data are typically aggregated to either the school, neighborhood, regional, or country level to provide a glimpse of how populations of children are doing. One advantage of the EDI is that it combines several domains of child development into one holistic measure which are all based on easily observable skills and behaviours [13]. The psychometric properties of the EDI have been examined in several countries [13,14,15,16,17,18,19,20,21,22], as well as with sub-populations of children [23,24,25]. The psychometric properties of the Arabic version of the EDI, implemented in Jordan, have yet to be examined.

Current study

Measurement of children’s developmental status at school entry over time provides an opportunity to monitor investments in ECD and ECCE and examine their association with children’s concurrent and future outcomes. Thus, it is vital to establish a measurement that is psychometrically sound, reliable, and equitable. The goal of the current study was therefore to examine the psychometric characteristics of the Arabic version of the EDI using the data collected in 2018 on a sample of children in Jordan. Accordingly, this study sought to answer the following two research questions:

• 1. What are the psychometric properties (construct validity and internal consistency) of the five developmental domains proposed by the developers in the Arabic version of the EDI?

• 2. Do the developmental domains of the Arabic version of the EDI have similar psychometric properties compared to the original English instrument?

Methods

Study design

A cross-sectional study examining the validity and reliability of the Arabic version of the EDI in a sample of children attending first grade in Jordan in 2018 was conducted. The Ministry of Education and other educational authorities were approached to obtain the schools' approval to participate, and the study procedures included a pledge to the schools that the information to be collected is for scientific research purposes. SMS have been sent to parents about the study, not to obtain their consent, but to inform them of the study's purposes only.

Study sample

The study sample came from a population of children enrolled in first grade during the 2017/2018 school year. The total number of children enrolled in first grade, based on data from the Educational Management Information System in the Ministry of Education, was 191,688, of which 98,570 (51.4%) were male. The study sample was selected in two stages. In the first stage, schools were selected to represent the location (i.e. rural or urban), the sex of children taught in the schools (males, females, mixed), the education directorate (Jordanian schools are affiliated with one of 43 education directorates, covering the different regions of Jordan), and the geographical region (North, Middle, and South). As these characteristics were considered strata, the size of each stratum was determined according to its relative weight in the schools sampling frame. In the second stage, 24 children were selected from each of the chosen schools. In the event a school had less than 24 children, all children were selected to take part in the study. If, on the other hand, the number of children in a school was greater than 24, children were selected using a systematic random sample method. If the number of classes in a school was more than one, the children were distributed equally among the classes so that the required number was selected based on a systematic random sample. The final representative study sample comprised 6016 children from 260 schools. Sixty-four children were excluded because they had data missing on more than one domain of the EDI. The final analytic sample therefore comprised - 5965 children with valid EDI data (99% of the original sample).

Measures

Early Development Instrument (EDI)

The EDI [13] was developed in Canada to provide population-level data on how children are doing in the year prior to their first year of formal schooling. The EDI is a 103-item, teacher-completed checklist used to assess children's school readiness in five general areas of development: physical health and well-being, social competence, emotional maturity, language and cognitive development, and communication skills and general knowledge. These five domains are further broken down into 16 subdomains. Table 1 shows the domains and sub-domains of the EDI, as well as number of items in each domain. In addition to the 103 core items, the EDI contains questions about children's demographic characteristics, their preschool experiences, the skills they possess, and the special problems they have (if any). It should be noted that these questions are not included in the computation of a child’s score on the five domains of the EDI. The EDI is completed by the teacher (or specialist in early childhood) in the second half of the school year, as this allows the respondent enough time to get to know the children in their class well and also allows children sufficient time to adapt to their new environment [13].

Table 1 The five developmental domains of the EDI, the number of items in each domain, and the subdomains comprising each domain

Domain scores are an average of the items in each domain and range from 0 to 10, with a higher score denoting greater ability. The mean scores are then divided into categories representing the highest and lowest scores in a given population. The distribution of scores is used to determine the percentage of children who are at different levels of school readiness. Children who score below the 10th percentile in a domain, based on a baseline or comparison population, are considered vulnerable in that area. The outcome measures used in this study were the scores on the five EDI domains.

The EDI has been well-validated as an assessment of child development (see [26] for a review). Several researchers have assessed and established the construct validity [13, 17, 18], predictive validity [3, 27, 28], between-group validity [23, 29], as well as cross-cultural validity [15, 16, 19, 20] of the EDI. The EDI is routinely collected at the population-level in Canada and Australia, and has been implemented in many countries, at various levels, including Brazil, Peru, China, Italy, Germany, Kyrgyzstan, Mexico, Jamaica, Indonesia, Vietnam, the United States, and the United Kingdom [26].

Demographics and contextual characteristics

In addition to the ratings of children’s development, other data on children, families and location were collected. Child’s sex at birth, age, maternal and paternal education were recorded on the EDI by teachers using the child’s school-based information. Rurality and region were provided by the Educational Management Information System in the Ministry of Education. These variables are defined and described as follows:

• Child age The child’s age at the time of EDI completion was calculated in years. Child’s date of birth was obtained from the child’s profile at the school and documented by the child’s teacher on the EDI. Age was dichotomized as less than or equal to 6.65 years, and greater than 6.65 years.

• Child sex The child’s sex was listed as either male or female and was recorded by the child’s teacher on the first page of the EDI.

• School location (urban/rural) Areas in Jordan are divided into urban areas (localities with a population of 5,000 people or more) and rural areas (localities with smaller populations). This information was provided by the Educational Management Information System in the Ministry of Education.

• Geographical area In the year 2000, Jordan was divided administratively into three regions: the North region, the Central region, and the South region. This information was provided by the Educational Management Information System in the Ministry of Education.

• Mother’s and father’s education This variable represented the mother’s and father’s educational attainment levels, using information found in the child’s school profile. The child’s teacher indicated this information on the first page of the EDI. Mother’s and father’s education were classified into of six levels: illiterate, less than general secondary school, general secondary school, community college diploma, bachelor’s degree, and master’s degree or higher. These variables were then dichotomized as follows: lower education represented educational attainment up to and including general secondary school, and higher education, comprising community college diploma, bachelor’s degree, and master’s degree or higher.

Jordan’s adaptation of the EDI

In order to collect school readiness information in Jordan using the EDI, the instrument went through a rigorous translation, adaptation, and verification process. The EDI was translated to Arabic by an early childhood expert, then it was assessed by university professors, researchers, and administrators in the childhood sector from the Ministry of Education, kindergarten supervisors working in the Ministry of Education and the private sector, and teachers in the field of early childhood. They judged the items in terms of the quality of the language and appropriateness for the purposes of the study. The views of these reviewers were taken in consideration, and a revised Arabic version of the EDI was produced and back-translated to English to verify that the significance of the items was not lost during the translation process. The translation and feedback of the experts were then sent to the developers of the EDI in Canada for approval. The finalized Arabic version of the EDI was piloted on 1341 children. The internal consistency of the five EDI domains were estimated with Cronbach’s alpha, which ranged from 0.66 for the physical health and well-being domain to 0.93 for the social competence domain. This sample was also used to conduct an exploratory factor analysis (EFA) to assess the construct validity of the EDI, and the results revealed the same five domains as the original EDI factorial structure [13]. However, it should be noted that there was a significant difference on how items loaded on the factors [30]. A decision was made at the time to preserve the original factorial structure of the tool to assess the level of children's readiness for school in Jordan, where the distribution of the EDI's items on the domains remained the same as the original version. The EDI has been implemented three times in representative samples of children in Jordan so far: in 2010 as a baseline study, then in 2014 and 2018 as a follow-up study. Preliminary psychometric analyses were conducted on the 2010 sample (unpublished), however, the 2018 sample was chosen for this study as it comprised a much larger number of children and also better reflected the state of early childhood in Jordan, given all the changes that have occurred in the last decade.

Statistical analysis plan

The analysis for this study was conducted in three parts. First, we ran descriptive statistics on the background and demographic characteristics of the sample of children. Using the statistical software IBM SPSS, version 19 [31], we ran frequencies for categorical variables. Second, means and robust standard errors for domain scores were examined, overall and by sample demographic and contextual subgroups, accounting for clustering within schools, using the statistical software Stata, version 13.1 [32]. Third, we conducted a categorical confirmatory factor analysis (CFA) for each of the four multidimensional domains and the one unidimensional domain of communication skills and general knowledge using Mplus 7.4 [33] to test the construct validity of the Arabic version of the EDI. Since the sampling methods employed meant that children were clustered within schools, we took clustering into account in our analysis. To evaluate model fit, we used multiple indices: the Comparative Fit Index (CFI), the Tucker–Lewis Index (TLI), the root mean square error of approximation (RMSEA), and the Weighted Root Mean Square Residual (WRMR). Another index of model fit computed was the chi-square (χ²) statistic, which allows us to assess the adjustment between the model and the observed covariance matrix, with a lower value indicating a better adjustment, and with the χ² fit statistic ideally being non-significant [34]. Nevertheless, because the χ² statistic is sensitive to sample size, we did not take this index into consideration in our analyses. We therefore based our model fit decisions on the CFI, TLI, RMSEA and WRMR indices. A value of 0.90 or greater is generally considered acceptable for the CFI and TLI, the RMSEA should be 0.06 or lower [35], and a recommended cut-off value of 1.0 for the WRMR is considered to indicate good fit [36]. We therefore based our model fit conclusions on these values. Last, we examined the reliability (internal consistency) of each of the domains and subdomains using Cronbach’s alpha reliability coefficient in IBM SPSS, version 19 [31].

Results

Sample characteristics

The study sample comprised 6016 children from across the country, of which 5965 (99.2%) had a valid EDI and were included in the analyses. Table 2 presents demographic and contextual characteristics of children included in the final analytic sample. Just over half the sample was male (52.1%) and about two thirds were enrolled in kindergarten (66.4%) prior to first grade. According to the data shown in Table 2, 47.6% of the children lived in rural areas. The table also shows that 39.4% of the children were from the Middle region of Jordan, while the percentages of children from the Northern and Southern regions were 30.4% and 21.3%, respectively.

Table 2 Demographic and contextual characteristics of the final analytic sample

Mean scores by sample characteristics

Next, we examined the scores on the EDI based on our sample characteristics, which are displayed in Table 3. Effect sizes and p-values corresponding to the differences in mean scores between the subgroups of the demographic and contextual characteristics are presented in Additional file 1: Appendix A. EDI domain scores differed according to some demographic and contextual variables. For instance, females had higher scores than males on all EDI domains and the biggest mean differences were observed for the social competence and emotional maturity domains. Similarly, age was associated with scores on all of the domains and children older than the mean age of 6.65 years had higher scores on all EDI domains. Domain scores also differed based on both mother's and father’s education, where the children whose parents had higher levels of education had higher scores. Here the biggest mean differences were observed in three domains—social competence, language and cognitive development, and communication skills and general knowledge. Last, when examining scores on the EDI based on the location of the school, we observed higher domain scores for children attending schools in urban areas compared to those attending schools in rural regions.

Table 3 Means and robust standard errors of the developmental domains of the EDI by demographic and contextual characteristics, accounting for clustering within schools

Confirmatory factor analysis

We ran a categorical CFA to examine the fit of each of the five domains of the EDI. The results are summarized in Table 4 below (see Appendices B and C for results of the categorical CFAs using the 2010 and 2014 data, and Additional file 1: Appendix D for detailed results showing thresholds and factor loadings). As can be seen in Table 4, the CFI and TLI for all five domains were higher than 0.92, indicating a good model fit. These indices were lowest for the emotional maturity domain, however. On the other hand, the RMSEA was higher than the criterion value 0.06 for two of the domains, that is, social competence and communication skills and general knowledge. The WRMR was greater than 1 for all domains except physical health and well-being, indicating the fit was not as good for that domain.

Table 4 Results of the confirmatory factor analysis: goodness of fit statistics for the five EDI domains

Internal consistency reliability

The results indicated that, in this sample, the internal consistency reliability coefficient, estimated by Cronbach’s alpha, for the five domains varied from 0.74 for physical health and well-being to 0.96 for the social competence domain. The subdomain reliabilities varied from 0.42 for physical independence to 0.94 for prosocial and helping behavior. Table 5 shows the internal consistencies for the various domains and subdomains.

Table 5 Cronbach’s alpha reliabilities for the five domains and 16 subdomains of the EDI

Discussion

The current study examined the psychometric properties of the Arabic version of the EDI in a sample of 5952 children attending first grade in schools across Jordan. We found that EDI domain scores differed by children’s sex, socioeconomic characteristics, such as parental education and household income, and the location of the school. We found a consistent pattern between domains scores and age, similar to the findings with the original version of the EDI. Results of CFAs demonstrated, for the most part, good model fits. Internal consistency indices of the domains ranged from 0.74 for physical health and well-being to 0.96 for social competence. For the subdomains, they ranged from 0.42 to 0.94.

Results of the CFA observed in our study are of similar magnitude to those of previous studies conducted elsewhere [20, 37]. For instance, the psychometric properties of the EDI were examined in four countries (Canada, Australia, United States, and Jamaica) and the authors noted similar patterns of goodness of fit indices across the countries, with items tending to load on the same factor. Furthermore, in a sample of children in the Philippines and Indonesia and using a shorter version of the EDI, Duku and colleagues [37] found adequate internal consistencies and parsimonious measurement models for all the domains, except physical health and well-being. The authors noted the smaller number of items and the lack of variability in responses as possible reasons for the lower Cronbach’s alpha for the physical health and well-being domain. In contrast to these two studies, the physical health and well-being domain had the best model fit of all the domains in the current study. Some potential explanations for these discrepancies include the fact the original structure of the EDI may not fully apply to the present study sample or that the structure of the instrument may have changed as a result of translation.

The internal consistency of the domains was generally very good (above 0.90), however, the physical health and well-being domain had a lower Cronbach’s alpha. This domain in the original EDI had a Cronbach’s alpha of 0.84 [13], which is much higher than the one found in this sample (0.74). It has been suggested by others that this domain may not reflect a unidimensional construct [38], even though the original structure of the EDI considered it as one. The authors argued that the subdomain of physical readiness for the school day may have less to do with the children and more to do with the behaviors of the children’s parents. Another possibility is that the subdomain of physical independence might be responsible, at least in part, for the lower reliability observed as the Cronbach’s alpha for this subdomain was 0.42. Previous researchers have questioned the inclusion of the item of sucks thumb/finger, found within this subdomain, as a low factor loading and a weak correlation with the total score for the domain have been observed [16, 38]. It is important to note that the developers of the EDI mentioned that seven items were kept even though they all had low factor loadings (less than 0.3) because teachers perceived these items to be important [13]. Three of these items were part of the physical health and well-being domain: independent in washroom habits, is well coordinated, and sucks a thumb/finger. More specifically, all three of these items comprise the physical independence subdomain which could explain why this subdomain had such a low Cronbach’s alpha.

Another potential explanation could be because of a smaller sample size compared to that used to examine the psychometric properties of the original EDI. Thus, a future study with a larger sample might be needed to confirm the internal consistency of the Arabic version of the EDI. One other potential reason for the lower Cronbach’s alpha in this study could be the age of the children. The mean age in this study was 6.6 years whereas it was 5.6 years in the original study of the EDI [13]. The one-year difference in mean age may also explain the differences in results observed in the current study compared to the original EDI study.

The pattern of internal consistency in Arabic EDI appears to be consistent with that seen in previous research. All studies performed to date examining the internal consistency of the EDI, in various locations across the globe, have all found that the internal consistency was lowest for the physical health and well-being domain [3, 13, 15, 19, 20, 39, 40]. It is interesting to note that for studies examining the internal consistency of the EDI in a language other than English, the statistics tended to be lower [16, 19, 37].

Strengths and limitations

This study offers the first evidence of the reliability and validity of an Arabic adaptation of the internationally renowned EDI. Despite this strength, we acknowledge several limitations of our study. First, the data come from 2018 and therefore do not reflect the more recent contexts in which children are taught and educated, especially throughout the COVID-19 pandemic. Nevertheless, data presented here might be used as a baseline if examining the impact of the pandemic on children’s development. Second, the results of the physical health and well-being domain suggest a weak reliability of this domain. As mentioned earlier, one reason behind this finding may be the translation of the instrument. In classrooms taught by one teacher, it is difficult to examine inter-rater reliability, and we were not able to do so. We were also unable to assess test–retest reliability with this sample. Despite these limitations, the study strengths, such as its novelty, the large sample size, and potential for the use as a baseline in assessment on the impact of COVID-19, are considerable.

Conclusion

This study evaluated the validity and reliability of the Arabic version of the EDI in a sample of children in Jordan. Our results provide empirical support for the adaptation of the EDI for population monitoring of school readiness in this country. With few exceptions, our study results are in line with those of the analysis of the psychometric properties found with the original, English version of the EDI. Janus et al. [20] indicated that the items in each domain were put together conceptually, therefore we did not expect to have fit indices that met all the criteria for a good model fit. By validating the EDI in various countries, including Jordan, children’s school readiness can be monitored over time. Validation of the Arabic adaptation could also open up the possibility of comparing school readiness of young children in Jordan with the many other countries who have successfully adapted and applied the EDI.