Development and validation of a novel self-assessment tool for breastfeeding mothers

Abstract

Background

Difficulties with breastfeeding can lead to early breastfeeding cessation. Empowering mothers through self-assessment and education of breastfeeding skills can help support them. We aimed to develop and validate a tool for self-screening and education on breastfeeding skills.

Methods

A six-item tool was developed through literature review and expert interviews, covering domains of breastfeeding position, nipple shape, breast engorgement, infant latch, swallowing and intake. Eight experts assessed the tool’s relevance, clarity, simplicity and ambiguity. Scores were used to determine item-level content validity index (I-CVI) and scale-level content validity index (S-CVI). Items with values ≤ 0.83 (best possible score = 1.00) were revised. Following revision, a convenience sample of breastfeeding mothers were recruited from maternity wards and clinics to determine the tool’s internal consistency, face, criterion and construct validity. Participants and lactation consultants (LC) independently completed the tool based on the same breastfeeding episode. Internal consistency was determined by Cronbach’s alpha. Criterion validity was assessed by comparing participant and LC scores using Bland-Altman plots. Construct validity was determined by comparing scores in participants who were and were not referred to a LC. Receiver operating characteristic (ROC) curve was used to determine a cut-off score for LC referral by optimizing sensitivity and specificity.

Results

The tool demonstrated acceptable content validity, with I-CVI and S-CVI values for relevance, clarity, simplicity and ambiguity above the threshold, except for S-CVI of simplicity and ambiguity, and the tool was revised accordingly. Most mothers who completed the tool (n = 58) found it easy to understand (87.9%), relevant (91.4%) and useful (86.2%). Cronbach’s alpha was 0.66, which improved to 0.74 with the removal of the item on “engorgement”. With the remaining five items, ROC analysis showed an area under the curve of 0.79 [(95%CI 0.67–0.90), p < 0.001], with a score of ≤ 6 indicative of an LC referral (sensitivity = 86%, specificity = 55%). Bland-Altman plots showed acceptable agreement between participant and LC scores with a mean difference of 0.22 (95%CI -3.02–3.47).

Conclusions

We developed and validated a simple five-item tool for mothers to assess and be educated about breastfeeding skills. Further study on the tool’s predictive validity and effectiveness within a clinical pathway is warranted.

Background

Breastfeeding is the preferred nutrition source for infants, conferring significant health benefits for both mother and infant [1, 2]. Yet, rates of breastfeeding are low internationally, with upper middle-income countries falling short of the World Health Organization (WHO) target of at least 50% exclusive breastfeeding in the first 6 months. In Singapore, national breastfeeding rates are high at birth (96% for any breastfeeding), but only 42% of mothers are still partially or exclusively breastfeeding at 6 months of age [3]. Exclusive breastfeeding rates show a similar decline, from 50% at birth to 28% at 2 months of age [3]. Reasons for low breastfeeding rates are complex and multifactorial, including medical, psychological and socio-cultural factors [1]. Personal barriers to breastfeeding include lack of knowledge of breastfeeding techniques, self-perceived inadequate breastmilk supply, and lactation problems [1, 4, 5]. In Singapore, frequent reasons for early breastfeeding cessation include a self-perceived low breastmilk supply [3], and struggles with latching on, leading to the introduction of formula milk [6]. A study conducted in 2016 in Singaporean mothers found that 37% of mothers at birth perceived that they were not producing enough milk, which correlated with lower rates of exclusive breastfeeding at 6 months [7].

As early establishment of breastfeeding is crucial for exclusivity and duration of breastfeeding [4, 8], it is important that breastfeeding difficulties and problems are identified promptly such that appropriate intervention can be provided to mothers. The concept of empowering mothers by teaching them to self-assess their breastfeeding skills can help identify areas for tailored lactation support [9,10,11]. To enable identification of breastfeeding problems, several tools have been developed [12,13,14], which have varying indications and context for use. For example, the LATCH tool was designed by Jensen et al. to allow healthcare professionals to systematically assess and chart five components of breastfeeding (latching, audible swallowing, nipple type, mother’s sense of comfort and breastfeeding hold) [12]. The Breastfeeding Evaluation and Education Tool (BEET) by Tobin et al. was designed to guide breastfeeding education to parents, who can then use the checklist to evaluate their own breastfeeding skills [13]. However, the main limitation of these existing tools is that they were not designed to be used as a self-assessment tool. They were either designed for use by healthcare professionals or require some existing breastfeeding knowledge to complete, while others were too lengthy to serve as screening tools. In addition, none of these tools have been adequately validated as a self-assessment tool in our local population.

We thus aimed to develop and validate a novel, brief, tool to screen for breastfeeding difficulties and educate on breastfeeding skills for mothers in Singapore. We wanted the tool to be used in the early breastfeeding journey by mothers, such that they could be empowered to recognize breastfeeding concerns, as well as by nurses, to standardize breastfeeding assessment and education.

Methods

The development and validation of this tool occurred in two phases. Phase 1 is the development, and Phase 2 is the validation of the tool (additional file 1). This study took place at KK Women’s and Children’s Hospital between November 2022 and April 2023, and was conducted in accordance with the Helsinki declaration. Ethics approval was obtained from the Singhealth Centralized Institutional Review Board (CIRB Ref: 2022/2428) with written informed consent from participants.

Phase 1– development of tool

In this first phase, a scoping literature review of existing breastfeeding questionnaires and assessment tools was conducted to identify aspects of breastfeeding commonly assessed in mothers. The search was performed in PubMed and EMBASE databases, using “breast feeding”, “screening tool” and “assessment tool” as keywords. Original research and review articles which described an assessment tool for breastfeeding in term infants were extracted.

The most commonly-studied breastfeeding assessment tools were the United Nations Children’s Fund (UNICEF) Baby-Friendly Hospital Initiative breastfeeding assessment tool [15], the BEET [13], the LATCH tool [12], the Infant Breastfeeding Assessment Tool (IBFAT) [16] and the Breastfeeding Assessment Score (BAS) [17]. Eight themes present in available breastfeeding tools included baby’s behavior, mother’s behavior, positioning, attachment, effective feeding, health of the breast, health of the baby and mother’s experience [14]. Most of the tools assessed themes of position, lactating and effective feeding, while fewer assessed baby’s behavior, mother’s behavior, breast health, baby’s health and mother’s view of the feed [18]. These factors were taken into account when developing our tool, which aimed to encompass as many aspects of breastfeeding as possible.

A list of breastfeeding skill items was first extracted and consolidated from the aforementioned breastfeeding assessment tools (UNICEF, BEET, IBFAT, LATCH and BAS). This list was refined after interviews with a three-member expert group consisting of two international board-certified lactation consultants (LC) and a neonatologist, each with more than 15 years of experience specializing in breastfeeding medicine, and who were members of the hospital’s Baby Friendly Hospital Initiative (BFHI) committee. Based on the expert group’s opinion, the most pertinent questions for a screening tool in our local population were identified and reworded as necessary. As this study aimed to create a user-friendly self-administered screening tool, potential items were chosen both for their brevity and clarity. In addition, in line with our aim of the tool providing education to mothers, we identified aspects of breastfeeding skills that were commonly covered in breastfeeding education sessions [19, 20].

After expert group discussion and interviews, six questions focusing on different themes of breastfeeding were included in our proposed tool, which we named the Breastfeeding Education and Screening Tool (BEST) (Additional file 2). The six domains are breastfeeding position, nipple shape, breast engorgement, infant attachment, infant swallowing, and intake. Each question was scored on a 3-point Likert scale (0 = poor, 1 = moderate, 2 = good), which was chosen to keep the tool simple, yet allow differentiation between the degree of problems faced in each domain. The tool instructs mothers to choose the score describing their breastfeeding experience in each of the 6 domains, giving a total best possible score of 12. Descriptions and accompanying images also serves to educate mothers on the appropriate techniques or experiences for each of the domains.

Phase 2– validation of tool

Content validity by experts

Following item development, a group of eight domain experts were asked to review the tool. Experts consisted of two LCs with between 5 and 10 years of experience, two physicians (neonatologist and pediatric endocrinologist) and four primary care nurses working in maternal and child health. Experts qualitatively assessed whether the questions appropriately measured breastfeeding competency and educated on breastfeeding skills in the initial stage of breastfeeding.

This was followed by quantitatively scoring the questions based on their relevance, clarity, simplicity and ambiguity on a 4-point Likert scale as described by Yaghmale et al., with 4 being the best possible score [21, 22]. Clarity refers to the construction and phrasing of the question, while ambiguity refers to the potential for multiple interpretations of a question. Content validity of each item was determined using the item-level content validity index (I-CVI) which measures the number of experts rating the item as relevant, clear, simple and not ambiguous, as well as the modified kappa (κ) statistic, which is the agreement between experts and adjusts the I-CVI for chance agreement [23, 24]. In addition, the scale-level content validity index (S-CVI) was used to measure overall content validity of the tool using two methods: first, the average of I-CVIs (S-CVI/Ave), and second, the proportion of questions universally agreed by all experts to be relevant, clear, simple and not ambiguous (S-CVI/UA) [23]. A threshold of ≥ 0.83 was considered acceptable for I-CVI, S-CVI and κ [23]. Items were revised if they did not reach an acceptable score, or were deemed by experts to require rephrasing.

Face, criterion and construct validity by breastfeeding mothers

Following feedback and revision by experts, the tool’s face, criterion and construct validity was assessed through convenience sampling of 60 breastfeeding mothers at KK Women’s and Children’s Hospital, Singapore between February and April 2023. At the hospital, there is currently no standardized assessment used to assess a mother’s breastfeeding skills and experience. Instead, maternity ward nurses would subjectively determine whether the mother is able to independently breastfeed during the postpartum hospital stay, and mothers who are assessed to require lactation support by their maternity ward nurse are referred to a LC. The sample size of 60 was determined based on an estimated 10 subjects required per question [25]. Inclusion criteria were mothers aged 21 years and above, with any duration and frequency of breastfeeding, with a singleton infant below one month of age. Women with preterm infants or infants with medical conditions, or who were not breastfeeding, were excluded. Participants were recruited from obstetric wards and outpatient clinics. To achieve a range of scores, 30 mothers who were referred to the LC and 30 mothers who were not referred to the LC were enrolled, based on an assumption that those requiring LC are likely to have worse scores compared to those not referred to the LC.

Participants were first given the tool to read, followed by a breastfeeding session that was observed by the LC. The participant and LC then completed the tool independently. Construct validity was assessed by comparing scores between mothers referred and not referred to the LC, to determine whether the tool might be able to identify mothers requiring LC support. Characteristics of mothers who were referred to the LC were compared to those who were not referred to the LC using Mann Whitney U for continuous variables due to skewed distributions and Pearson chi-square or Fisher’s exact tests as appropriate for categorical variables. We hypothesized that mothers requiring intervention by a LC would have lower (poorer) scores compared to those not requiring intervention by a LC. In addition, a receiver operating characteristic (ROC) curve was used to determine the optimal cut-off score on the BEST tool for recommending breastfeeding intervention, defined as the need for a referral to the LC. To determine the cut-off score, we utilized the Youden’s index, which identified the threshold that maximizes the sum of sensitivity and specificity on the ROC curve [26]. Criterion validity was assessed by comparing participants’ total scores with LC’s scores using Bland-Altman plots, and acceptable agreement was defined as at least 95% of the points within 2 standard deviations of the mean difference in scores on the Bland-Altman plot [27]. Percent agreement (i.e. responses that were same between the LC and participant out of total responses) of each individual domain item was also calculated. Internal consistency of the tool was measured using Cronbach’s alpha score, and a score of ≥ 0.70 was considered acceptable [28]. In addition, face validity was assessed by asking participants to determine if the tool was relevant, useful and easy to understand, measured on a 5-point Likert scale (strongly agree, agree, neutral, disagree, strongly disagree). The proportion of those who strongly agreed and agreed were calculated, with a score of ≥ 0.80 considered acceptable [29]. Participants were also invited to provide comments in free-text format.

Data were analyzed using IBM SPSS software v20 (SPSS Inc., Chicago, IL) with a two-tailed test and p-value < 0.05 considered to be statistically significant.

Results

Content validity by experts

All eight domain experts concurred that all items were relevant and important for the initial breastfeeding journey. Individual I-CVI values for relevance, clarity, simplicity and ambiguity were all above the threshold of 0.83 (Table 1). There was also good inter-rater agreement among items (modified κ = 1.00 for relevance, 0.84 to 1.00 for clarity, 0.84 to 1.00 for Simplicity and 0.84 to 1.00 for ambiguity). Overall, the tool demonstrated acceptable content validity, with S-CVI/Ave scores for relevance, clarity, simplicity and ambiguity of 1.00, 0.98, 0.96 and 0.94 respectively. S-CVI UA for relevance and clarity were above the acceptable threshold, but simplicity and ambiguity were below the threshold (0.67 and 0.50 respectively), hence the phrasing of the questions was revised according to the experts’ suggestions to enhance the simplicity and reduce ambiguity (Additional file 2).

Table 1 Detailed results of item content validity indices

Internal consistency, face, criterion and construct validity by breastfeeding mothers

A total of 58 participants completed the tool, and were included in the analysis (Table 2). Majority (51/58, 78%) of participants completed the tool in the first 1–2 days postpartum, while the remaining completed the tool between day 3 and 23 of the infant’s life. Participant responses are shown in Fig. 1. Participants who were referred to the LC were more likely to be primiparous (69% vs. 31%), have completed tertiary education (80% vs. 20%), and have worse BEST scores (median 6.5 vs. 8.0) compared to those not referred to the LC (all p < 0.05). Other variables including maternal age, mode of delivery and presence of gestational diabetes were similar (all p > 0.05).

Table 2 Participant characteristics

Fig. 1

Breakdown of participant responses on the breastfeeding education and screening tool

There was acceptable face validity by participants, who found the tool easy to understand (87.9% strongly agree or agree), relevant (91.4% strongly agree or agree) and useful (86.2% strongly agree or agree). Free-text comments by participants included the tool being “easy to use”, having “useful visuals”, and “increasing their confidence”, although a participant felt that “there could be more questions”. Based on the participant comments, no revision was required of the tool. However, internal consistency of the tool measured using the Cronbach alpha score was 0.66, which improved to an acceptable range of 0.74 with the removal of question 2 (“Engorgement”) (Table 3). The decision was thus made to remove question 2. With the remaining five items, an ROC curve analysis identified a cut-off score of 6 or less indicative of the need for an LC referral. The analysis also showed an area under the curve (AUC) of 0.79 [(95%CI 0.67–0.90), p < 0.001] with a sensitivity of 86% and a specificity of 55% (Fig. 2). The final tool is shown in Additional file 2.

Table 3 Internal consistency of initial tool items

Fig. 2

Receiver operating characteristic curve for scores of participants referred to lactation consultant

Bland-Altman plots of participant and LC scores based on the revised five-item tool demonstrated an acceptable mean difference of 0.22 ± 1.65 (95%CI: -3.02–3.47), with 95% of points (55 of 58) within the limits of agreement [27] (Fig. 3). Percent agreement of the domain items ranged from 40 to 86% (Table 4).

Fig. 3

Bland-Altman of lactation consultant vs. participant scores. Legend: LC, Lactation consultant

Table 4 Agreement between participant and lactation consultant scores

Discussion

Despite the well-known health benefits, breastfeeding rates in developed countries remain low (2), and a multi-faceted approach is needed to support breastfeeding mothers in order to improve these rates. In a healthcare setting, a critical time for intervention is during the immediate postpartum recovery period when lactation is being established, and where resources and support are readily available. The BFHI recommends that all mothers to receive breastfeeding support from trained professionals during their postpartum stay. Assessment of breastfeeding skills and difficulties at this time can enable timely delivery of tailored lactation support to mothers. Thus, we have developed and validated a simple tool for self-screening and educating mothers on breastfeeding techniques and the need for intervention, particularly during the early postpartum period. Our tool had acceptable content validity, demonstrated by I-CVI and S-CVI/Ave scores above the threshold of 0.83. There was also acceptable face validity, with breastfeeding mothers reporting that the tool was easy to understand, relevant and useful. Our tool was also able to discriminate between mothers who required lactation intervention by a LC and mothers who did not.

Core breastfeeding skill items that have been covered in existing breastfeeding assessment tools include infant and mother’s positioning, attachment at breast, effectiveness of feeding and health of the breast, which we attempted to include in our tool [14]. Our original tool consisting of six items (nipple shape, engorgement, positioning, latch, swallowing, emptying) had unacceptable internal consistency, with a Cronbach alpha score below the acceptable threshold of 0.70. This was primarily due to a low item-total correlation for question 2 on “engorgement”. We postulate that this may be due to the response options for engorgement, where a “poor” rating is defined as “your breasts feel engorged, warm with some redness, and you are in pain” and a “good” rating is defined as “your breasts are soft and non-tender”. As our participants were mostly 1–2 days postpartum, they may have been experiencing breast engorgement, which did not completely resolve even with feeding. This suggests that different questions may be required for early and late stages of breastfeeding, as concerns and problems regarding breastfeeding may change with infant’s age or lactation stage. Since our goal was for the tool to be used primarily in the early days of breastfeeding to facilitate early intervention, we decided to omit the question on engorgement, which improved the overall Cronbach alpha score.

A Bland-Altman analysis comparing LC and participant administered scores demonstrated acceptable agreement between the expert and the participant, representing adequate criterion validity. In comparing agreement on individual items, the lowest agreement between LC and participants occurred for the question on “emptying” of the breast. One possible explanation for this is that the question addresses a mother’s perception of how empty her breast feels after breastfeeding, which is best self-assessed rather than being evaluated by an external party. Another possible reason is that, in the early postpartum days prior to secretory activation, it may be difficult to determine fullness and hence emptying of the breast. Other more objective measures on our tool such as positioning and nipple shape demonstrated higher agreement percentages of 86% and 79%, respectively. We retained the question on “emptying” despite the low agreement because we considered the mother’s experience to be an important contributor to the overall breastfeeding experience [14].

Breastfeeding education within one week of delivery has been shown to reduce rates of breast engorgement and pain, and increase rates of exclusive breastfeeding at 4–6 weeks postpartum [19]. In addition to screening for potential breastfeeding difficulties, our tool also aimed to provide simple education to mothers on their breastfeeding skills in the early postpartum stage. This is facilitated by detailed descriptions and images of appropriate and inappropriate breastfeeding techniques, such as positioning and latching, which are novel features of our tool.

Our five-item tool is concise, simple to complete and has shown adequate validity in our population for maternal self-assessment and identification of breastfeeding concerns in the immediate postpartum period. The benefits of self-administration include flexibility for mothers to complete the tool at any time and place during the postpartum phase, and an empowerment of mothers in their breastfeeding and parenting journey. Importantly, our study suggests that breastfeeding support is welcome by mothers in our population, demonstrated by the high percentage of mothers deeming our tool as relevant. As referrals to LCs for breastfeeding problems are currently subjective and opportunistic, this tool could potentially be used to standardize breastfeeding assessment and onward referral to lactation specialists within a clinical lactation pathway. We aim for the tool to be used by maternity ward nurses to educate mothers during the initial breastfeeding episodes postpartum, and allow mothers to screen for breastfeeding concerns using the tool herself or with assistance. Based on their score, interventions of varying intensity and level of support can be provided to the mother. For example, mothers with high scores will be supported by a maternity ward nurse, while mothers with poorer scores (i.e. ≤6) will be referred onward to a lactation consultant. Of note, research has shown that continued lactation support after discharge from the hospital is important in promoting continued breastfeeding, and this tool can also inform the frequency of follow-up required after hospital discharge [30]. For example, mothers with poorer scores may require more frequent follow-ups to achieve adequate breastfeeding skills. Thus, the BEST tool will need to be studied as part of a larger clinical pathway to support postpartum mothers in breastfeeding. Proper implementation and evaluation of the pathway would be critical, with considerations for timely assessments, close support of mothers with low scores, and accessibility to skilled lactation support both in and out of the hospital.

There are several limitations of this study. First, we did not include questions on breastfeeding self-efficacy in our population, which has also been shown to be predictive of breastfeeding duration [31]. The main reason for this was to keep the tool brief, minimizing respondent burden, especially since the immediate post-partum stage can be a busy time for mothers. Other tools that measure a mother’s breastfeeding self-efficacy have been developed and validated, and may be used in conjunction with our tool to provide a more comprehensive assessment of postpartum breastfeeding support needs. Secondly, participant self-assessments were conducted only on a single breastfeeding episode, which may not be representative of all breastfeeding sessions. Although certain breastfeeding techniques and anatomical features do not change with time, responses to some of the questions (e.g. swallowing, emptying) may vary among feeds, and with onset of secretory activation. Mothers in our study mainly completed the tool during the first 2 days postpartum, based on a single breastfeeding episode. Further study may be worthwhile to determine whether a single feed assessment adequately represents all feeding episodes, and re-administration of the tool may be required to determine the optimal postpartum period to complete the tool. Lastly, our study was cross-sectional in nature, preventing determination of the tool’s predictive validity. Future steps include studying the use of BEST as part of a clinical care pathway, in a larger cohort using a longitudinal prospective study design to determine long-term breastfeeding outcomes, such as rates of exclusive breastfeeding, and breastfeeding duration.

Conclusions

We have developed and validated a simple five-item tool, named the Breastfeeding Education and Screening Tool (BEST), for self-assessment of breastfeeding concerns in mothers during the early postpartum stage. Further study on the predictive validity of this tool, particularly within a clinical care pathway, will help determine if it can effectively empower mothers and predict breastfeeding duration and exclusivity.