Readability Analysis of the Package Leaflets for Biological Medicines Available on the Internet Between 2007 and 2013: An Analytical Longitudinal Study

Background The package leaflet included in the packaging of all medicinal products plays an important role in the transmission of medicine-related information to patients. Therefore, in 2009, the European Commission published readability guidelines to try to ensure that the information contained in the package leaflet is understood by patients. Objective The main objective of this study was to calculate and compare the readability levels and length (number of words) of the package leaflets for biological medicines in 2007, 2010, and 2013. Methods The sample of this study included 36 biological medicine package leaflets that were downloaded from the European Medicines Agency website in three different years: 2007, 2010, and 2013. The readability of the selected package leaflets was obtained using the following readability formulas: SMOG grade, Flesch-Kincaid grade level, and Szigriszt’s perspicuity index. The length (number of words) of the package leaflets was also measured. Afterwards, the relationship between these quantitative variables (three readability indexes and length) and categorical (or qualitative) variables were analyzed. The categorical variables were the year when the package leaflet was downloaded, the package leaflet section, type of medicine, year of authorization of biological medicine, and marketing authorization holder. Results The readability values of all the package leaflets exceeded the sixth-grade reading level, which is the recommended value for health-related written materials. No statistically significant differences were found between the three years of study in the readability indexes, although differences were observed in the case of the length (P=.002), which increased over the study period. When the relationship between readability indexes and length and the other variables was analyzed, statistically significant differences were found between package leaflet sections (P<.001) and between the groups of medicine only with regard to the length over the three studied years (P=.002 in 2007, P=.007 in 2010, P=.009 in 2013). Linear correlation was observed between the readability indexes (SMOG grade and Flesch-Kincaid grade level: r2=.92; SMOG grade and Szigriszt’s perspicuity index: r2=.81; Flesch-Kincaid grade level and Szigriszt’s perspicuity index: r2=.95), but not between the readability indexes and the length (length and SMOG grade: r2=.05; length and Flesch-Kincaid grade level: r2=.03; length and Szigriszt’s perspicuity index: r2=.02). Conclusions There was no improvement in the readability of the package leaflets studied between 2007 and 2013 despite the European Commission’s 2009 guideline on the readability of package leaflets. The results obtained from the different readability formulas coincided from a qualitative point of view. Efforts to improve the readability of package leaflets for biological medicines are required to promote the understandability and accessibility of this online health information by patients and thereby contribute to the appropriate use of medicines and medicine safety.


Introduction
Today, health care professionals are no longer the only source of information for patients on matters related to health because the new information and communication technologies have increased the capacity of patients to seek information independently [1][2][3].That is why patient education is growing in importance; in particular, written health-related materials available on the Internet [4], such as package leaflets provided by manufacturers.
One aspect that influences understanding of written information is literacy, which is defined as "using printed and written information to function in society, to achieve one's goals, and to develop one's knowledge and potential" [5].On the basis of this concept, health literacy is defined as "the degree to which individuals have the capacity to obtain, process, and understand basic health information and services needed to make appropriate health decisions" [6].The capacity of the individual is an important aspect in the definition of health literacy, referring to both innate and acquired skills [7].
Health literacy plays an important role in the evaluation of the online health information [8].An improvement in health literacy leads to a greater ability of patients to understand how to use medicines appropriately and that is why adequate levels of health literacy can improve medication safety and reduce adverse drug reactions [9,10].
Another aspect that also influences the understanding of written information is its readability, which can be measured using readability formulas.By the 1980s, there were some 200 published readability formulas and a large number of studies that attested to their validity [11].Nowadays, these formulas are widely applied to measure the readability of health-related written materials [12][13][14][15].It is recommended that the readability level of health-related written materials for end users is not higher than the sixth (age 11-12 years) [7,16] or seventh (age 12-13 years) grade [17].
Among health-related written materials, the package leaflet is "a leaflet containing information for the user which accompanies the medicinal product" [18].It is very important for the transmission of drug-related information to citizens and it can help to supplement and reinforce the information received from health professionals [19], possibly leading to increased adherence and a consequent decrease in health care costs [4,20].
Several studies have highlighted problems associated with the quality of the information contained in the European model of the package leaflet [21,22].This is in contrast with current European regulations, according to which the contents of the package leaflet must be clear and understandable, enabling the users to act appropriately, and it must be clearly legible in the official language or languages of the Member State [23].To achieve this aim, and in accordance with Article 65(c) of Directive 2001/83/EC, in 2009, the European Commission published the Guideline on the Readability of the Labelling and Package Leaflet of Medicinal Products for Human Use, which offers guidance on the writing of the labeling and package leaflet to facilitate understanding and the accessibility of their content [24].The guidance recommends, among other things, the use of simple words with few syllables and avoidance of long sentences.These explicitly refer to variables included in readability formulas (word length and sentence length).
The European Commission has final authority for authorizing the commercialization of human medicines within the European Union via a centralized authorization procedure.This centralized procedure is compulsory for biological medicines, among others [25].In 1982, an insulin product became the first recombinant medicine approved for sale [26].Since then, many biological medicines have been authorized for the treatment of a wide variety of diseases.The biological product market is projected to grow over the coming years and that may affect the introduction both of new products and of existing products for additional indications [27].In January 2013, there were 171 biological medicines authorized in the European Union [28], 12 of which were biosimilars [29].
In a previous study, the readability levels of package leaflets of some biological medicines (n=33) downloaded from the European Medicines Agency (EMA) website was studied between 2007 and 2010.No variation in readability was found between the two years-the readability of the package leaflets was above the recommended values for health-related materials-and there were differences between the readability levels of the different sections of package leaflets [30].In view of these findings, it seems important to extend the study to 2013 to see whether the guidance on readability has had an effect in package leaflets until that date.Thus, the main objective of the current study was to calculate and compare the readability level and length (number of words) of the package leaflets of biological medicines in 2007, 2010, and 2013.We hypothesize that there will be an improvement in the readability of package leaflets, especially in 2013, 4 years after publication of the guidance on readability.We considered 4 years to be sufficient time for manufacturers and marketing authorities to have applied the recommendations in the guidelines.We also wanted to analyze a potential link between readability and length.Moreover, the possible influence of some categorical variables on readability and length was also studied.

Type of Study and Inclusion/Exclusion Criteria
We designed and performed an analytical longitudinal study.The study sample consisted of biological medicine package leaflets that were authorized by the EMA by January 2007 and continued to be authorized in December 2013.Of these, one medicine per active substance was chosen.
The following types of medicines were excluded from the study so as not to introduce bias into the results: vaccines because they are used as a prophylactic measure and, moreover, most are administered by physicians to pediatric patients leading to reduced importance of package leaflets; insulin products because diabetic patients usually know their illness and medicines very well and package leaflet readability is secondary; and botulinum toxin because it was the only biological medicine at the time of the study that contained a toxin as an active substance.

Data Collection
The package leaflets were downloaded from the EMA website [31] at three different times: January 2007, July 2010, and August 2013.The same pharmaceutical form was chosen in all three years for each medicine.

Sample Characteristics
As in our previous study [30], the sample of package leaflets (n=36) was divided into five groups depending on the source of the drug [32]: monoclonal antibody (mAb) products, cytokines, therapeutic enzymes, recombinant blood-related products, and recombinant hormones (see Multimedia Appendix 1).Five of the six sections according to the EMA package leaflet model/template [33] were evaluated: (1) what X is and what it is used for, (2) what you need to know before you take (or use) X, (3) how to take (or use) X, (4) possible side effects, and (5) how to store X.The section (6) "contents of the pack and other information" was excluded because it is considered by patients less important than the other sections [22] and its content was highly similar for all the package leaflets.The "annex" section, which provides information about instructions for use, was also evaluated in the package leaflets where it appeared (10 leaflets in 2007, 11 in 2010, and 12 in 2013).
The evaluated sections of the package leaflets were copied as plain text into individual Microsoft Word 2007 files.Before calculating quantitative variables, the following modifications were made: 1. Titles, subtitles, citations, tables, graphs, images, references, header tables, figure captions, and the brand name of the medicines were deleted. 2. All abbreviations, unit and magnitude symbols, numbers, and acronyms were replaced by their full version because when applying the readability formulas these must be treated as if read aloud [34]. 3. Bullets (eg, dashes, numbers, asterisks) were deleted. 4. Compound words and numbers were considered as a single word.

Quantitative Variables: Readability Indexes and Length
The quantitative variables calculated for the package leaflets were the length (number of words) and three readability indexes: SMOG grade [34], Flesch-Kincaid grade level [35], and Szigriszt's perspicuity index [36].
The readability indexes were chosen taking into account the following criteria: 1. SMOG grade and Flesch-Kincaid grade level are commonly used in recently published health care literature and they have been validated by different methods [37].This can make it useful to compare the results obtained with the two formulas. 2. SMOG grade is recommended for use in health-related written materials [1,37,38] because it is the only formula with 100% expected comprehension and is based on more recent criteria for determining reading grade level.For this reason, SMOG grade values are usually higher than Flesch-Kincaid grade level values when both formulas are applied to the same text [37]. 3. The qualitative interpretation of Szigriszt's perspicuity index was designed to assess the readability of written materials in Spanish, which is the language of the package leaflets analyzed in this study.
The SMOG grade formula has as a variable the number of words with three or more syllables, whereas the Flesch-Kincaid grade level and Szigriszt's perspicuity index use the number of words per sentence and the number of syllables per word.SMOG grade was calculated manually following the author's instructions, Flesch-Kincaid grade level was calculated using Microsoft Word 2007 software, and Szigriszt's perspicuity index was calculated following the author's instructions (before applying this formula, both the number of words and the number of sentences were obtained using Microsoft Word 2007 software and the number of syllables per word was obtained from previously calculated Flesch-Kincaid grade level values).
The length (numbers of words) of the texts was the fourth variable obtained from the package leaflets because more words on package leaflets can decrease the capacity to find certain information and decrease motivation to read the package leaflet and confidence in using the medicine correctly after reading it
To obtain these variables, the whole text was evaluated in this study to avoid bias that could be introduced in the choice of samples.

Categorical Variables
The influence of some categorical variables on readability and length was studied.These variables were: 1. Year of downloading the package leaflet (three groups): 2007 (n=36), 2010 (n=36), and 2013 (n=36).2) what you need to know before you take (or use) X (n=36), (3) how to take (or use) X (n=36), ( 4) possible side effects (n=36), ( 5) how to store X (n=36), and annex (n=10 in 2007, n=11 in 2010, n=12 in 2013). 3. Group of medicine according to its source (five groups): mAb products (n=6), cytokines (n=11), therapeutic enzymes (n=4), recombinant blood-related products (n=9), and recombinant hormones (n=6).For the second variable (section of package leaflet), the values per section were taken into account; for the rest of the variables, the mean of each readability index per package leaflet and the total length of each package leaflet were considered to have a single value per package leaflet.

Effect of Package Leaflet Year on Readability and Length
Table 1 shows readability (mean values for package leaflet) and length results for package leaflet and year studied.
All the SMOG grade and Flesch-Kincaid grade level values exceeded the recommended readability levels for health-related written materials (all SMOG grade and Flesch-Kincaid grade level values were much higher than 6).In addition, all the Szigriszt's perspicuity index values were less than 75; that is, no package leaflet was easy to understand according to this scale.The respective statistical analysis of the results in Table 1 is presented in Table 2.
No statistically significant differences were found in the readability values between the three years (P=.40 in SMOG grade, P=.22 in Flesch-Kincaid grade level, P=.20 in Szigriszt perspicuity index), but differences emerged in terms of the length (P=.002), with the length of the package leaflets increasing over the 6-year period (Figure 1).

Effect of Package Leaflet Section on Readability and Length
Table 3 and Multimedia Appendix 2 describe the readability indexes and length per package leaflet section in the three years studied.Statistically significant differences can be observed between the four variables and the six sections: (1) what X is and what it is used for, (2) what you need to know before you take (or use) X, (3) how to take (or use) X, (4) possible side effects, (5) how to store X, and ( 6) annex (P<.001).
These results (Table 3, Multimedia Appendix 2) show the following order of readability of the sections of the package leaflets (from easiest to understand to most difficult): (1) section 5, (2) annex, (3) section 3, (4) section 2, (5) section 1, and (6) section 4. According to the Szigriszt's perspicuity index medians (Table 3) and the scale of perspicuity level, section 4 was rather difficult, section 1 was standard in 2007 and rather difficult in 2010 and 2013, sections 2 and 3 were standard, and section 5 and the annex were rather easy.Thus, sections 1 and 4 were rather difficult to understand in 2010 and 2013, and the readability of section 4 (the most difficult section) decreased during the period studied (Figure 2).

Effect of Source of Biological Medicine on Readability and Length
When comparing readability levels as a function of the source of the medicine, no statistically significant differences were observed between the five groups (mAb products, cytokines, therapeutic enzymes, recombinant blood-related products and recombinant hormones) of package leaflets (SMOG grade: P=. 44  (P=.03).However, differences could be observed between the five groups in the length of the package leaflets in the 3 years studied (P=.002 in 2007, P=.007 in 2010, P=.009 in 2013).These differences in length were mainly due to the difference between therapeutic enzymes and cytokines when applying the Bonferroni posttest for multiple comparisons (Table 4

Effect of Marketing Authorization Holder on Readability and Length
Even though differences were found between groups for the readability indexes in 2007 and for Szigriszt's perspicuity index in 2010 and 2013 (SMOG grade: P=.01 in 2007; Flesch-Kincaid grade level: P=.01 in 2007; Szigriszt's perspicuity index: P=.01 in 2007, P=.03 in 2010, P=.04 in 2013), these differences not were observed in the Bonferroni posttest for multiple comparisons (P>.05).This was probably due to a lack of statistical power because the samples were considered too small.For this reason, regarding the comparative study of groups of package leaflets as a function of marketing authorization holder, no statistical evidence was available to show the influence of marketing authorization holder on readability and length.

Relationship Between Quantitative Variables
The results of studying the relationship between the three readability indexes and length are presented in Table 5.Although the P value was significant for all pairs of variables, taking into account the coefficient of determination, only a linear correlation between the readability indexes was evident.This correlation can be observed in the respective scatterplots (Figure 3).

Principal Findings
The objective of this study was to determine the readability and length of package leaflets for biological medicines in three different years.Our results show that none of the package leaflets evaluated met the recommended readability levels for health-related written materials.Therefore, the package leaflets are not in line with European legislation, according to which they must be clearly legible and understandable [23].This finding could negatively affect patient understanding of the information contained in package leaflets and result in a reduction of adherence intentions [40,41].A recent study in which patient information materials were evaluated in terms of readability and variety of content found that these materials did not promote health literacy and were only accessible to a proportion of higher skilled patients, which could ultimately increase inequalities in health [42].
Moreover, no improvement in readability was observed over the 6-year period analyzed.This result was not in line with our expectations because the European Commission Guideline [24] recommended the use of words with few syllables and avoidance of long sentences in 2009.In accordance with this guideline, the readability of package leaflets should have increased, but this is not the case.
Furthermore, the number of words in package leaflets increased between 2007 and 2013, which is also a problem for patients.This trend has existed in Europe for a number of years [39].The observed increase in the length of the package leaflets over the 6-year period studied could be a consequence of prevention policies of the EMA and the time since first authorization that may lead to there being more information related to pharmacovigilance.
When the sections of package leaflets were compared, differences between them were observed in both readability and length in all the years studied.The most difficult sections contain information about therapeutic indications (section 1: what X is and what it is used for) and side effects of the medicines (section 4: possible side effects).This information is considered very important by patients [22] because it highlights the importance of promoting an understanding of the need for medicines and individual risk for side effects when taking medicines [43], and allows them to undertake a rational benefit-risk assessment of their medication [44].Herber et al [44] stated that "package leaflets need to convey potential risk information in a language that is less frightening while retaining the information content required to make informed decisions about the prescribed medication."In contrast, the most understandable section (section 5: how to store X) is considered less important by patients [22].
Regarding the length of sections, the shortest section was section 5 (how to store X) and it was also the most understandable.In contrast, the annex section was the longest section, but it was more understandable than other sections because it has shorter sentences and contains fewer technical terms.Moreover, section 4 (possible side effects) was not the longest section, but it was the least understandable as mentioned previously.This is because in most package leaflets it contains a long list of difficult medical words, which can prevent appropriate interpretation by patients [39].
In relation to the studied correlation between the three readability indexes and total length, only the readability indexes showed an acceptable linear relationship, which was not observed between the readability indexes and length.Thus, we can conclude that the readability of package leaflets is not associated with their length.Fitzsimmons et al [1] assessed the readability of online consumer-oriented Parkinson's disease information using the Flesch-Kincaid grade level and SMOG grade formulas.They found that webpage length was not associated with readability, suggesting that reading difficulty of websites evaluated was independent of word count.Nevertheless, it is recommendable to reduce the length of package leaflets to make them easier to read and to motivate the patient to access and understand them [1,39].Indeed, some authors have proposed an alternative template for European package leaflets [45].

Limitations
Our study has some limitations.First of all, one limitation is the lack of a test of package leaflets according to target patient groups.According to Article 59(3) of Directive 2004/27/EC, package leaflets should reflect the results of user testing to ensure readability.Methods to assess package leaflets using patients have been published [24,46,47], but there are also different studies published in which the results obtained by applying readability formulas and those using user testing have been consistent [48].Thus, before undertaking a user test or any other form of user consultation [24], the use of formulas can be considered a first step to predict understandability and to initially identify readability problems independently of the type of patient.
A second limitation of this research is the lack of assessment of other characteristics of package leaflets that also improve their readability, such as font, figures, design, and layout [16,24,47,49].It is important to point out that there are several indirect instruments that assess these characteristics in health-related written materials, such as Suitability Assessment of Materials (SAM) [16] and Patient Education Materials Assessment Tool (PEMAT) [50].
Lastly, not all biological medicines were considered in this study.A sample of 36 biological medicines authorized in both 2007 and 2013 was analyzed taking into account the inclusion and exclusion criteria established.This sample constituted 36 of 61 (approximately 60%) medicines that complied with these criteria in January 2007 and only 36 of 126 (approximately 30%) in January 2013 [28].
Nevertheless, three readability formulas were used (and one of them especially for Spanish) to provide an objective measure of difficulty [11] of reading package leaflets.Moreover, the criteria for obtaining readability indexes have been explained in detail, allowing future comparisons with other studies.Furthermore, to avoid interpretation problems due to a bias in the selection of text samples, all the information contained in the package leaflets was used to calculate the quantitative variables.Finally, this longitudinal study considered adaptation to EU guidelines over a 6-year period of the same health-related written materials.

Future Implications and Conclusions
Most studies to date have shown that the materials targeted at patients are written with readability levels that make it difficult for the materials to be understood by most people.Package leaflets are this type of written material; therefore, they are no exception to what has been confirmed.
We continue to believe that it would be more than reasonable for applicants and marketing authorization holders, who are responsible for drawing up the package leaflets according to the EMA instructions, to be more conscious of the need to improve the readability of package leaflets together with a decrease of their length.This would increase the usefulness of package leaflets and access to the information they contain by patients.
In the same way, it is advisable for those responsible for drawing up the package leaflets to measure the readability with some of the formulas applied here (which we have shown to be highly correlated), as a method adopted in parallel to direct methods using patients.In this way, taking the measures as soon as possible and, depending on the results, package leaflets could be revised if necessary.Future studies are needed to explain the reasons why the readability of package leaflets has not improved over the years considered in this study, even though the EU legislation and guidelines have changed.
Finally, we suggest an alternative type of written medicine-related information targeted at patients that covers their information needs because the current package leaflet in the EU has proved to be a barrier that has not introduced significant changes in either its readability or length.The alternative written material should be more concise (shorter) than the current package leaflets and it should be written in clear language that is comprehensible for patients.

Figure 1 .
Figure 1.Evolution of the medians of length of the package leaflets studied.

Figure 2 .
Figure 2. Evolution of the medians by year of SMOG grade, Flesch-Kincaid grade level, Szigriszt's perspicuity index, and length for the package leaflet sections.Section 1: what X is and what it is used for; section 2: what you need to know before you take (or use) X; section 3: how to take (or use) X; section 4: possible side effects; section 5: how to store X; and annex.
2. Section of package leaflet (six groups): (1) what X is and what it is used for (n=36), (

Table 1 .
Mean values of the three readability indexes and total length for package leaflet (n=36) and year studied.

Table 2 .
Descriptive statistics of the three readability indexes and length by studied year and results of the hypothesis tests.

Table 3 .
Descriptive statistics of the three readability indexes and length by package leaflet section and year studied.
a Kruskal-Wallis test.

Table 4 .
P values obtained by applying the Bonferroni posttest for multiple comparisons of package leaflet length.

Table 5 .
Correlations between the three readability indexes and length.
a ANOVA.