ABSTRACT
Objective:
Our aim is to assess the content quality and reliability of Youtube videos on bone tumors.
Method:
We searched Youtube using the keyword “bone tumors,” and included the first 50 videos listed in our study. Two orthopedic surgeons analyzed the videos, and then, we examined them in terms of length, number of views and likes, and source. To evaluate their content quality, the Global Quality score (GQS) (0-4), Journal of the American Medical Association (JAMA) (0-5), and DISCERN (15-75) scoring systems were used; based on these, the obtained data were statistically analyzed.
Results:
Of the 50 videos analyzed, four (8%) included animation, while 46 (92%) did not; 12 (24%) were uploaded by doctors, 23 (46%) by health channels, 10 (20%) by universities, three (6%) by clinics, and one (2%) each by a hospital and a trainer. The average video length was 16 minutes and 18 seconds (38-6.088 seconds), average number of views was 14,856.24, and average number of likes was 130.50 (1-1448). The mean scores of GQS, JAMA, and DISCERN were 2.22 (1-4), 2.12 (1-3), and 33.48 (15-75), respectively. There was no statistical significance in the scores and video length, or the number of likes and views between the two researchers.
Conclusion:
Youtube videos on bone tumors have low content quality. Improvement of the same will help dispense correct information to patients, so that they can continue their treatment. We believe that patient treatment compliance can be increased by accelerating the patient’s preparation and adaptation process for treatment with accurate information.
Introduction
In today’s times, Internet is the easiest way to access information. A majority of people refer to sources online to obtain information prior to their medical applications, as well as in other areas. Video sharing sites such as YouTube have become very popular in this respect.
More than 1.9 billion people visit Youtube every month and can find a huge number of medical education videos (1). Patients frequently visit the platform and benefit from its rich video archive while trying to gather information about their diseases (2-5).
Those suffering from orthopedic disorders are among these patients. In particular, patients with suspected malignancy are more anxious and may need a more comprehensive search; they may increase their levels of anxiety by constantly thinking the worst and referring to the most unfavorable examples online. Our Google search for “bone tumors” on March 10, 2020 yielded 62 million results, which may be an indication for that this particular search is very popular. However, studies show that the content quality of health information videos online is low (6). In order for people to access more accurate information, videos with better quality content on health information are needed (7).
Bone tumors can be perceived in various ways, and because patients suspect of having them and worry, they conduct online searches.
Thus, the purpose of our study was to evaluate the content quality of Youtube videos on bone tumors for those who want accurate information.
Materials and Methods
We did a search on Youtube using the keyword “bone tumors” on March 10, 2020 and included the first 50 videos that were listed in our study. Videos in English that were repeated and those that comprised only product advertisements and not related to bone tumors were excluded.
The videos were analyzed by two independent orthopedic surgeons. We recorded information on the videos’ length, number of views and likes, and the source (i.e., the uploader), and then scored them using the Global Quality score (GQS), Journal of the American Medical Association (JAMA) and DISCERN scoring systems to evaluate their content quality (8-10) (Figure 1,2) (Table 1).
Scoring was done by taking the means of the scores provided by both surgeons. The GQS is a system that scores between 0 and 4, from the lowest quality to the most quality. The JAMA scoring system comprises four criteria, each scored either 0 or 1, adding up to a maximum of 5 points and a minimum of 0 to measure the content quality of the videos. The DISCERN scoring system comprises 15 questions; its scores between 63 and 75 points are classified as “excellent,” 51 and 62 as “good,” 39 and 50 as “average,” 28 and 38 as “poor,” and <28 as “very poor.” Higher scores obtained from the scale indicate a higher quality of information. This study does not contain any human or animal resources, ethical approval was not needed for this study.
Statistical Analysis
The Shapiro-Wilk test was conducted to evaluate the fitness of the numerical variables to a normal distribution, and then, the Mann-Whitney U test to compare the non-normally distributed variables in two independent groups. Relationships between the variables that were not normally distributed were analyzed using the Spearman’s rank correlation coefficient. The SPSS 22.0 Windows software package was used for the analyses. For all analyses, p<0.05 was considered as statistically significant.
Results
Of the 50 videos analyzed, four (8%) included animation, while 46 (92%) did not; 12 (24%) were uploaded by doctors, 23 (46%) by health channels, 10 (20%) by universities, four (8%) by clinics, and one (2%) each by a hospital and a trainer. The average video length was 16 minutes and 18 seconds (38-6.088 seconds), the average number of views was 14,856.24, and the average number of likes was 130.50 (1-1.448). The mean scores of GQS, JAMA, and DISCERN scoring systems were 2.22 (1-4), 2.12 (1-3), and 33.48 (17-66), respectively (Table 2).
The statistical analysis showed a positive, medium correlation between the video length and GQS score (p=0.001), and a positive, weak correlation between the video length and GQS and DISCERN scores (p=0.000/ p=0.006). There were no statistically significant differences between the video length and JAMA score (p=0.978).
There was a significant correlation between the number of views and GQS and DISCERN scores, but not between the number of views and JAMA score (p=0.049/p=0.079/ p=0.038).
There was no significant correlation between the number of likes and GQS, JAMA, and DISCERN scores (p=0.058/ p=0.067/p=0.071). The statistical analyses are summarized in Table 3.
On the other hand, with respect to the videos uploaded by doctors, the length (p=0.012), number of views (p=0.025), number of likes (p=0.003), and JAMA scores (p=0.011) were significantly higher than those of videos uploaded by other sources. The JAMA scores of videos uploaded by health channels were significantly higher than those of videos uploaded by universities (p=0.001).
There were no significant differences about scoring system between two surgeons. (p=0.012).
Discussion
Our study shows that the content quality of Youtube videos on bone tumors, searched using the keyword “bone tumors,” is low.
This study is the first to focus on orthopedic malignancies, and it provides information on the effects of video content quality on patients. Of the 50 videos reviewed, 12 were uploaded by doctors, 23 by health channels, 10 by universities, four by clinics, and one each by a hospital and a trainer. This indicates that the topic is addressed mostly by professionals. Although a literature review has shown that video uploading by patients is done at a level that cannot be underestimated, in our study, health professionals are prominent (11).
The mean video length in our study was 16.18 minutes. Previous studies have shown a range of 6.17-10.35 minutes (12,13), which means that our mean was higher.
Furthermore, in our study, the length, number of views and likes, and JAMA scores of videos uploaded by doctors were higher than those uploaded by other sources. This indicates that the doctors’ videos are of a higher quality in terms of content. This may indicate that the videos uploaded by doctors are more scientific.
Several scoring systems have been used in the literature to determine the content quality of Youtube videos (14,15), we used three- GQS, JAMA, and DISCERN- and our data analysis revealed their scores as 2.22, 2.12, and 33.48, respectively. Thus, the mean scores according to these systems correlate with low content quality. Previous studies have compared this with the source of videos, showing that those uploaded by clinicians had high content quality (16). However, for most of the videos reviewed in our study, which were uploaded by health professionals, we did not find it relevant to make such a comparison.
There was no significant correlation between the number of likes and scores. This shows that the likes do not correlate with the content quality of the video, and there are other studies showing that videos of low content quality have more likes (17,18).
The selection of this topic is based on the consideration that cancer patients may want to do more research on their diseases. This is because malignancies are always a matter of curiosity for patients- they are stressed because of their illness and feel the need to gather more information. It is known that Youtube videos with low content quality negatively affect the patient-doctor relationship (19). Additionally, a literature search on Youtube videos related to other malignancies reveals that there is no study focusing on bone tumors (20). Orthopedic studies in the literature focusing on quality assessment of Youtube videos are limited in number. These are on scoliosis, femoroacetabular impingement, hip arthritis, anterior cruciate ligament injury, and reconstruction (21-24) among other topics. However, there is a lack of studies on orthopedic malignancies. Therefore, we contribute to the literature by studying the content quality of videos on bone tumors and reporting that patients who are curious about this topic receive information from videos with low content quality.
While our method of selecting and analyzing the first 50 videos in the search list can be considered a limitation, it should be noted that there are studies in the literature that have conducted similar types of evaluations (25,26). In addition, considering that patients watch those videos that show up first in search results, the number of our videos is acceptable. Doing the search on different days and performing the assessment at different times would have been another limitation; hence, to prevent that, both of us completed our analyses on the same day.
In this study, the all videos related to bone tumors were analyzed by two independent surgeons. All videos rated by surgeons got close scores in the scoring system. One of our main goals here was to demonstrate that videos got the same results by all surgeons watching.
The dissemination of accurate and reliable information regarding the “bone tumors” may also play a significant role in educating patients and optimizing outcomes. More accurate information about the bone tumors can have a positive impact on patient outcomes in a multifactorial manner. If patients are properly informed, they will be more likely to seek timely treatment, and better outcomes may ensue.
In sum, a majority of the Youtube videos on bone tumors have low content quality, which may become patients’ primary source of information due to easy access to videos. Thus, improving the content quality of videos would be beneficial for both patients and health professionals. Our study is the first to assess the content quality of Youtube videos on bone malignancies, and further research is needed on this topic.