Question 2
As this systematic review looked at self-monitoring, all of the data was self-reported by participants. How much of an impact does that make on the results of the various studies? Can you think of any ways to validate the self-reported data more?
posted by Kara @ 6:13 PM
32 comments
32 Comments:
In order to study a large sample size, I believe that self-reported data is the most advantageous way to conclude whether or not self-monitoring aids in weight loss. Some of the data may be subjective, such as serving sizes when eating at a restaurant or exercise intensities. Therefore, there could be fluctuations in the data if a researcher were to report the data for each person individually, just as there may be when the data is self-reported. When collecting data on a large sample of adults that is fairly detailed, such as self-monitoring in weight loss, self-reporting is more than likely the most realistic route to take.
The fact that all of the data was self-reported by the participants is clearly going to have an impact on the results of the studies. Just as any other study, self-reported data questions the validity of the results because no one can truly know, except for the participant, whether the information is truthful. However, like Liz mentioned, self-reported data is the most realistic. Besides having a live-in dietitian with each of the participants, there really is no way to get 100% accuracy with the information being provided. We can only hope that people do not stand to gain or lose anything by lying so hopefully this will steer them to tell the truth. A more time consuming and expensive route the research could have taken, is have the participants meet with a dietitian weekly or even monthly, showing them their self-monitoring records and having them weighed and their body composition taken. This method, however, still strongly relies on self-reported data and I do not believe there is any way to get around that.
I believe that using self-reported data in this type of research is appropriate, especially for a study with a longitudinal design with many participants involved. Self-reported data is likely not entirely accurate due to lack of knowledge on portion sizes, ingredients/preparations in restaurant foods, inaccurate information reported on how long a person exercised, etc. We are human and make mistakes....unless someone is looking over the participants' shoulders day in and day out throughout the span of the study documenting portions, types, preparations, exercise levels, etc. the information is bound to be slightly off. Using food models, measuring cups, and food/exercise diary instructions to explain exactly how the participant is supposed to track their dietary intake and exercise efforts would be helpful in validating the self-reported data.
Liz,
I agree with you that using self-reported data is most realistic with this type of study. The variables being studied would rely on comprehensive documentation and participants may not always have their food/exercise diary with them (may have forgot it or did not have it with them at the time of their meal/snack/workout), which would make it hard for anyone to recall exactly everything they did exercise wise or consumed. In my opinion, this was the most applicable way for the findings to be compiled.
With a large sample size I believe self-monitoring is the most efficient way to gather data. To increase the likely hood of accurate reporting the researchers could give participants information on topics such as how to measure portions and to weigh themselves at the same time every day to aid in accuracy. The only way to ensure accurate reporting would be to have a trained researcher with the participants to record their daily habits. To do that would not be cost or time effective.
Melanie,
I agree with you that many different educational tools could be used to help the client with accurate reporting. Education is probably one of the only ways that would allow for all the participants to record correct information without the help of a trained professional.
I think using self reported data is the most efficient and effective way to determine whether self-monitoring is effective or not. One thing that could have been done to increase the validity of the results would be to provide educational materials to the participants before the study began. Handouts about serving sizes, measurement, weighing, and proper ways to document in the food diary would help to ensure that participants are on the same level and are consistently and correctly reporting what they are supposed to. There is no way to be 100% accurate but providing education may help.
Bethany- I agree that there is no real way to get 100% accuracy in this type of a study. The suggestions you gave would help to validate the study slightly but as you said, the benefits do not outweigh the costs.
Like most of you mentioned, I don't see any better way of reporting the data other than self-reporting. I think that those who are motivated and want to see results will be as honest and accurate as possible. I agree that some educational materials would be helpful so participants know portion sizes, etc. I think that the self-reporting will most definitely impact the results because their is bound to be some errors ( I am sure that even an RD could make an error in forgetting to write something down - I can't even remember what I had for breakfast some days :) but I don't think that they will be significantly impacted especially if some education is given.
I think the accuracy of the self-monitoring is not as important as the frequency and routine of self-monitoring. Although I think a few follow-ups with dietitians could help clients/patients to better grasp portion sizes and label reading, nobody is perfect, and that should not be the goal of common place (not study-based) self-monitoring. It is more a matter of personal accountability and empowerment, not a perfect calorie count.
Jordan - I like your suggestion for clients to weigh themselves at the same time every day. Offering them a scale for a reduced price could also help them have accurately self-monitor weight. Fortunately, while researching for my thesis, I found some studies that showed self-reported weights are surprisingly accurate (e.g. Manson et al, 1990; Stewart, 1982; Stunkard & Albaum, 1981). However, there is always room for improvement, so tips and calibrated scales might be just the thing to help clients/patients be more informed about changes in their weight status.
The self-reporting nature of the data in these studies may impact the validity of the results but that is something that always comes up when self-reported data is used. As far as food/exercise journals are concerned, the participants may be underestimating their serving sizes thus underestimating total kcal intake and they also may be underreporting their weights.
Some ways to increase the validity would be to teach the participants how to properly weigh out or measure their food as to get more accurate kcal intake and to have them weigh at a clinic to get more accurate weights.
Liz:
I agree that self-reporting is the most realistic and cost effective route to take for data collection. With such a large sample size it is the only real way to collect all of the data in a timely inexpensive way.
This could potentially have a large impact on the results if a significant amount of data was falsified by the participants. For these types of research studies, self-reported data is the most logical way to obtain information. A significant amount of time and effort would need to be put into validating this data. The researchers would need to interview each participant to validate the data they reported on food intake. and even with qualified individuals conducting the interviews, you are still going to have some inaccuracies. I think for this type of research study, self-reported data is the most feasible way of collecting information.
Melanie made a good point that self-reported data would most likely be inaccurate due to knowledge of portion sizes, preparation techniques and duration of exercise. I agree that using food models, measuring cups and having instructions on how to complete the food and exercise diaries would be beneficial. This would still not be fool-proof, but it may be helpful in increasing the accuracy of the data.
There seems to be an agreement in response to this question that self reported methods are the best way to go about evaluating self monitoring for weight loss. However, although I do believe it is important to see how accurate the self reports are to the actual diet I have to say that I am not sure if it is really that important to know for the intentions of this study. If an individual eats less calories, exercises more often and loses more weight through self reporting, to me it is not as relevant how accurate these self reports are. The main focus is that by going through the action of self reporting, there is a link to successful weight loss. If the data was used to evaluate the amount of calories these women were eating a day, or the composition of their diets in order to come up with a conclusion about weight loss methods, then I believe the accuracy of the data would need to be carefully looked at.
Bethany,
I think that your response to a solution to having reliable self reported data was very accurate. There is no way to have a live in dietitian be with people as they fill out their recalls and prepare their food. However, meeting with a dietitian weekly and having someone keep tabs on their records and body composition may increase the reliability of the recalls.
Self-reported data has inherent bias in any research. However, it would be interesting to note if the research was reported on a self-selected basis or not. Bias or not, researchers depend on self-reported data in studies of this magnitude. As long as researchers identify this as a limitation, their bases are covered. \
Sarah, you bring up an interesting point. The validity of the study could be impacted be purposeful or inadvertent error in self-reported data. In other words, subjects purposely falsifying weight, or lack of education such as weighing themselves at the same time each day.
I think the factor of self-reported data definitely carries some weight on the results of the various studies. As we've mentioned several times within past blogs, self-reported data is easily skewed, but other options are not always reasonable. Ensuring that questionnaires and interviewers do not ask leading questions is vital. If there is anyway to videotape or photograph exercise and meals, that may possibly help to validate participants' actions. Otherwise, the fact that self-reported data is used, is simply just a limitation.
Yes, self-reported data needs to be utilized for cost and efficiency purposes. I agree with Anna that a possible follow-up session can also help validate the weight loss (if any) that has occurred during the time of self-monitoring. I feel it would be useful to educate the participants on portion sizes, food prep, restaurant food options, and food purchasing prior to starting the trials.
I think those that mentioned meeting with a dietitian and having the RD teach the participants about portion sizes, weighing at the same time each day, and self-monitoring techniques would be an excellent way to reduce the amount of error that could occur due to self-reporting. However, I still stand by what I said previously that no matter what there is going to be some human error or deliberate lying so it's important that, like Sarah G mentioned, the researcher report this in their results.
The fact that the data was self-reported by participants significantly impacts the results. As mentioned previously, the limitations include the possibility of falsifying data. It could be common for clients/patients to lie or twist their story to avoid embarrassment or fear of judgment. There may be current technology to get regular submissions of progress from clients. This could eliminate the falsification of monitoring on a regular basis, but will not eliminate the possibility of false submitted data. Clients could come in on a regular basis or have a home visit. The important thing to consider is self-efficacy, what will these clients does independently when they are no longer being monitored? It is important for people to get out of the process what they put into it and individuals may have different methods that work better than others. While it may be appropriate to monitor closely in the beginning, one must be able to be independent to achieve what we promote.
I agree with Michela that the self-reported data does not have too have been 100% accurate. The main goal and the reason behind all this research was to see if self-monitoring was an affect weight loss technique. The overall results have to be accurate and at least to a level that we can trust that there actually was a weight loss. That is the real goal.
Self-reported data will always have flaws. I believe our role only goes so far one this one, we can ensure confidentiality and that their data will be only used for research purposes, and couple that with encouraging them to be as truthful as possible.
Errors in self-reporting would influence the findings of a study but I think the goal is weight loss for the client regardless of whether or not everything is accounted for on the food record. Making use of technology, where possible, could help improve reliability. Cameras can be used to take pictures of foods or meals eaten (taking the guess work out of portion sizes) while logs can be kept on cell phones or laptops. Maybe it would be helpful for the client to email records to the dietitian or researcher so that they can get any questions they may have between sessions answered.
Anna,
I agree with your idea that self-monitoring is more about accountability and enpowerment than perfect calorie counting. Keeping a record is an intentional practice that fosters awareness but I think it also takes practice. Clients might improve their reporting over time as they get into the habit of it or receive further education.
Kathy:
What a great idea with taking pictures of the food you're eating! That would definitely take the guessing of portion sizes out of the equation.
Jordan-
Some people may not even understand how to count calories in the first place or may have a skewed idea of portion sizes. That's why I like your proposal on educating participants on proper portion counting. This would be extremely helpful in ensuring accurate responses for the data.
This can make an impact on the study. Participants could have either slightly exaggerated or even underestimated certain data. However, as many of you have mentioned, it would almost be impossible to find someone to follow around each participant to make sure they are recording values correctly. However, I think that the results of these studies were significant enough to an impact.
2) Self-reporting, as I have learned, can influence either what an individual writes down or even how an individual acts during the time that they are required to report their personal information. This can significantly impact the results of the study. It is difficult to validate the results when the researcher cannot personally obtain concrete evidence, but I did think of one possible way to help validate the self-reported data. I read one research article that concluded that people gave more accurate information when given dietary recalls than when they were required to keep a food journal at home. With this said, maybe if participants were required to verbally self-report information to the researcher instead of writing them down, maybe results would be more valid.
Self-reporting can have a large impact since often invalid information can be given which may alter the study's findings. Possible ways to validate data may be to take participants BMI, body measures, or weight.
Comprehensive education on portion sizes and other techniques would definitely help to correct some of the error in this type of self-reported data as many have mentioned. As funding is almost always an issue, we will continue to have to rely on self-reported data regardless, like many have mentioned as well.
Post a Comment
<< Home