Does WP:MEDRS apply to medical information about individuals?
In a previous discussion on this talk page, some editors concluded that no, WP:MEDRS does not apply to diagnosis (see here); however, this was a brief discussion with few comments and no formal closure. The issue was also raised on the Julian Assange talk page, and different views on the relevance of WP:MEDRS were expressed (see here). In two RfCs on Trump's mental health (RfC 2019 and RfC 2021), the consensus was "Do not bring up for discussion again until an announced formal diagnosis or WP:MEDRS-level sources are provided" [1], implying that WP:MEDRS applies to medical diagnoses. I suspect that this issue has arisen elsewhere (e.g. regarding the athlete Imane Khelif here). The intersection between BLP and MEDRS has far-reaching implications for content (e.g., should we remove from Vladimir Putin the statement In April 2022, tabloid newspaper The Sun reported that based on video footage Putin may have Parkinson's disease?) that are hard to foresee. Gitz (talk) (contribs) 10:52, 27 August 2024 (UTC)
Firstly, to state the obvious, even if WP:MEDRS didn't apply, both WP:RS and WP:BLP would. Which implies that we shouldn't be making statements regarding any diagnosis concerning a living individual without (a) very good sourcing, and (b) a legitimate reason to consider such diagnosis to be of real enduring significance to public life of the individual concerned. In such a context, I'm finding it hard to see how a non-medical source could be used for sourcing in very many circumstances, beyond possibly the subject themselves stating that they had been diagnosed with something. Wikipedia certainly shouldn't be republishing tabloid speculation without good cause, even if framed in text attributing it to said tabloid sources.
As for whether WP:MEDRS should apply, I'm doubtful that it could in practice. Very few medical diagnoses will involve peer-review etc, or be discussed in systemic reviews. Quite possibly what we need is an amendment (or rather clarification) to 'WP:BLP policy, making it clear that speculation about a living individuals' medical status does not belong in articles except in very exceptional circumstances, and that repeating poorly-sourced 'diagnoses' is an unacceptable breach of the requirement to respect the privacy of individuals etc, regardless of how it is attributed or framed. AndyTheGrump (talk) 12:34, 27 August 2024 (UTC)
Yup, and I think common sense applies. The yardstick is whether or not the statement implies anything about biomedicine. Saying Taylor Swift had a cold (say) does not; saying a celebrity was diagnosed with Morgellons would imply that was a real condition, so would. Bon courage (talk) 13:30, 27 August 2024 (UTC)
Agreed. MEDRS does not apply to an individual's diagnosis, but BLP/RS does. In the Trump case you quote, there's an "or" as an announced formal diagnosis in a regular RS would be fine. We should generally strive for the best sourcing available, so MEDRS-level sourcing is to be encouraged generally. Bondegezou (talk) 14:12, 27 August 2024 (UTC)
I agree with all the comments above, but I'd like to understand if there's anything specific to medical diagnoses (e.g. the high level of expertise required to make one) that makes their coverage different from other sensitive areas of a BLP such as, say, sexual orientation. If this is the case, then perhaps a few lines could be added to either WP:MEDRS or WP:BLP to make it clear. Otherwise, the usual WP:BLP + RS apply, including WP:BLPGOSSIP, and if multiple news organisations report, for instance, that according to The Sun Putin may have Parkinson's, we can do the same. If they report that a public figure may have some medical condition (something that does imply questionable biomedical statements: not Morgellons or any other alternative diagnoses/delusional disorders) we can include this. Gitz (talk) (contribs) 15:32, 27 August 2024 (UTC)
IF The Sun, a British tabloid that is deprecated by RSP at WP:THESUN, speculates that a public figure has a medical condition, and this gets repeated by other media companies, then we should probably not include that.
Spend a while thinking about what some celebrity articles (Britney Spears?) would look like if the standard was "One unreliable source said it, and a bunch of media companies decided that they wanted some of that traffic". WhatamIdoing (talk) 17:14, 27 August 2024 (UTC)
I agree that the example from Putin is a bit extreme - we should probably remove that content from the article. But what about articles like Claims of Vladimir Putin's incapacity and death and Age and health concerns about Donald Trump? These subjects are notable and the content is potentially contentious. It would be impossible to write these articles if MEDRS-compliant sources were required. This suggests that WP:MEDRS does not apply to diagnoses of individuals, but only to content that presupposes or explicates biomedical knowledge. It seems that the purpose of WP:MEDRS is not to protect the privacy of living persons, but to ensure that reporting on biomedical topics reflects scientific consensus. Gitz (talk) (contribs) 08:48, 28 August 2024 (UTC)
This "context" is irrelevant: the issue is of general interest and does not concern me or my partial block. I opened this thread because 331dot suggested that if I felt the policy needed clarification (which I do), this should be sought at WT:MEDRS rather than in the unblock request. But this discussion has no direct bearing on my block: even if everyone agreed that WP:MEDRS does not apply to individual diagnoses, my block would still remain in place. Gitz (talk) (contribs) 11:00, 28 August 2024 (UTC) .
Do you just want to provide "further context" or do you also have an opinion on the topic of this thread? I remember you arguing that we need MEDRS-level sources to include a diagnosis in a BLP if it belongs to the GENSEX topic area (or something like that). Gitz (talk) (contribs) 15:29, 28 August 2024 (UTC)
On the Trump RFC outcomes: The community is allowed to set higher rules for an individual article, particularly when repeated discussions are wasting the community's time and wearing on its patience. "Come back when you've got a MEDRS source" is not very different from "One-year moratorium on this subject, because the answer is not changing" or "All future discussions will be subject to WP:EXTCONFIRMED rules, because we've wasted enough time on throw-away accounts". WhatamIdoing (talk) 17:10, 27 August 2024 (UTC)
Yes, I understand. Let me expand on what you've said. In a talk page discussion, when an editor says "we need WP:MEDRS sources for this statement", this might mean either: 1) "WP:MEDRS applies here". Based on our discussion, this is wrong if the statement is about an individual's diagnosis as opposed to general biomedical knowledge. To include content about a diagnosis in a BLP, we need sufficient coverage from generally reliable sources to ensure due weight, but MEDRS-level sources aren't strictly necessary; 2) Or, "Given the contentious and complex nature of this topic, we should require MEDRS-level sources - the usual NEWSORGs are not sufficient". This would be a suggestion, not a policy requirement; it may gain consensus on the talk page, but doesn't reflect the standard use of MEDRS. Gitz (talk) (contribs) 09:46, 29 August 2024 (UTC)
I think there's a danger of over-thinking this. For Wikipedia to say any living person has a serious medical condition there would need to be a proper diagnosis and (in reality) that private information would need to be made public in a decent, reputable WP:RS. Speculation about a condition, or armchair diagnosis – if ever due – would need to come from a MEDRS source of some sort. Bon courage (talk) 12:02, 29 August 2024 (UTC)
I think that if there is a consensus on this approach, then we should write it down somewhere in unambiguous terms, because I keep coming across BLPs that do not follow it. From the article about a recently deceased woman, Tamara Press: Both sisters were accused of being either secretly male or intersex; from the article about her sister Irina Press: Some have suggested that the Press sisters were male or intersex. Another allegation was that they were being injected with male hormones by the Soviet government in order to make them stronger. Sources are the Telegraph, thestraightdope.com, TransGriot, sports-reference.com. Nothing here remotely resembles a "proper diagnosis", which in the case of the Press sisters was never made. At best, we should write something like "The Press sisters retired in 1966, coinciding with the introduction of required gender verification in track & field. This led to widespread, yet never proven, rumors regarding the Presses' genders" (from the last cited source). Gitz (talk) (contribs) 19:20, 1 September 2024 (UTC)
I'm not sure that there is a problem with this (other than the last sentence probably needing to say sex instead of genders). "She was accused of..." is not the same as "She had..." We don't need a medical journal to say that some athletes get accused of having the "wrong" type of body, or that the accusations, even if completely unfounded, can lead to real-world consequences for the accused athlete. In such cases, the article content should focus on the fact that the accusations historically happened, without implying that the accusations are medically true. WhatamIdoing (talk) 20:13, 1 September 2024 (UTC)
Seems like the answer to the question in the title to this section (Does WP:MEDRS apply to medical information about individuals?) is already covered by Wikipedia:Biomedical_information#What_is_not_biomedical_information?. A relevant excerpt of examples of what is not covered by MEDRS:
I had noticed this page, which is indeed quite relevant to this discussion. However, it is only an explanatory essay (This page is not one of Wikipedia's policies or guidelines). So, if I'm not mistaken, there's nothing to stop a local consensus being reached that diagnoses in a particular BLP (because it's particularly controversial, because the diagnosis is difficult, because news organisations don't seem reliable enough...) require MEDRS-level sources. But in general there seems to be no doubt that MEDRS does not apply to individual diagnoses: the medical conditions of a living person can be covered with the usual WP:RS and the usual policies and guidelines, including WP:WIKIVOICE and WP:INTEXT (if the diagnosis is controversial or just "rumours" or "accusations"). Gitz (talk) (contribs) 15:14, 3 September 2024 (UTC)
There's nothing in MEDRS that says BLPs are "biomedical information", either.
I'm not familiar with the backstory here but isn't the problem from the other direction – that is with editors wanting to say that Dave the Doctor has said Paul Politician obviously has dementia or that Alice the Athlete is a man, actually? Bon courage (talk) 16:58, 3 September 2024 (UTC)
Yes, I think that the point is whether WP:MEDRS prevents these editors from making this case, or whether it's just a normal dispute about WP:RS and WP:WEIGHT. Do they need Dave the Doctor to have published his theories in a reputable medical journal, or is it enough that Dave's views have been covered by many news organisations? Gitz (talk) (contribs) 17:12, 3 September 2024 (UTC)
O God that exists eh? An "ideal" MEDRS source is never going to exist for such material but I do think a MEDRS source of some sort is needed for diagnosis of a medical condition in a BLP, a source sufficient to support the weight of the claim made. I suspect the real issue here is AP2/GENSEX and editors wanting to advocate positions, which is something this WikiProject cannot fix. I see we also have Age and health concerns about Joe Biden which delightfully has "No signs of cognitive decline or dementia were noted.[citation needed]". It seems to be attempting to source stuff to the Whitehouse's published presidential physical exam, which would be some sort of MEDRS, FWIW? Bon courage (talk) 03:55, 4 September 2024 (UTC)
Of course it exists. These armchair diagnoses got so much attention in the press during the previous elections that I heard the APA put out a warning to its members about it being unethical to claim you've diagnosed someone who isn't your patient (and illegal to disclose your diagnoses if he is). It would be WP:UNDUE to exclude it, but impossible to source statements like "He's scientifically proven to have _____". Therefore the contents end up being some variation on "Alice claims he has _____", and editors fight over negotiate about which claims are necessary to include and how to phrase them.
A published presidential exam would be a primary source, which is not the MEDRS ideal. Based on the media kerfuffle around one of Trump's published reports allegedly having been written by his (political) staff, we should probably insist upon WP:INTEXT attribution for such sources. WhatamIdoing (talk) 04:00, 4 September 2024 (UTC)
What has prompted this discussoin is editors, Gitz included, wanting to include material in an article that includes speculation about a medical diagnosis in the absense of reliable evidence. TarnishedPathtalk00:12, 4 September 2024 (UTC)
You need a MEDRS-quality source to say, e.g., that human sexual differentiation exists on a spectrum rather than a binary, and therefore at some level we are all intersex. You do not need a MEDRS-quality source to say that Imane Khelif has been accused of having a medical condition that could render her ineligible for competing in women's sports. WhatamIdoing (talk) 01:34, 4 September 2024 (UTC)
I am not clear what you mean, Bon courage, but doesn't WP:OR apply there? We shouldn't be drawing conclusions. If RS have drawn conclusions, we report what RS said, subject to normal BLP rules. Bondegezou (talk) 11:55, 4 September 2024 (UTC)
My assumption was it would be sourced. I have no interest in this particular case (and have not followed it) but, hypothetically, if Doctor Dave says something about biomedical about Athlete Alice (or Paul Politician, or Celebrity Caleb) along the lines of "I can see from observation X that they have condition Y" I don't think that's okay to relay that if it just comes from a lay source, as much for reasons of BLP/NPOV as for weakness of the sourcing. Bon courage (talk) 12:02, 4 September 2024 (UTC)
Exactly, and in the content dispute that has brought about this discussion (Imane Khelif) we have some pundits, some of whom happen to have degrees in adjacent areas and one or two that are subject matter experts, either engaging in commentary (in sources) which are presumptive or speculative about the subjects medical conditions when it would be unethical for them to discuss a patient diagnosis when they are not the treating doctor or illegal if they are. Editors, such as the one who started this discussion, have tried to push the comments of those pundits as evidence that there is reasonable discussion about the subject's diagnosis and not a whole bunch of misinformation. TarnishedPathtalk12:37, 4 September 2024 (UTC)
I don't want to go into the specifics of the case as this is a general discussion, but some background information might be helpful. In 2023 Imane Khelif was disqualified by the International Boxing Federation (a Russian-led, much-criticised sport organisation) because (they claimed) she has XY chromosomes. She waived her appeal against that decision and never released a medical test showing that she has XX chromosomes. In 2024 the International Olympic Committee took the athletes' passports at face value and did not carry out gender testing. So some sports journalists and commentators took it for granted that Khelif has XY chromosomes. Some sources have reported either that she is intersex (Italian press) or that she may be intersex (BBC, New York Times - all major English-speaking NEWSORGs). Even if that were the case, it doesn't mean that she went through male puberty and had a competitive advantage at the Olympics: certain forms of DSDs don't provide any advantage. However, some (more or less WP:BIASED) "experts" ventured a speculative diagnosis of 5-ARD (which would give her a competitive advantage) based on observations such as visual evidence of high levels of testosterone and the like. On the Imane Khelif talk page, there was a broad consensus (myself included) that she should not be described as intersex, not even hypothetically or with attribution ("some have argued that ... she may be"). The point of contention in this RfC is whether to include in the lead that she's been the subject of public debate about her eligibility to compete with women (and also about the soundness of the IOC policy of "stick to the passport, no testing"). In the context of that discussion, I was partially blocked for making this comment. Gitz (talk) (contribs) 13:20, 4 September 2024 (UTC)
Well there you have it. On such multiple-WP:CTOPs, sourcing and behaviour needs to be impeccable or sanctions will likely follow. Personally speculating on a Talk page about somebody's medical condition(s) is really irrelevant to this discussion and not anything WP:MED or MEDRS can influence. FAFO. Bon courage (talk) 13:34, 4 September 2024 (UTC)
Fair enough. This thread, however, is significant if the issue comes up again in talk page discussions, as it's likely to happen. In my view, there is a clear consensus that WP:MEDRS does not apply to individual diagnoses (as per WP:NOTBMI), unless they based on or express controversial biomedical theories. If I were not involved, I would venture to close it with this statement. Gitz (talk) (contribs) 13:43, 4 September 2024 (UTC)
a) A reliable sources don't say she waived her appeal. A reliable sources say she couldn't continue due to not having the funds.
b) The onus isn't on her to disprove the IBA which is unreliable and discredited.
c) Italian sources posting that she is intersex are obviously engaging in disinformation on the basis of nationalism.
d) any expert who isn't her doctor speculating about medical diagnosis is engaging in unethical behaviour. They can't claim high levels of testosterone when they are not her doctor and there are is no reliable evidence.
e) the point of the RFC is to insert language into the article which paints the picture that there has been legitimate concerns, when there has not. Legitimate concerns are based of reliable evidence/facts and there is an abundant lack of those. TarnishedPathtalk01:17, 5 September 2024 (UTC)
A reliable sources don't say she waived her appeal. Times of Israel, "Khelif initially appealed but then withdrew her motion"; Forbes, "Khelif later withdrew her appeal"; BBC, "Khelif did take her case to the Court of Arbitration for Sport (Cas), but then withdrew the appeal"; 3 Wire Sports, "Both athletes were afforded the chance to appeal the DQs to the Swiss-based Court of Appeal. Lin did not. Khelif did but then opted last July not to pursue the matter"; France 24: "Khelif did take her case to the Court of Arbitration for Sport (CAS) but then withdrew the appeal". And so on...
As for your other comments: the athlete's body has always been a subject of public debate. Ideals of beauty and physique, performance enhancement and doping, gender and sexuality, body image and eating disorders... you name it. It's part and parcel of being a professional athlete, it's not a matter of The onus isn't on her to disprove ... any expert who isn't her doctor speculating about medical diagnosis is engaging in unethical behaviour ... Legitimate concerns are based of reliable evidence/facts. Your questionable reading of BLP+MEDRS+GENSEX denies and trivialises a significant public debate in sport, preventing its coverage based on RS. Gitz (talk) (contribs) 02:01, 5 September 2024 (UTC)
I am partially blocked, not topic banned. Anyway, you're right - there's no point in continuing to discuss Khelif here. The issue raised by my original post also seems to have been sufficiently clarified. Gitz (talk) (contribs) 09:33, 5 September 2024 (UTC)
From the Sydney Morning Herald "Three months later, CAS issued a termination order because Khelif could not fund the costs of the matter".
@Bon courage, I think I disagree with you about hypothetically, if Doctor Dave says something about biomedical about Athlete Alice (or Paul Politician, or Celebrity Caleb) along the lines of "I can see from observation X that they have condition Y" I don't think that's okay to relay that if it just comes from a lay source. To illustrate why, I'm going to give you my own totally non-RS diagnosis about a BLP: Bryce Dallas Howard has a significantly elevated risk of skin cancer due to her genetics.
How can I, a lay person, make such a strong medical claim about a person I've never met, much less examined, and actually don't know anything about except that her name turns up in lists of actresses with naturally red hair and green eyes? It's very simple: Every single human with naturally red hair and green eyes has a significantly elevated risk of skin cancer.
We wouldn't want to put this in an article about her (unless, e.g., she announced that she had skin cancer, or got involved in skin cancer prevention advertisements), but that's because it'd be UNDUE, not because it's untrue. But this is such a lightweight claim that any barely passable source should be considered sufficient.
(Why did I pick that as an example? Because a few years back, the California legislature has decided that students should not be exposed to their own genetic information as part of a class, and a professor gave that as an example of an unintended consequence.)
I don't think that all medical claims need the same level of sourcing. Sometimes a person's appearance alone is sufficient; often it's not. And some conditions are more stigmatized, which, as you indicated, has BLP and DUE implications. WhatamIdoing (talk) 06:56, 9 September 2024 (UTC)
I think the decision needs to be multi-factorial. "Obviousness" is one factor (e.g., "He has a bad sunburn"). Part of the difficulty here is that what's obvious to "me" isn't obvious to everyone.
Self-disclosure should be another. People do sometimes lie about their health, and misdiagnoses happen, so a press release saying that Chris Celebrity can't eat gluten should never be taken as the final word, but generally you assume that a qualified healthcare professional was involved.
Did you ever work on the Terry Schiavo article? I remember a physician-turned-politician saying that he was convinced she was conscious on the basis of a very brief video clip. It turned out that the video clip was carefully chosen because if you saw those few seconds alone, she looked like she was watching a balloon move. (The autopsy report said she was totally blind as a result of brain damage.) So I think you also want some judgement call about credibility. Someone looking to score political points is less credible than someone who isn't. Someone with relevant expertise is more credible than ordinary people. Someone with more information is more credible than someone less information. Someone whose voice was elevated by a high-quality publisher/source is more credible than someone whose claim only got published because a junky source thought it'd drive traffic.
And then sometimes we have to throw this all away and say: So much has been written about speculations on Donald Trump's mental and physical health that the speculation should be covered one way or the other. The question there is not whether the speculation is true (e.g., "Is he really a narcissist?"), but whether the speculation is impactful (e.g., "Did the claim that he's supposedly a narcissist have any effect on whether people are likely to vote for him?"). WhatamIdoing (talk) 22:03, 9 September 2024 (UTC)
Yes, there are people who have been 'cured' of cancer but where the diagnosis was questionable in the first place. I don't know about Terry Schiavo. I agree sources should always be credible. Bon courage (talk) 05:34, 10 September 2024 (UTC)
I agree with what others are saying. I'm not sure that whoever said a MEDRS is required for saying an individual has a medical condition has really thought it through. We have other guidelines that deal with speculative, negative information about individuals. We likely have countless biographical articles that note people got cancer or had epilepsy or died of a stroke that are sourced to newspapers, and those aren't MEDRS. Contentious extraordinary claims require high quality sources. Wrt historical figures, even medical journals can be prone to armchair diagnoses of dubious quality. -- Colin°Talk18:30, 3 September 2024 (UTC)
We have a lot of articles that cite newspapers or obits to say "Alice died at the age of 67 from cancer", which are entirely acceptable and not MEDRS' ideal. See Angelina Jolie#Cancer prevention treatment for an WP:FA that discusses one individual's personal medical information without citing a single medical journal, reference book, or anything similar. Instead, it relies almost entirely on news stories (or worse). WhatamIdoing (talk) 01:38, 4 September 2024 (UTC)
I don't think anybody's arguing (are they?) that MEDRS is needed to relay the knowledge that somebody has a medical condition. What's at stake is the sourcing required to make the determination that somebody has (or might have) such a condition. Right so; to repeat: For Wikipedia to say any living person has a serious medical condition there would need to be a proper diagnosis and (in reality) that private information would need to be made public in a decent, reputable WP:RS. Speculation about a condition, or armchair diagnosis – if ever due – would need to come from a MEDRS source of some sort. I don't think any change of WP:PAG is necessary. Bon courage (talk) 01:31, 5 September 2024 (UTC)
If I've understood the distinction you're drawing, then we need:
Pretty much any reliable source to say that Chris Celebrity has been diagnosed (e.g., by their own doctor) with scaryitis. See, e.g., Charles III#Health.
MEDRS-level article to say that, according to people who have never examined them, Chris Celebrity appears to have scaryitis. See, e.g., Retrospective diagnosis#Examples.
And (perhaps? are we agreed?) multiple decent sources (e.g., news and magazines) to say that Chris Celebrity has been the subject of speculation about health conditions, but these should not say that the speculations are either true or definitely false. See, e.g., basically all of Age and health concerns about Donald Trump.
Yeah something like that. In the second case obviously a strong MEDRS isn't needed to say (e.g.) somebody 'sounded hoarse' while singing; but to say their gait was characteristic of a neurological condition would need something with some MEDRS chops. Bon courage (talk) 07:33, 9 September 2024 (UTC)
For that second category, I'd expect "some MEDRS chops" to be generally on the lower side (perhaps a peer-reviewed primary source), and I think we need to use sensible judgment, so that things that are more common and obvious don't require much, but wild speculation requires stronger sources. WhatamIdoing (talk) 07:41, 9 September 2024 (UTC)
I think Chris's surname is unhelpful. None of the people at Retrospective diagnosis#Examples are "celebrities" and I would hope medical journals would have better things to do than speculate on the Kardashians or the latest Love Island contestants. The Marfan's syndrome source is sadly typical of superficially "MEDRS" sources, where a famous name is dropped in the introduction to a more serious article that isn't about them at all. Presumably journal editors/reviewers permit this kind of speculation as "harmless" and don't stop to think "but some Wikipedian might cite this".
I'm not sure there is good evidence that for historical figures, supposedly MEDRS sources are any better than a serious history book. Possibly a serious historian has dug up detailed descriptions of their illness and described them to a medical expert like any scholar might about any fact outside of their expertise. So some Oxford University Press history book could well diagnose an illness or cause of death and I actually think historians would be more professionally motivated to get their history right vs some random consultant neurologist attempting an entertaining introduction to their latest review. -- Colin°Talk14:29, 10 September 2024 (UTC)
Health of Abraham Lincoln lists several possible diagnoses. We would presumably accept a med school textbook saying that he might have had Marfan's (or, perhaps more likely, depression), and perhaps a similarly serious work of history would also be appropriate. I think it's worth thinking over, but right now, I'm having trouble explaining why a scholarly history book feels okay, but the "peer-reviewed literary analysis" mentioned at Talk:Cass Review concerns me. Perhaps the latter is closer to the concept of using a primary source to debunk a secondary one. I'm still thinking about it. WhatamIdoing (talk) 03:21, 15 September 2024 (UTC)
Maybe the thing to think about is what would the peer reviewers (and editors and publishers) be concerning themselves with? A neurological paper, outlining the state of understanding of a condition and its treatment, is concerned with whether that is an accurate and comprehensive medical review, not whether the introduction correctly identifies some long dead historical figure. Whereas presumably a university press history book is concerned with getting history right.
A literary analysis would seem to be utterly unconcerned with truth, facts, or indeed the real world, but seeks to view a piece of writing under various conceptual frameworks and thought processes. It seems that two literary analysis papers could produce entirely contradictory conclusions about a work, and that would be just fine as long as the analysis is self consistent. It seems to me that using it to say something about the real world is as much a conceptual error as using an academic theology paper about some Hindu script's story about a god, is evidence that this god does or does not exist. -- Colin°Talk18:34, 15 September 2024 (UTC)
In policy terms, I think we'd call that "appropriate", as in Use sources that directly support the material presented in an article and are appropriate to the claims made. The appropriateness of any source depends on the context. A gold-plated scientific source is not appropriate for information about movie ticket sales, and a source that's appropriate for box office success is not appropriate for scientific claims. WhatamIdoing (talk) 03:07, 16 September 2024 (UTC)
That's probably too vague a word to be useful. I think the problem we had with that literary analysis was that even non-literary-analysis claims (like factual claims, some of which were inaccurate) were held to be true because it was published in a scientific journal and had been peer reviewed. Elsewhere we have seen what I would regard as opinion pieces but not always labelled as such, because they are more some kind of intellectual academic thought piece. Again, there's a misconception that the author's thoughts are thus held to be factual rather than "Hmm, that's an interesting and coherent argument you made". Understanding these different kinds of academic works, and their limitations, is outside of my education. I don't know who might know better. But I think it might be useful if wiki had some kind of guideline on the different sorts of academic works, and what peer review means for each of them, to help people know what is "appropriate". -- Colin°Talk07:22, 16 September 2024 (UTC)
Another area where this sort of question comes up is around Lucy Letby, the nurse who was twice found guilty of murder/attempted murder. There has been considerable media discussion about the safety of those convictions, often revolving around expert issues of medicine and statistics. How do we ensure good sourcing? There's an ongoing discussion at Talk:Lucy_Letby#Determination_of_WP:UNDUE_should_be_per_RS and input would be welcome. I feel parts of the article spend too much time on discussing individual commentaries by experts that are getting close to WP:PRIMARY, for example. Bondegezou (talk) 11:27, 8 September 2024 (UTC)
Well I'm not going there, but I'd have thought Wikipedia should stick to a summary: i.e. that's she's been convicted but doubts have been raised about the safety of that conviction, without going into the weeds about breathing tubes, statistical analysis and so on. Bon courage (talk) 12:27, 8 September 2024 (UTC)
NOTNEWS is opposed to Wikipedia creating original news reports. It's got nothing against citing the news or keeping articles up to date. I think this is more of a WP:VNOT situation: Just because you can find a source that quotes someone who says ____ is/isn't evidence of guilt/innocence doesn't mean that an encyclopedia article needs to include it. We need more bottom-line summaries ("She was convicted, though some experts have doubts") and less blow-by-blow detail ("Well, Dr. Expert says that discoloration alone isn't proof of air embolism, but Dr. Authority said that discoloration wasn't considered alone"). WhatamIdoing (talk) 07:33, 9 September 2024 (UTC)
Per Bon courage, yes it has got something against citing the news or keeping the articles up to date. It has multiple statements to the effect of "Wikipedia doesn't want everything you read about in the news" in it. But as Bon courage notes, editor enthusiasm is hard to contain while it is an ongoing story. And for an encyclopaedic POV, it is awful because the deluge of stories is based around editorial policy/position rather than what the current best sources say. We are at the mercy of whether some newspaper editor wants to run with the Evil Nurse angle or the Miscarriage of Justice / Cover Up angle. Which angle sells more papers or suits our political agenda? -- 14:03, 10 September 2024 (UTC) Colin°Talk14:03, 10 September 2024 (UTC)
Discussion on the relative reliability of medical sources in regard to Pornography Addiction
Hello, I'm an instructor guiding students in creating Wikipedia articles on medical topics. We use the list of requested articles for medicine as a starting point. I am now planning the semester 2 syllabus and would like to establish best practice around 2 areas: ...
...regarding NICE CKS sourcing in this edit (self-reverted as a scruple), which I feel helps provide key introductory information to [[Osteoporosis#Epidemiology]] succinctly and really quite conveniently. CKS was briefly discussed here back in 2014 (I was actually the OP then), and I believe the consensus then that it was permissable to use CKS, even though it is not accessible outside the UK. Now, CKS comes with a scary licence agreement, which states [my italics]:
2.1 You agree that you are only allowed to Use the Topics if you:
(i) Are an individual; in which case:
You are allowed to Use for personal and/or your own educational purposes only and not on behalf of or for the benefit of any company, organisation, or business.
...
2.2 If you are not an individual or a clinician (as defined above), you must contact Us for a commercial licence. If you do not, you understand that by Using the Topics you or your organisation will be infringing Our intellectual property rights.
2.3 For the avoidance of doubt, the following, without limitation, are not permitted to Use the Topics (but may do so by contacting Agilio and entering into a commercial licence): ...
(iii) companies, businesses, and any other private enterprises that are not part of the National Health Service.
I doubt that it's a problem. See 3.1(v): "You cannot Use the Topics to create other material, such as books, articles, or guidance. This does not prevent you from referring to appropriately referenced extracts of Topics." Citing it as a source behind text that you have written in your own words presumably counts as "appropriate referencing". WhatamIdoing (talk) 17:58, 29 October 2024 (UTC)
Thank you for that WAID. It would seem crazy to provide reliable medical information that can't even be cited, but hey what do I know? I'll restore the edit given that this is a really useful medrs, imo (I've sometimes found it tricky to find a good medrs that summarizes key basic info in a readily citable form). 86.174.206.40 (talk) 18:57, 29 October 2024 (UTC)
I suggest changing "medicine-articles" to "medical articles" - what do you think?
The following sentence appears in the first paragraph of this content guideline (WP:MEDRS):
Sourcing for all other types of content – including non-medical information in medicine-articles – is covered by the general guideline on identifying reliable sources.
I want to solicit others' opinions, but at this point I believe medicine-articles should be changed to medical articles because, unless I'm missing something, medicine-articles is incorrect usage.
I also asked for feedback over at the Guild of Copy Editors talk page, where I explain in more detail problems such as lack of parallel structure and making readers work harder than necessary "when a noun is used adjectivally in place of the more usual adjective." [Bryan A. Garner, Garner's Modern English Usage (4th ed. 2016) at 416–417.]
Definitely no hyphen. "Medical articles" is fine. The only other alternative is "medicine-related articles", but why make it three words rather than two? Tony(talk)10:08, 5 January 2025 (UTC)
Years ago, the text previously said "sourcing for all other types of content – including non-medical information in medicine-related articles – is covered by...". Since "medical" means "relating to the science or practice of medicine" then I guess that's a fine replacement for the clunky "medicine-related" and the incorrect "medicine". -- Colin°Talk19:29, 5 January 2025 (UTC)
Much appreciated Colin. ¶ This is a great example of Wikipedia at its best. A moderately experienced Wikipedian (me) posts a suggestion, and within two days very experienced Wikipedians efficiently discuss and promptly make an agreed-upon edit. Little things like this keep me coming back to contribute as I can and outweigh the discouragement I sometimes feel when encountering internecine conflict insigated by a minority of problematic editors. -- Mark D Worthen PsyD(talk)[he/him]11:20, 8 January 2025 (UTC)
Improving the "referencing a guideline" illustration
Check the quality range illustration used in the WP:MEDORG section.
CFCF posted this in 2016. I do not think it has been discussed anywhere.
I question that it puts "Health technology assessments" as the highest quality guideline. I live in the United States and am unfamiliar with this concept. I know this concept is not used much in Wikipedia. At Talk:Health technology assessment I asked for an example of one of these assessments.
I propose deprecating or updating this image. If we were to re-make it or present this guide in any other form, I support keeping everything else in place and just removing HTA.
I might also switch "national guidelines" with "professional societies", because in the past 10 years or so, multiple major governments have recommended health policies which conflict with medical professional society recommendations. My feeling is that when this happens, Wikipedia editors have prioritized expert physician statements over political statements when they differ. Examples of areas where politicians and medical professional societies take differing positions include on health issues for infectious disease like COVID, labor rights and occupational health and safety, LGBT+ health issues, health effects related to climate change, and access to healthcare in poverty conditions when patients cannot pay for recommended care.
It is fairly common for guidelines like these to be established without completely connecting to the WP:MEDRS standard of evidence, which is a meta analysis published in a peer reviewed journal. Often guidelines either interpret a study beyond what the paper about a study actually says, or they may not clearly even connect to research at all. They are still consensus statements from groups of experts at authoritative institutions, and pass WP:MEDORG, but they are not what usually comes into Wikipedia.
This image is Wikipedia's editorial policy and not much discussed. When something is proposed without evidence or discussion, then we can update and change it just as casually, and I think that is the situation we have here.
@Boghog: Thanks, I know AHRQ. If I am reading that page correctly, they have done one assessment since 2020, and it is about one particular treatment for sleep apnea. They have a list of reports between 1990-2020, and by line count there are 160 of those reports. These may be good reports but given that there are so few of these, I question whether we should recommend these as the best kind of report from a medical or scientific organization, which is what that image is doing.
This section MEDORG is supposed to be about when we accept medical info from outside the usual scholarly review articles. Right now it is framed for what kinds of organizations make claims we can accept. Could there be another dimension for what kinds of documents they produce, like guidelines, health technology assessments, and any other claims? Do you have any insight into what other categories of high-quality, non-scholarly-journal, expert publications exist besides these two? Bluerasberry (talk)22:15, 11 January 2025 (UTC)
Health_technology_assessment#By_country lists agencies outside the U.S. that do these types of assessments.[1][2] Since this is the English language Wikipedia, the most likely to be used sources are:
Let me take some time to think about all this. I am still skeptical. I clicked on a few of these, and in the ones I checked, I still see the same thing - major national organizations typically publish fewer than 1 of these per year.
One interesting one that I found was an HTA on HTAs, and it even has a CC license, so maybe I can archive that in Commons and share it in the wiki article. Thanks for the great reactin, will get back. Bluerasberry (talk)14:28, 12 January 2025 (UTC)
Bluerasberry - I think a few points are relevant to mention here.
1) A lack of volume of HTAs doesn't really detract from stating that they are generally the most reliable. Inherent to the HTA process is conducting meta-analysis, systematic review, and combining this with ethical and often economic perspectives using a standardized protocol. It is inherently time-consuming, and there will therefore not be as many as other source-types.
2) The fact that AHRQ in particular publishes so few HTAs is because funding has been successively cut as their findings were not appreciated by certain congressional groups. This has happened under multiple administrations, both because of a personal spat that a congressman had with one of the reports in the 1990s/2000s, and then again in the late 2010s with some health care lobby groups. This says absolutely nothing about the quality.
3) I agree that the diagram is somewhat simplistic, but this should be elaborated upon in text. The fact is that not all HTAs, like any source have the same quality. Something by NICE and something by a regional HTA body will not be the same, just like every review isn't the same. This doesn't detract from how the methodology is generally the best.
There is also the University of York - Centre for Reviews and Dissemination - HTA Database[2], which allows you to search for HTAs.
That website points users to https://database.inahta.org/ It looks like about a thousand HTAs are listed for each year recently, with 23,000 total listed. At a glance, they are mostly not on hot-button politicized topics. For example, searching for transgender finds only one (on phalloplasty), and the MeSH term gender dysphoria finds only three (from UPenn, Sweden, and Spain).[3] There are 20 on cannabis use.[4] There are only 13 on abortion (nine of which are not in English),[5] and another 13 on miscarriage.[6] There are none on assisted suicide and only one on euthanasia.
The issues with AHRQ haven't been on topics that are controversial among the general population, rather specific findings which call into question either lucrative treatment practices, or treatments that are near-to-heart for specific congressional delegates due to personal views (such as on back pain). This NY Times article is a decent overview, if not very in-depth [8].
If anything the concept is thoroughly "unsexy" and uninteresting for the general public - which means that cuts can be made without much concern. The issue has been ongoing for quite some time [9]
Also, you are right the INAHTA database has replace the York University one (even though the latter still works to some degree). CFCF (talk) 13:06, 23 January 2025 (UTC)
P.S.WhatamIdoing - it seems that MeSH-search doesn't work so well in the database, and that those two were not registered as HTAs because they intentionally left out ethical and economic aspects - so they are just, if you will, high quality systematic reviews. CFCF (talk) 21:58, 23 January 2025 (UTC)
Before I get too involved in the details here - let me be clear also in this discussion that I have worked with and for SBU, with funding from Wikimedia Sweden as well as being a Wikipedian in Residence. (This was formerly very clear on my userpage, before my hiatus - and nothing I have made attempts to hide. I have now reinstated this disclosure).
Adapting the guideline to be clearer would seem a good idea. In general I think the use of visual aids, including the multitude that I made years ago - remain okay. One can discuss rearranging the order - but as a rule I think it holds. As for the comment by you Bluerasberry, that both national and international guideliens have been shown fallible through the pandemic, yes I agree. However, I think any change needs to be very carefully thought through. To some extent content by national expert agencies remains a cornerstone of Wikipedia - and most material is still very high quality. With regard to HTAs, I really don't see these as impacted by any controversy from the past years. In essence the strength of the HTA process lies in the method (e.g. SBU method). As long as we are also using the pyramids for hierarchies of evidence, and clarify in text that all of that is just a general guide - I don't see why we would remove this specific visual aid from MEDRS either.
In general I find HTAs tend to fly under the radar from the general public, because the questions they raise are often very technical, somewhat narrow, and mostly of value for policy-makers and health financing groups.
As a first point of clarity, I would probably suggest highlighting the rise in the past 5+ years, of commercial HTAs (such as legit vendors as far as I can see such as IQVIA and perhaps Clarivate, but also some commercial actors stating that they do HTAs without a clear process). Mostly this seems benign, with the bigger issue being that these can be very hard to access, costing in the range of 5-50.000 USD to read. I don't really think this poses a problem for Wikipedia, but it could potentially in the future if misuse of the HTA-term becomes an issue.
P.S. A terminological point - this is the EU definition:
(1) [...] Health technologies encompass medicinal products, medical devices, in vitro diagnostic medical devices and medical procedures, as well as measures for disease prevention, diagnosis or treatment. (2) Health technology assessment (HTA) is a scientific evidence-based process that allows competent authorities to determine the relative effectiveness of new or existing health technologies. HTA focuses specifically on the added value of a health technology in comparison with other new or existing health technologies.
The point to be made here is that technology in this sense is a very broad term which should include pretty much any treatment. CFCF (talk) 21:54, 23 January 2025 (UTC)
CFCF, I've heard several editors express concerns recently that are less about "controversy from the past years" and more about "the NIH and CDC during the next four years", though for editors specifically working in trans-related topics, the Cass Review seems to have caused a lot of pain and fear last year. WhatamIdoing (talk) 00:06, 24 January 2025 (UTC)
I've changed the indenting for clarity here - and intend to respond. This is an intricate question, and I will get to it. CFCF (talk) 07:17, 24 January 2025 (UTC)
Fundamentally, you raise important questions, and I’ve spent some time thinking about it. Recent developments—specifically the unprecedented gag order on federal agencies—are noteworthy and may signal a need for adressing the issue in the future. However, we should be cautious about reacting too quickly. This particular restriction is significant but also fairly limited in scope, and it is still too early to understand the broader effects of what follows.
If I may elaborate on a few points, especially regarding why I view this as potentially relevant for MEDRS more broadly but less so for HTAs: By definition, these assessments aim to be apolitical and methodologically robust. Their credibility rests on the quality of the systematic review and other established processes involved. Unless there is evidence of compromised methodology, I see no immediate need to reassess their reliability. Of course, if anything changes substantially, we can address it then.
With regard to the Cass Review: Although I am not intimately familiar with it, the very existence of an article dedicated to it suggests two things: (1) it was influential, and (2) it was controversial. Both factors are indicated by how editors found sufficient motivation to produce a detailed entry. At a glance, it seems appropriately discussed at Puberty blockers, where its findings—and the criticisms—are laid out.
This brings me to what I view as the core duty—of Wikipedia: to present the best possible sources in an unbiased way. The current coverage of the Cass Review appears to meet that standard, detailing both the findings and criticisms. Whether we agree with its findings remains slightly beside the point, and steers close to criticism of WP:NPOV - which I disagree with.
Regarding concerns about political interference in expert agencies like the CDC or NIH: I remain cautiously optimistic about the independence of these institutions, given their long administrative traditions. While budget cuts could certainly constrain the scope of their work (as happened with AHRQ), there is little indication that their existing outputs would be twisted to say something other than what evidence-based methodologies would conclude. A more likely chilling effect, if it does occur, might be the decision not to investigate certain issues in the first place.
On a more general note, not all government outputs are created equal. In the U.S., for instance, there is a distinction between: Politically shaped outputs, such as Congressional Oversight and Investigations reports (often produced by offices staffed by political appointees), and Technocratic or expert-driven outputs, such as reports by the Congressional Research Service or by independent agencies.
A similar setup exists in many other countries. Sweden’s Government Public Inquiries (SOU) come in different flavors: some are penned by political appointees, while others come from state agencies and are less likely to be politically slanted. The UK also has a multi-layered system involving Quangos, agency reports, commissions, etc.
Another point, getting back to you Bluerasberry - NIHR and SBU are both partners with PCORI - so there is interest among HTA bodies for patient-centered outcomes through something called EViR (The Ensuring Value in Research Funders’ Collaboration and Development Forum).
I don't think this should change the fact that patient organizations are below the cut-off for MEDRS - rather I wonder whether generic descriptions of patient-centered-outcomes actually constitutes biomedical statements, and whether MEDRS at all needs to apply unless there is a statistical/treatment statement (or similar). Just stating that people find a specific outcome relevant doesn't need MEDRS in my book. I don't see that question being raised in Wikipedia:Reliable_sources/Noticeboard/Archive_464#Patient-Centered_Outcomes_Research_Institute_(PCORI).
With regard to the specific statement about incidence of side-effects there - I do find that the it probably needs a MEDRS source. I am on the wall as to whether PCORI should be authoritative editorial board or patient-interest-group. However, I think we are right to be very sceptical of patient-interest-groups in general when it comes to statistics or treatment. Not least from personal experience in research and government work. The groups are often very well intentioned, but lack statistical and medical knowledge, as well as often being unable to understand priorities and crowding-out effects, and sometimes not even questioning whether their statements make much sense. (For example, I had to stop one patient-interest group from stating in a summary of my paper that 50% of the sample had poor outcomes, because we had binarized the outcome variable to use it in a specific regression model and had set 50% as worse and 50% as better, by definition. They had missed that their statement, which they wanted to relay to a news agency really read "The half of the sample with worse outcomes, made up 50% of the sample."). Of course there are better and worse organizations, but as a general rule - I think that for MEDRS, the heuristic still holds. CFCF (talk) 19:00, 25 January 2025 (UTC)
Had one more thought - maybe you're right that PCORI is a research group - and this is where the distinction lies. I also got thinking about CRUK, which I think is authoritative whereas some professional bodies are less reliable, such as the U.S. ME/CFS Clinician Coalition. It's not straightforward... that is without even getting into psuedo-professional bodies, that only say they are made up of professionals, but really aren't. And then there are organizations that are professional interest groups in fields they don't work in, such as International Physicians for the Prevention of Nuclear War (an admirable cause, but likely not MEDRS). CFCF (talk) 19:11, 25 January 2025 (UTC)
Have you considered writing a Wikipedia:Review articles page? Or Wikipedia:Types of medical sources or something like that? When and how to use or prefer different kinds of reviews (e.g., Umbrella review and Scoping review) might be a useful thing to write down. And along those lines, perhaps another page for various health-related government reports. A list of some key names to know might be useful.
About patient outcomes, I have some concerns about our autism content (which, overall, is IMO pretty bad), as it can be difficult for editors, especially editors who happen to hold a particular POV, to differentiate between "patients want this" and "this advocacy group, which promotes a particular POV, published the results of an unvalidated, non-random survey that said their biased and self-selected sample wants this". WhatamIdoing (talk) 00:58, 30 January 2025 (UTC)
I am quite pressed for time, I think I might consider it on the tacit assumption that it would be linked here. But it really doesn't detract from the need to update parts of this page. CFCF (talk) 23:16, 6 February 2025 (UTC)
Advocacy groups run the spectrum from nonsense to political activism to purely mainstream conventional medicine. It is a complex landscape. WhatamIdoing (talk) 04:26, 14 February 2025 (UTC)
NIH
tatements and information from reputable major medical and scientific bodies may be valuable encyclopedic sources. These bodies include the U.S. National Academies (including the National Academy of Medicine and the National Academy of Sciences), the British National Health Service, the U.S. National Institutes of Health and Centers for Disease Control and Prevention, and the World Health Organization.
No organization is infallible, and WP:DUE is still a core content policy. So far, the effect of the Trump administration has mainly been to remove information, rather than to put up untrustworthy information. But if/when that becomes a problem, we can address it at that point in time.
As for funding criteria: The US funds a disproportionately large share of the world's medical research.[10] Consequently, changes here could have really significant effects across the world. However, given how long it takes for a drug to get from bench to bedside, the results might not be seen for another decade. WhatamIdoing (talk) 22:05, 13 March 2025 (UTC)
So you aren't left on read and since answering this reiterates what I am saying in the other messages I wrote on this page:
I don't know. I have read a lot of medical studies, and just a lot of studies in general, and just a lot of everything and there are a lot of mostly trustworthy sources that look sketchy and a lot of terrible sources with mostly biased takes or blatant lies that have a shiny professional appearing website and some, of both groups, and every and any other possible configuration of what might or might not look to be a source you can trust also have credentials that reinforce that you should trust them and if this is hard to follow the point is even with credentials and money and websites and friends with credentials in high places and majority of public support/belief you still have to rely on logic and critical thinking. No human is infallible. None. Nobody. All humans don't like to admit they were wrong especially in highly visible or expensive or otherwise notable events.
Long story short is if you don't know if that source has information that checks out you probably should either not be editing pages where you need to cite that source (something I too am guilty of, no offense) or, what you really actually probably should do in that case is . . . read some more and make sure everything checks out.
This may not be surprising to you, but it was somewhat to me, and that is that for all of the endless words about bad information coming from "mainstream media", "social media", or as I stated in "professional publications" - which is absolutely true - Wikipedia is smack dab in the middle. It's amazing things work as well as they do Relevantusername2020 (talk) 14:22, 19 April 2025 (UTC)
WP:MEDRS is over-conservative and needs changing
Wiki medical articles would be better if they could include content sourced from published, peer-reviewed research.
Insisting on review sources only leads to over-safe articles that
(a) are less valuable, up-to-date and interesting than they could be (readers are adults), and thus
(b) positions Wikipedia weakly in the coming survival battle vs AI (e.g. Grok DeepSearch).
A middle way would be to include content sourced only from published content with a 'B' grade quality label, with content based on reviews labelled as 'A' grade quality.
I'm not commenting upon the rest, since I'm under editing restrictions, but using LLMs for medical information should be considered suicidal. tgeorgescu (talk) 00:39, 21 March 2025 (UTC)
The problem is that primary research does not reliably reflect the "accepted knowledge" Wikipedia should be summarising, and a good proportion of it is wrong or fraudulent, See WP:WHYMEDRS for more. Bon courage (talk) 02:36, 21 March 2025 (UTC)
A better middle ground in my opinion is to include important but unverified/primary content only in Research or similar sections and also with proper attribution (e.g. "A study showed ..."). Bendegúz Ács (talk) 12:35, 19 April 2025 (UTC)
It would be "claimed" rather than "showed". Allowing this would open the door to a boatload of fraud and quackery (as well as just bad science). If something is "important", then that becomes apparent via secondary WP:MEDRS sourcing. Bon courage (talk) 12:48, 19 April 2025 (UTC)
fraud and quackery (as well as just bad science)
Obviously, these should still be filtered out but my impression has been that there is a lot of actually valuable research that cannot be reported on Wikipedia because there is only one study about the particular question and it is also unlikely to be ever repeated or reported in a meta-analysis or similar review article. Here are two examples: [11], [12].
If something is "important", then that becomes apparent via secondary WP:MEDRS sourcing
"Important" is relative here because as I mentioned above, some studies are important to gain insight into particular questions, but not important enough to be repeated just for replication. And even if they are, it could take a really long time so I feel like currently Wikipedia remains very outdated in many cases (which I think is the main criticism in this post). Bendegúz Ács (talk) 13:08, 19 April 2025 (UTC)
The problem is of course that "feeling outdated" is better than carrying statements that MMR vaccines cause autism or that ivermectin prevents COVID-19, as would have happened if MEDRS didn't enforce high standards. The purpose of this Project is only to be up-to-date with respect to accepted knowledge, and not to be cutting-edge at all. Bon courage (talk) 14:40, 19 April 2025 (UTC)
I do understand that the main point of having this policy is to prevent such content. And I agree that it serves that purpose very well. In any case, the current wording does not fully exclude what I described in my previous reply here, as it says: "If conclusions are worth mentioning (such as large randomized clinical trials with surprising results), they should be described appropriately as from a single study:".
What I feel is somewhat missing here is a distinction between heavily researched areas and more niche topics. An image caption in the policy says "A lightweight source may be acceptable for a lightweight claim, but never for an extraordinary claim.", perhaps this idea could be included somehow in the part discussing primary vs secondary sources too?
In many cases, Wikipedia is cutting-edge or at least up-to-date already though, for example, newly approved drugs. This policy does not really go into this, but the practice I've seen for those is to report the results of the phase III trial very precisely and not just a conclusion based on review sources ([13] is such an example).
Now of course the fact that it has been approved allows for this content, but then the question is could we somehow bring content about health effects of things other than pharmaceutical drugs closer to this up-to-dateness of approved drugs? Bendegúz Ács (talk) 11:33, 20 April 2025 (UTC)
It would open the very gates of Hell wrt fringe medical content and POV-pushing. Honestly, when the quality of medical content is one of the most conspicuous successes of this Project I fail to see why the foundations for that success come under such frequent attack. If people want to see what the current research on a topic is, they can jolly well use a search engine. Bon courage (talk) 11:39, 20 April 2025 (UTC)
I am not attacking it, I am just trying to further improve this (already great!) policy. Perhaps my first idea for an improvement would not even be about making it less conservative/more up-to-date but for it to allow showing research results and proofs more precisely, because in general, I think that's how you can convince people, rather than just declaring something to be true without showing the proof.
And of course, using a search engine or an LLM is an option, but isn't that true for every other content type as well? For many types of content, I turn to Wikipedia rather than those, and that's partially because it is up-to-date. Bendegúz Ács (talk) 12:05, 20 April 2025 (UTC)
No, Wikipedia is meant to deal in knowledge, i.e. by diligently selecting only the best sources and summarising what they say. Search engines don't do this and LLMs make a poor job of it usually. We don't show results and proofs in detail because of this need for summary for a lay audience without dwelling on the minutiae of sources: WP:MEDSAY. Bon courage (talk) 05:35, 21 April 2025 (UTC)
Yes, they are inferior usually, but then my point is, how can one get a reliable summary of cutting-edge research? I think Wikipedia could fulfill that need while still not losing the high quality of medical content in general.
I agree that dwelling on the minutiae of sources is not useful in general, but I think presenting the concrete results is. So for example, the recommended text in WP:MEDSAY could be extended like this:
"washing hands after defecating reduces the incidence of diarrhoea by 89% in the wilderness".
You're putting the cart before the horse, here. I don't think that the Wikipedia community sees 'a reliable summary of cutting-edge research' as on mission for the encyclopedia. You need to convince people that this should be done at all before jumping to how we might do it. And given the ongoing replication crisis in medicine, I think that will be difficult. MrOllie (talk) 15:32, 24 April 2025 (UTC)
Especially since cutting-edge research has become even more of a fools' playground than before with the advent of MAHA. Medical research is rife with fraud, grift, incompetence and irrationality - and this is about get even worse. The last thing we want to be doing is empowering the antiknowledge movement on Wikipedia. Bon courage (talk) 04:42, 25 April 2025 (UTC)
“a reliable summary of cutting-edge research” is called a systematic review. (A less reliable summary is science & medicine news reporting.) In order to be reliable, there’s methodological processes required. Wikipedia editors can summarise the results of the systematic review, but wikipedia is not the place to do a systematic review. Daphne Morrow (talk) 10:33, 25 April 2025 (UTC)
See my other comments throughout this thread for more info, or feel free to send me a message/email/whatever - I am always happy to chat - but I disagree completely.
Wikipedia should not be anywhere near the first place for new research, in any field, to be communicated. Prior to the internet Encyclopedias, and for that matter, various Diagnostic Textbooks, or even dictionaries were not frequently updated.
That being said, I also see the other side of this of the benefit of the internet and freely accessible knowledge, and the specific point that conflicts with my previous reasoning which is more people having access to information to poke at the questionable bits is a good thing. Experts are not infallible. Experts also have conflicts of interest. So do the literally anyone that this could be writing this message to you. Plenty of examples but most obviously is the simple phrase in academic publishing of "publish or perish". This isn't even getting in to the self fulfilling prophecy fallacy, or the one which probably has a name describing when a group of experts get together they are prone to affirming each others ideas whether or not those ideas are any good.
Anyway I don't think any major policies need changed or whatever (not that I've read them, yet, but they are open in another tab believe it or not) but the important thing is to question conclusions and not to shut down dissenting voices because it doesn't take a rocket surgeon to pull out the bottom jenga stick if that stick is just some other angle of seeing things. It happens. If you are in a big crowd and all are going one way and nobody is seriously questioning things I would suggest turning around or at least taking a seat to see if they're heading for a cliff. Admitting mistakes sucks and the amount of suck directly correlates with the size of the mistake. On that note, correlation does not equal causation and there is a worrying amount of ideas where the chronological relationship between those is switched completely unbeknownst to the person who is doing the big think
---
On that note, and so this is not another wall of text wherein I do a big think, I happened across a message about an upcoming vote on the Universal Code of Conduct for WMF and that seems on topic. See I told you I had things open in another tab! I have a lot of tabs.
WP:MEDRSinsists on having content from published, peer-reviewed research. And, for the reasons outlined in WP:WHYMEDRS, being published in peer-reviewed venue is the very bare minimum, and usually not sufficient. Meta-analysis is when you can start to make definite statements, rather than "studies by Smith,[1] Appleton,[2] and Binnington[3] says chocolate is evil, while studies by Jones[4] and Dillinger[5] says chocolate is a healthy food". Headbomb {t · c · p · b}14:36, 19 April 2025 (UTC)
I am very hesitant… first, saying “X study shows…” opens the door to original research (analysis and interpretation) by Wikipedians. If we are to pull back on MedRS, it is important that we keep it at “X study states…” and directly quote the conclusions from the study.
Secondly, DUE is a factor. If all we have is a single study that reaches a particular conclusion, that is not enough to say that the conclusion is DUE. We need to have additional studies that corroborate those conclusions. Blueboar (talk) 14:56, 19 April 2025 (UTC)
The level of sourcing for an article depends on what's available. For an ultra-rare condition like Oculodentodigital dysplasia, even a textbook or a systematic review is not actually very different from a primary source. On the other hand, if "what's available" is significant (e.g., for a common or heavily researched condition, like Hypertension, then the latest, greatest primary source should basically never be mentioned, or even hinted at.
A ==Research directions== section is supposed to be forward looking. The contents should sound like "In 2022, The Medical Organization called for research in X, Y, and Z" and not at all like "Dr I.M. Portant did a cool little pilot study".
The first point is very important I think, do you feel like the wording of the current policy explains that adequately?
As for ==Research directions== sections, why not have both? Don't you think it feels strange to say it calls for research without mentioning why it does so? Bendegúz Ács (talk) 11:39, 20 April 2025 (UTC)
(from Wikipedia:Canvassing) could be adapted to MEDRS's approach to primary sources. The usual approach is that primary sources are more acceptable in rare diseases (best source you've got...), in sections/contexts that have little immediate bearing on real-life human health decisions (e.g., an explanation of drug mechanism, a famous historical paper, veterinary information), or to support an "ideal" secondary source by providing a fun fact or an expanded detail (e.g., the drug is safe,[review][review] even during pregnancy and breastfeeding[primary]).
I think that the reason for a proposed direction is often pretty obvious. You don't really need an explanation when the recommended direction is something like "treatments with fewer dangerous side effects" or "prevention". If it's obscure, of course, then an explanation is desirable, but it needs to be "recommended more basic research, because this will provide necessary background knowledge for future drug design", not "recommended more basic research, because I.M. Portant's lab just published this cool little pilot study WP:IN MICE, which gave us experts some hope that it's actually possible to do something more practical than just counting the number of people who get diagnosed each year".
Whether to include individual past studies (i.e., things that can't be a recommended direction for research to go in the future, because they've already happened), our experience is that the studies tend to be cherry-picked for contradicting the medical consensus (One small study found the opposite of all 172 other clinical trials ever run!) or the editor is engaging in self-promotion (Our lab published a paper!).
TLDR: Attempting to be "on the cutting edge" of things, presenting "new" discoveries, is literally guaranteeing lower credibility because new discoveries are inherently based upon less evidence.
The thing is with health studies - any science that wants to be valid - is it takes time and effort to do studies and for those studies to be made available for peer review and that peer review to return to the original researchers and their correspondence to be further viewed for flaws in logic by more peer reviewers and ultimately most things are not ever really fully 100% settled - and, to my point, attempting to be "on the cutting edge" of these things, presenting "new" discoveries, is literally asking to have lower credibility because new discoveries are inherently based upon less evidence, and this actually causes many problems - this extends beyond Wikipedia and health studies - where since 69420 places repeated what once appeared to be true, in order for the one dude who noticed the flaw in the logic to disprove whatever it is, they must now deconstruct what 69420 different entities repeated and staked their institutional reputation on. So, point being, not only does it lower the credibility of Wikipedia by attempting to "be on the cutting edge" more places - especially ones as widely known and frequently accessed and trusted as Wikipedia - repeating things that have not passed sufficiently through the gauntlet of the scientific method quite literally makes all of us dumber and causes a spiraling number of problems that are nearly incomprehensible.
>"our experience is that the studies tend to be cherry-picked for contradicting the medical consensus"
You have a mouse in your pocket? My experience is exactly the opposite.
Frankly I don't think I have ever found a single publication - whether in a professional, academic, or "for public consumption" sources - that actually call discoveries in to question. The closest I can think of are ones that do so in indirect ways that don't actually address the fundamental issues of the evidence, eg questioning the study on ethical grounds - which is valid, but you can bypass the debate of ethics if you simply invalidate the evidence entirely. If the evidence is false there is no debate.
On that note, and as I believe I have stated elsewhere in this talk page, there is a monumental increase in studies being retracted for being based upon false evidence, so it must happen, yet from what I have read, which is a lot, the amount of studies being retracted is still a drop in the ocean compared to the amount of studies based upon questionable grounds.
The problem is, the people doing the research, are incredibly financially incentivized to continue it and to not invalidate similar research - since "healthcare" is a kajillion dollar industry (yet somehow the actual front line providers are underpaid and overstressed...) - and further, the general public is incentivized to believe in new discoveries for old problems because, "surely there must be some fix?!" - and additionally, the powers that be, whether they be government, industry, or academia, are further incentivized beyond the financial reasons because beneath all of the nonsense the important thing is to have hope for better.
The problems you describe are not Wikipedia's problems. Wikipedia's problems look more like this:
A hundred studies demonstrate that vaccines do not cause autism. All the meta analyses agree. All the systematic reviews agree. All the medical textbooks agree.
POV pusher says: But I wanna cite this one weird outlier study to say that everyone else is wrong!
Yes. In those cases, I find it more effective to mention that the study exists (just that it exists, sometimes as distant as "On [date], Big University issued a press release about research done by the I.M. Portant lab which claimed a cure for cancer"). If necessary, editors make a note on their calendars to remove it once the media attention has died down. WhatamIdoing (talk) 18:47, 23 April 2025 (UTC)
Anti-cholesterol food fad text and citation at Oat
An IP editor (see User talk:2601:642:4F84:1590:9D68:3412:E33:7827) has just made a series of edits to Oat about the food fad for oat bran in the 1980s concerning the belief that oats lowered cholesterol. I reverted their first attempt, adding a note "do not add individual primary research studies, see WP:MEDRS", but this was ignored with their later series of edits and their edit comment I didn't make a health claim. I noted a study's historical significance as the basis of a fad. There is some truth in their claim, but the edits have inserted a 1986 study, a piece of primary medical research, Van Horn, Linda; Liu, Kiang; Parker, Donna; Emidy, Linda; Liao, You-lian; Pan, Wen Harn; Giumetti, Dante; Hewitt, John; Stamler, Jeremiah (June 1986). "Serum lipid response to oat product intake with a fat-modified diet". Journal of the American Dietetic Association. 86 (6): 759–764.. I would be grateful for the opinion of MEDRS editors on whether the inserted material is compliant with policy, and whether any changes need to be made to the article text. Note that the new text spans both the "Health effects" and the "As food" sections of the article, with a "(described below)" textual cross-reference between the two. Thank you for your time. Chiswick Chap (talk) 06:35, 22 April 2025 (UTC)
Does MEDRS apply to "further reading" on a medical topic?
There is a [now very] long discussion at Wikipedia talk:External links#Proposal: expand the scope of ELNO to include "Further reading" that I opened last week but have since taken a bit of a back seat. In essence, it observes that we have quite firm rules about WP:external links but essentially none about wp:further reading – illustrated by the fact that the former is a formal guideline but the latter is only an essay. So I'm curious to know whether I could add a book to the further reading section of an article about a health condition, which makes wild assertions about its cause and which proposes a unique blend of snake oil and homeopathy to cure it. On what policy basis could I be sent packing?
The debate has crystallised around one or other of two opposing positions:
it is not only OK but a positive benefit to readers for Wikipedia editors to addto a Further Reading section whatever sources they personally consider useful
Wikipedia editors are also "some random person on the internet" and should not be recommending anything; at best, the entries in such a list should have some kind of RS support.
Full disclosure: I am in Camp 2 but this is not intended as a canvas for either perspective.
Does this debate raise any particular issues for MEDRS? and if so, is it best handled by a specific local guide-line? (To be clear, I don't contribute to medical articles but felt it appropriate to make the debate known. If this provokes another debate here, I wouldn't have any meaningful contribution to it.) 𝕁𝕄𝔽 (talk) 21:27, 13 May 2025 (UTC)
I'd say "not necessarily", but that there would need to be an exceptionally good argument (and a consensus) for such a source to stay if editors were objecting to it and it was a source discussing WP:BMI. Bon courage (talk) 21:38, 13 May 2025 (UTC)
I'm in the other discussion, but I should add that three types of not-exactly-MEDRS sources have been discussed in the past for ==Further reading== sections. They are:
a historical source (e.g., the first published description of the disease the article is about),
I have advocated for the first of these to be listed under ==Further reading== instead of listed as reliable sources under ==References==. Other editors prefer to cite them as reliable primary sources. WhatamIdoing (talk) 00:17, 14 May 2025 (UTC)
For consistency with the ethical motivation behind the MEDRS self-restraint, I would like to think that the entries for any famous altmed/pseudoscience sources include a clear denunciation. We should not expect or require readers to have read the entire article beforehand. We should never underestimate the ability of pseudo-scientists and conspiracy theorists to cherry-pick for sources that appear to endorse their lunacies. Or indeed their ability to plant "evidence", which is why we have a particular responsibility to verify the quality of anything recommended in Further Reading of BMI articles at least. 𝕁𝕄𝔽 (talk) 09:46, 14 May 2025 (UTC)
Yes, historical and inaccurate sources should be labeled, though perhaps "denunciation" is the wrong word for it. One wants to write something like "First book promoting this idea" rather than "Lunatic nonsense".
And lest anyone be concerned, telling people facts about a book is 100% consistent with NPOV and also does not constitute censorship as understood by librarians. Librarians are opposed to saying things like "this book is not suitable for children" or "only healthcare professionals should be allowed to read this"; they have no concerns about accurately labeling porn as "pornography" or hoaxes as hoaxes, and IMO we shouldn't, either. WhatamIdoing (talk) 18:24, 14 May 2025 (UTC)
Nor am I, but when we recommend a particular set of sources, we are in effect saying that we endorse these sources unless there we also give a caveat that says otherwise. There is a big difference between stocking a wide variety of sources in an academic library, where visitors are expected to inform themselves (on the one hand) versus (on the other) advising high schoolers to read Uncle Tom's Cabin without also advising them clearly that, although the literary merit of the work still stands, the values of the time are not acceptable today 𝕁𝕄𝔽 (talk) 19:24, 14 May 2025 (UTC)
I don't think that we're "endorsing" sources. This has come up, just in the last couple of years, in discussions about refs, with a few editors claiming that we can't use this or that source, because if we cite it, we're not just saying it's reliable to support the claim that the article is making, but also publicly endorsing it as being somehow morally acceptable or otherwise desirable. I have assumed that this is one of the inevitable consequences of real-world acceptance of Deplatforming: "How dare you invite that wrong-headed politician to my university!" becomes "How dare you cite that wrong-headed political author in Wikipedia!" – even if the actual facts themselves aren't in contention. WhatamIdoing (talk) 21:17, 14 May 2025 (UTC)
Straw man argument, I expected more from you. The only proposal is that the basis be shown for putting an entry in the FR list, so that readers don't have to rely on the word of some anonymous person on the internet that this source is worth reading – despite the fact that it wasn't [good enough to be?] cited and so didn't have to pass the RS test (and in this context, the even higher MEDRS test). 𝕁𝕄𝔽 (talk) 22:16, 14 May 2025 (UTC)
You said: when we recommend a particular set of sources, we are in effect saying that we endorse these sources
I said: I don't think that we're "endorsing" sources
Where do I say anything that sounds like a weakened version of your "argument"? (It's not really an argument; you present this, sans evidence, as a given.) WhatamIdoing (talk) 01:35, 15 May 2025 (UTC)
if we cite it, we're not just saying it's reliable to support the claim that the article is making, but also publicly endorsing it as being somehow morally acceptable or otherwise desirable ← not quite, but Wikipedia supposedly publishes "only the analysis, views, and opinions of reliable authors", so using a source implies quite a lot about its (authorial?) worthiness. Bon courage (talk) 05:08, 15 May 2025 (UTC)
True but we have clear guidelines (in the case of BMI, very clear guidelines) on what is a valid source. For Further Reading we have none. 𝕁𝕄𝔽 (talk) 09:35, 15 May 2025 (UTC)
I don't think that's true. Consider that our usual guidelines generally prefer:
independent sources
non-self-published sources
sources with visible editorial structures (e.g., a newspaper editor, a peer review process)
sources with reputations for fact checking
secondary sources rather than primary sources
Now contrast that against the fact that tweets have zero of those qualities, but {{cite tweet}} is used in tens of thousands of articles. We're not "endorsing" these tweets, and our rules aren't "clear" that non-independent, self-published, un-reviewed, un-fact-checked, primary-source tweets constitute "a valid source". But we're using them anyway.
It is true that we have only one officially tagged {{guideline}} for what belongs in the FR section (NB: one ≠ none). You might think that MOS:FURTHER, which provides only three rules about which sources can be listed – namely:
"publications that would help interested readers learn more about the article subject"
avoid duplicating sources used to support article content, except in unusual
never duplicate sources in the External links section
In reply to WhatamIdoing, examples 1 and 3 appear to be cases where one is pointing the reader to primary sources. That is not, I suggest, the job of an encyclopaedia. We are writing a tertiary source and we generally use secondary or tertiary sources as citations. A "Further reading" section should be to other secondary or tertiary sources. Bondegezou (talk) 08:32, 15 May 2025 (UTC)
@Bondegezou, I don't think that's true. Consider an article such as A Midsummer Night's Dream: A copy of the play is a primary source. WP:ELYES #2 says that if a legal copy of the work can be read online, then we should add such an ==External link==. But you are saying here that we shouldn't ever include a link to that play under the ==Further reading== heading. That sounds silly to me. I bet it sounds silly to you, too. WhatamIdoing (talk) 17:45, 15 May 2025 (UTC)
That is a link to the actual thing itself. That’s different to providing primary source links about a disease. Bondegezou (talk) 07:43, 16 May 2025 (UTC)
"The actual thing itself" is a primary source.
When we take a disease-related article to FAC, editors usually want to provide a citation to an original description of the disease. It's a primary source, but remember that WP:PRIMARYNOTBAD. Sometimes this is added as a ref. Sometimes this is added as a Further reading link. Since the ref would usually be put after a sentence that it can't fully verify (e.g.,"The first description of ____ was published in 1888 by John Smith" – it can't usually verify "the first"), I usually prefer a Further reading link. What would you do?
To give another example, if there's a famous memoir about a disease (e.g., by someone who clears the high bar at Wikipedia:Manual of Style/Medicine-related articles#Notable cases), then Further reading is IMO the best place for it. Why would you write "Alice Activist wrote a memoir[1]" in the article text, when you could just write "Activist, Alice My Life and Hard Things (memoir)" in the Further reading section? WhatamIdoing (talk) 16:14, 16 May 2025 (UTC)
classification of the different types of scientific literature
Hello, I am trying to build a diagram to sum-up visually this page. Can you please provide your feedback, and suggestions of changes ?
Note A: I know that there is no mention of Letters to the editor in this page, but I took the freedom to add them in the diagram, as they have been used multiple-times for disinformation (e.g. 1, 2).
I think that this is a bit too one-size-fits-most, because Wikipedia:Biomedical information#The best type of source depends on the claim that the source is supposed to be supporting. "Some research has been done on ____" needs a different kind of source than "Wonderpam cures cancer".
While I agree with your assesment of this seeming a little too "one size fits all" I'd like everyone to imagine their first time editing a medical page and how daunting WP:MEDRS can look. I know personally I found it very hard to wrap my head around the whole "use tertiary sources but also those don't exist for some topics" when starting out editing. I think not having more basic, easier to understand versions of MEDRS does Wikipedia diservice (yes even at the risk of leaving out some important details).
I understand the one-size-fits-all problem. I have narrowed down the scope of my annotations to medical claims only. I have added that invalid MEDRS sources may still be acceptable for non-medical claims.
I also share @IntentionallyDense's opinion that the lack of a more basic, easier to understand version of MEDRS does Wikipedia disservice. It's better to give a nutshell-diagram, to communicate the broad lines, and make readers immediately understand that MEDRS guidelines are not just "obvious common sense".
MEDRS scientific information flowI find that a lot of the information is not in the same location at each step (e.g. information about peer review is in the title of grey literature, but is a subtitle for the other boxes). I think you should standardise each box. I also think you would be better off separating the iconic/diagrammatic elements into a separate layer below the list of literature types.Daphne Morrow (talk) 05:39, 6 February 2025 (UTC)
@Daphne Morrow you're an artist! Your diagram is indeed much clearer.
I also like what you did for the Popular Science category; with the arrow pointing to it from all categories. It's a good reminder that popular press often prematurely cites pre-prints or working papers.
- Grey literature: in my original diagram I was only talking about "non peer reviewed grey literature", not all grey literature. Indeed some grey literature is released by institutions with an internal peer reviewing process. On 2nd thought it's probably better to avoid the term "grey literature", and merely name this category "non-peer reviewed writings" instead. And in that case, it's better to remove the mention of conference proceedings published as supplements, and animals&petri-dish studies.
- Regarding self-published books, as pointed out by @CFCF, my annotation was unclear. I should have mentioned that the publisher is NOT a recognized scholarly publisher. I've updated my image to reflect this.
- For the arrow from "non-peer-reviewed writings" (f.a.k. grey literature), only pre-prints make it to the "primary literature" category. So the arrow should originate from pre-prints.
- I don't understand what you mean by "grey lit and early stage research informs, study focus and methodological design".
- It's a detail, but I meant the funnel icon as a way to symbolize "synthesis". As there is no synthesis from "non-peer-reviewed writings" to "primary studies", it's better not to put the funnel.
I love it! The only draw back is that tertiary literature seams to be preferred over secondary literature (even though it's the opposite), as it sits on top. But maybe a simple comment on the diagram could correct that perception. Galeop (talk) 07:17, 3 February 2025 (UTC)
I would like feedback on whether I should include more literature types (eg clinical practice guidelines), whether animal studies / in vitro belong in the bottom section, and whether there are any other kinds of information I should add. Daphne Morrow (talk) 13:02, 4 February 2025 (UTC)
Thank you so much for your contribution @Daphne Morrow
I think both our diagrams complement each other. Your diagram ranksthe sources for medical claims on Wikipedia. My diagram represents the flow of scientific literature, and mentions what kind of literature is preferred for medical claims.
There may be a need for both:
1) I am convinced there's a need for an illustration of the flow, as most people have never heard about the categories of scientific literature. But maybe my diagram should be lighter ?
2) There may well also be a need for a ranking of sources.
Suggestions for your diagram:
- Ranking sources for "grey literature" and "tertiary literature" is quite difficult however. Indeed, those categories are not codified/standardized. I think it would be less risky to bulk their respective items together in the same big box, without attempting to rank them. So I suggest a "grey literature" box with an unsorted list of items; and same thing for "tertiary literature".
I think stylistically the pyramid is fine, however the content needs to be reworked before it can have any chance of being included.
Just as Galeop says, grey litterature is a very broad category, which contains basically all literature that lacks a PMID, DOI, or ISBN (and depending on definition some that have DOI:s such as preprints). This contains some of the highest quality reports, be they HTA:s, metaanalysis or review by government agencies, major reports by the WHO, CDC, FDA etc. These do not run through academic peer review, although they often employ many other types of peer reviews. These are among the best sources out there - both from a scientific vantage point, but even more so for building Wikipedia content.
This causes issues when you rank sources in a pyramid. It isn't always as clean as we try to make it. There is also the issue of a low quality meta-analysis being far worse than a high-quality RCT. The current guideline includes two pyramids to show that there are different rankings, and one of them places clinical practice guidelines at the top. Sometimes clinical practice guidelines can not only be the best evidence, but they can define the condition. To state that a meta-analysis is better in those cases is ... how should I put it - nonsensical.
Also, in vitro studies are not grey litterature, and if you want to be that nit-picky you're missing in silico studies below in vitro, and umbrella reviews above meta-analysis. And what you call "literature reviews" are often referred to as "narrative reviews" or "narrative literature review". Also you have scoping reviews, that should place above narrative reviews, but below systematic reviews. And "other reviews" is to me not a useful category.
And what do you mean with researcher's book - do you mean self-published? Or just any book? There are biomedical tangential topics where a book is the best resource, for instance psychological, sociological, or anthropological books that are directly linked to medical outcomes. For instance you have Goffman's Stigma: Notes on the Management of Spoiled Identity, which is probably the most cited work extrapolated to HIV-related stigma, which is a field where MEDRS would apply. What differentiates a medical handbook from a researcher's book? Is it just that it has handbook in the name? CFCF (talk) 23:20, 6 February 2025 (UTC)
And what about outbreak reporting, and mortality data. I realize the COVID-19 Pandemic article is completely non-compliant to MEDRS when it reports on Deaths. However, I don't think one should insist on only academic and government sources there either. CIDRAP does excellent reporting, ... I need to take a look at that as well. Things change when you're gone from Wikipedia. CFCF (talk) 00:02, 7 February 2025 (UTC)
>I realize the COVID-19 Pandemic article is completely non-compliant to MEDRS when it reports on Deaths.
Ironically I feel like everything about covid is entirely non-compliant to any logical views of validity and what cause and effect are.
That's not the reason I'm writing here though. I wanted to ask about RCT's and this is the only chain of comments on this page mentioning them so I suppose it goes here.
Main point I wanted to make was I don't think it is exactly undisputed that RCT's are the end all be all for validity of medical literature. I am no expert of course and this is only my gist of it, but it seems like even if that is the accepted SoP in medical literature, great minds outside of medicine have looked things over and are asking a lot of questions for valid reasons.
I don't think any of the authors are specifically in medical fields but last I checked medicine and healthcare does not have any special interpretation of what cause and effect means. The authors of those papers are highly respected and well known and qualify as experts if experts do indeed exist.
I don't exactly have a question or any specific suggested change here, just thought I would mention just because something was the accepted "fact" twenty years ago, the thing about science is usually it is updated over time as more things are understood and studied and more people input their thoughts. Just my .02 Relevantusername2020 (talk) 04:06, 8 February 2025 (UTC)
I could be wrong but I believe these weaknesses of RCTs in specific circumstances is part of why MEDRS prefers meta-analyses and systematic analyses as sources. They tend to evaluate the strength of RCTs against the strength of other studies. Daphne Morrow (talk) 06:53, 8 February 2025 (UTC)
About researchers' books, Daphne and I indeed meant "self published books", or published by non-scholarly publishers. I am thinking for instance of books by star-scientists, who write books for the general public, and mix in such books peer-reviewed results with never-published-anywhere-else own results. Typically such books are published by publishing houses that have nothing to do with academia (but everything to do with selling lots of books).
About covid mortality data, although I do agree with your point, I think it's okay if this pyramid doesn't feature any category for them. Indeed, such data is more "raw data" than "evidence" (i.e. results from analysis). Galeop (talk) 14:25, 8 February 2025 (UTC)
I agree Galeop - we do not need to include specifics on Covid-19 mortality data here, as for the comment by Daphne Morrow on RCTs and MEDRS - I think you are precisely right. Relevantusername2020: There are also other issues with RCTs, in part because they are very expensive, and this steers which topics are explored. I would suggest anyone with an interest in the topic to read Justin Parkhurst's The Politics of Evidence [14], which is OA.
As for the points on high quality grey litterature, and "other reviews" - I think that must be addressed before we can suggest including any infographic. CFCF (talk) 14:34, 8 February 2025 (UTC)
@CFCF, from your experience, would you say that clinical practice guideline could be considered as tertiary literature ? I know it's not published by publishing houses such as University Presses or Reference Works publishers; but aren't those clinical practice guidelines mostly based on published primary and secondary studies? (it's a honest question; I really don't know the answer) Galeop (talk) 07:57, 11 February 2025 (UTC)
Coming back to this, I am not so sure it isn't tertiary litterature. It depends, and I'm not sure it matters that much - but rather points to the somewhat arbitrary and artificial divide between secondary and tertiary litterature in highly technical fields such as medicine. CFCF (talk) 11:39, 20 February 2025 (UTC)
Probably not. Generally speaking, in wikijargon, tertiary sources are encyclopedias, dictionaries, and other sources that provide brief, general information summarizing pre-existing knowledge without adding anything of their own. This includes textbooks for children but not necessarily at the university level (and rarely at the graduate level). It sometimes includes bibliographies, directories, lists, timelines, and databases that provide bare facts, but not something like OMIM (whose entries usually include multiple paragraphs of custom description).
A clinical practice guideline adds 'something of its own', namely a recommendation for/against something. That makes it a secondary source. WhatamIdoing (talk) 05:00, 14 February 2025 (UTC)
I've added Clinical Practice Guideline as a separate box in my attempt to illustrate the flow of scientific literature (which is a different diagram than the pyramid currently debated, which attempts to create a hierarchy). Any comment?
Overall, I think I'm not the best person to tell you what's useful to a newer editor.
I suspect that what's useful to a newcomer is going to depend partly on their background. For example, med students get some explicit training on these things, so they already know some of this. Other people, even with equal or more academic accomplishments, don't know what some of these words mean. WhatamIdoing (talk) 00:50, 18 February 2025 (UTC)
Hopefully all involved and interested get pinged from this. I'm glad to see there is a lot of care involved in getting this right and keeping things high quality.
I have a lot of links I could share here, and will share examples if needed, but I guess my overall point is that due to the nature of Wikipedia combined with the reality of healthcare, health research, publishing incentives (ie, publish or perish) and to the point, there is a lot of retractions and I suspect even more that should be retracted. There are a lot of questions about things that have been established fact for a long time and even more questions about things that never reached a conclusion satisfactory to anyone involved. If this were any other encyclopedia, it would be a simple answer. Wait til the experts establish the official textbook definitions and views and whatnot. Obviously that is not the case. So, this is much less of an issue in things that are strictly physical in nature but when it comes to brain things, the facts are on shaky ground to put it mildly.
I am not a professional. I am not always confident about things I say (though you would never know that lol) but on this topic I am 100%
I guess I don't have an overall point or a nice conclusion and that is kind of appropriate for the overall content of this message I suppose. Overall I think it is going to have to rely on base logic and barring any major change in policy on behalf of Wikimedia, if the power enabled via having the know how on how to "set the story" for whatever it may be hasn't quite hit you yet - not sure how it hasn't after 2020 - I suggest taking a step back and realizing what is posted here on whatever issue has a lot of validity to a lot of people so make sure what you add holds up under scrutiny - scrutiny the cited sources may not have received.
On that note, and specifically in regards to that last message about med students receiving explicit training in these things: not necessarily. I mean. I am entirely self taught on the things I know but there are a lot of experts in a lot of fields who I would consider numerically illiterate. That is they are terrible at critically thinking about the underlying information statistics are communicating. This is the source of a lot of issues with a lot of health studies. It doesn't take an expert to figure out where the fault lies a lot of times though, usually it is as simple as seeing an angle the original authors did not. Poor example, but here is a reddit comment *I made awhile back doing just that. For a much better written explanation of statistics and how they (might) lie (as the old saying goes): https://unherd.com/2024/05/the-danger-of-trial-by-statistics/
Forgive my language, attitude/excessive sarcasm/etc and weird punctuation/grammar/etc. I am trying to do better :D
Thank you for your comment @Relevantusername2020. I agree with your overall message, and I think the problems you point out highlight the importance of an evidence-based academic publishing process. Namely, it stresses the importance of Meta-science studies, to guide the policies of academic journals (as it is already the case). I think the metascience study here articulates quite well some of the problems you pointed out : https://doi.org/10.1007/s11192-018-2969-2
About the perverse incentives of publish or perish, I think that, ironically, high impact factor journals are best placed to fight them ("ironically", because such journals are a main source of those perverse incentives). Indeed, only such journals have the clout to "punish" the breaches of scientific ethics, that the publish or perish system incentivizes authors to do. Indeed, by temporarily black-listing authors who lied in their disclosures or findings, they create a strong deterrent to lying. Galeop (talk) 15:30, 27 April 2025 (UTC)
About the pyramid with lots of blue lines: It would probably be interpreted as "this is slightly better than that". If that's not wanted, perhaps each main row should be split horizontally, like this stack of blocks? WhatamIdoing (talk) 04:37, 14 February 2025 (UTC)
Hi all,
I'd like your final opinion on my last edit, of the diagram here. The goal of this diagram is not to rank literature, but to illustrate the flow of scientific literature.
Any comment or objections?
classification of the different types of scientific literature
Hi @WhatamIdoing, apologies for my slow response too. And thank you so much for your response.
- You're making a good point about COIs. I felt this was an overall principle in Wikipedia, but I am biased by my own readings*. I will gladly simply remove this recommendation, if I've over-interpreted the rules on COIs.
- About popular science, I would not categorize it as belonging to "scientific literature", so that's why I've put it in its own box, separated from the rest by a vertical line. But your comment makes me realize that my annotation of clinical practice guideline as "internal peer review" makes it sound better than it actually is (it's usually not a proper independent peer-reviewing). I am thinking of replacing it by "self-organized peer-review only"; what do you think?
_________*Why I am biased by my own readings________
I am influenced by two umbrella reviews. In short:
- They found that COIs did NOT influence strongly the conclusions of clinical trials (Risk Ratio: 1.34; which has to be put in perspective with the fact that efficacy results and harm results were also more likely to be positive (respectively RR 1.27 and 1.37), so there's not a big misalignment between results and conclusions). Lundh 2017
- But they found that COIs do influence strongly conclusions of systematic reviews (RR: 1.98), even though COIs don't seem to influence efficacy results. This suggests that results of systematic reviews with COIs are reliable, but conclusions have to be treated with caution (probably because there's an excessive use of spin). Hence their conclusion: "We suggest that patients, clinicians, developers of clinical guidelines, and planners of further research could primarily use systematic reviews without financial conflicts of interest. If only systematic reviews with financial conflicts of interest are available, we suggest that users read the review conclusions with skepticism, critically appraise the methods applied, and interpret the review results with caution." Hansen 2019Galeop (talk) 10:56, 13 April 2025 (UTC)
The problem with COI in MEDRS-related sources is that you typically can't do any research on an as-yet-unapproved drug without the manufacturer agreeing to give you the drug, so all the available sources have some level of COI.
The perception of COI also varies. We have occasionally had new editors suggest, for example, that experts have a COI against anything they do professionally, e.g., that the only non-COI sources about knee surgery are those written by knee surgeons.
Because of this, COI has not been a functional model for identifying reliable sources. Of course, even though we don't use COI to invalidate a source (even for non-MEDRS subjects, sources are allowed to be WP:RSBIASED), it is helpful to keep an eye out for COI. You want to be careful about what you write with a source. For example, if all the mammogram device manufacturers say that the latest greatest expensive mammogram machine is much better and everyone must buy millions of dollars' worth of new devices right away, you might decide to write a very soft sentence – perhaps "incremental improvements over time" instead of "dramatic advancements". WhatamIdoing (talk) 21:27, 21 April 2025 (UTC)
classification of the different types of scientific literature
I don't know what the main page is you're referring to, but I have thoughts still lol. Apologies for my lateness.
From the main article: I'll let you review the section yourself, but what is written almost contradicts itself or maybe leaves room for interpretation. I would say as long as it is a higher quality "popular publisher" it is at least equivalent to a lot of academic publishers, and many times better than, since - similar to Wikipedia - if they are publishing an article on a topic, that topic has reached a point of notoriety. Contrast that with journals incentivized to publish "new" findings, and I would conclude Wikipedia should be closer to a popular publisher than a health/medicine journal. WP:NOR, WP:TECHNICAL, WP:NOTTEXTBOOK
>[P]opular science magazines such as New Scientist and Scientific American are not peer reviewed, but sometimes feature articles that explain medical subjects in plain English. As the quality of press coverage of medicine ranges from excellent to irresponsible, use common sense, and see how well the source fits the verifiability policy and general reliable sources guidelines. Sources for evaluating health-care media coverage include specialized academic journals such as the Journal of Health Communication. Reviews can also appear in the American Journal of Public Health, the Columbia Journalism Review, and others.
If you want to propose changing WP:MEDPOP, then I suggest doing it in another section.
(For the avoidance of doubt, and speaking as someone who has been editing Wikipedia's medical content for almost 20 years now: I think you have a very, very low chance of getting MEDPOP changed. But if you want to try, then please start a separate discussion in a new section.) WhatamIdoing (talk) 19:00, 26 April 2025 (UTC)
I hope it's okay if I just say a few words about this here. @Relevantusername2020 Although I understand your point, and I agree that some popular science journals are very good, I think that when editing a Wikipedia article, it's best to go straight to the secondary source (I mean, review articles). I personally read more popular science than academic journals, but when editing a wikipedia article, it is our "job" to follow a common methodology (ie the MEDRS guidelines). Going straight to review articles is therefore our "job", even though we wouldn't do that if we were just browsing the news during breakfast ;-)
About the preference for "new findings" that some academic journals indeed have (but not all ; e.g. PLOS, BMJ Open, etc.), I would say that this preference is not as strong when it comes to review articles. Galeop (talk) 14:39, 27 April 2025 (UTC)
@Relevantusername2020, I've updated the picture, and replaced "Beware of popular science that cites non-MERDS sources" with "Do not cite popular science that relies on non-MEDRS sources". The previous version could have been understood as suggesting that such popular science is predatory ; but that's not what I meant. Let me know if you have suggestions on how to rephrase it better. Galeop (talk) 14:55, 27 April 2025 (UTC)
I apologize for the late response, I have no further suggestions at this time.
I'm simply trying to do what little I can to help slow the spread of incorrect health information since, as with everything, and irregardless of the factuality of the underlying information, it is much more difficult to change the belief in an idea than it is for it to become widely agreed upon.
This article from science.org says a lot of the same things, which is appropriate since science.org is more 'pop science' but kind of an in between source.
As far as your points about large journals, it reminds me of the debate about open source vs proprietary and the differences between centralized vs decentralized - and appropriately a back and forth I recently had on reddit about that topic. I think really there is no clear "better way" and there will always be a bit of a pendulum effect on the way things are done, in all of the relevant topics of discussion. Relevantusername2020 (talk) 01:06, 20 May 2025 (UTC)
Wikipedia talk:Identifying reliable sources (medicine)/Archive 32
