Sunday, February 04, 2018

Should you send the scientist your draft article?

The Twitter discussion sparked by this poll was very illuminating. There’s a clear sense that scientists largely think they should be entitled to review quotes they make to a journalist (and perhaps to see the whole piece), while journalists say absolutely not, that’s not the way journalism works.

Of course (well, I say that but I’m not sure it’s obvious to everyone), the choices are not: (1) Journalist speaks to scientist, writes the piece, publishes; or (2) Journalist speaks to scientist, sends the scientist the piece so that the scientist can change it to their whim, publishes.

What more generally happens is that, after the draft is submitted to the editor, articles get fact-checked by the publication before publication. Typically this involves a fact-checker calling up the scientist and saying “Did you basically say X?” (usually with a light paraphrase). The fact-checker also typically asks the writer to send transcripts of interviews, to forward email exchanges etc, as well as to provide links or references to back up factual statements in the piece. This is, of course, time-consuming, and the extent to which, and rigour with which, it is done depends on the resources of the publication. Some science publications, like Quanta, have a great fact-checking machinery. Some smaller or more specialized journals don’t really have much of it at all, and might rely on an alert subeditor to spot things that look questionable.

This means that a scientist has no way of knowing, when he or she gives an interview, how accurately they are going to be quoted – though in some cases the writer can reassure them that a fact-checker will get in touch to check quotes. But – and this is the point many of the comments on the poll don’t quite acknowledge – it is not all about quotes! Many scientists are equally concerned about whether their work will be described accurately. If they don’t get to see any of the draft and are just asked about quotes, there is no way to ensure this.

One might say that it’s the responsibility of the writer to get that right. Of course it is. And they’ll do their best, for sure. But I don’t think I’ll be underestimating the awesomeness of my colleagues to say that we will get it wrong. We will get it wrong often. Usually this will be in little ways. We slightly misunderstood the explanation of the technique, we didn’t appreciate nuances and so our paraphrasing wasn’t quite apt, or – this is not uncommon – what the scientist wrote, and which we confidently repeated in simpler words, was not exactly what they meant. Sometimes our oversights and errors will be bigger. And if the reporter who has read the papers and talked with the scientists still didn’t quite get it right, what chance is there that even the most diligent fact-checker (and boy are they diligent) will spot that?

OK, mistakes happen. But they don’t have to, or not so often, if the scientist gets to see the text.

Now, I completely understand the arguments for why it might not be a good idea to show a draft to the people whose work is being discussed. The scientists might interfere to try to bend the text in their favour. They might insist that their critics, quoted in the piece, are talking nonsense and must be omitted. They might want to take back something they said, having got cold feet. Clearly, a practice like that couldn’t work in political writing.

Here, though, is what I don’t understand. What is to stop the writer saying No, that stays as it is? Sure, the scientist will be pissed off. But the scientist would be no less pissed off if the piece appeared without them ever having seen it.

Folks at Nature have told me, Well sometimes it’s not just a matter of scientists trying to interfere. On some sensitive subjects, they might get legal. And I can see that there are some stories, for example looking at misconduct or dodgy dealings by a pharmaceutical company, where passing round a draft is asking for trouble. Nature says that if they have a blanket policy so that the writer can just say Sorry, we don’t do that, it makes things much more clear-cut for everyone. I get that, and I respect it.

But my own personal preference is for discretion, not blanket policies. If you’re writing about, say, topological phases and it is brain-busting stuff, trying to think up paraphrases that will accurately reflect what you have said (or what the writer has said) to the interviewee while fact-checking seems a bit crazy when you could just show the researcher the way you described a Dirac fermion and ask them if it’s right. (I should say that I think Nature would buy that too in this situation.)

What’s more, there’s no reason on earth why a writer could not show a researcher a draft minus the comments that others have made on their work, so as to focus just on getting the facts right.

The real reason I feel deeply uncomfortable about the way that showing interviewees a draft is increasing frowned on, and even considered “highly unethical”, is however empirical. In decades of having done this whenever I can, and whenever I thought it advisable, I struggle to think of a single instance where a scientist came back with anything obstructive or unhelpful. Almost without exception they are incredibly generous and understanding, and any comments they made have improved the piece: by pointing out errors, offering better explanations or expanding on nuances. The accuracy of my writing has undoubtedly been enhanced as a result.

Indeed, writers of Focus articles for the American Physical Society, which report on papers generally from the Phys Rev journals, are requested to send articles to the papers’ authors before publication, and sometimes to get the authors to respond to criticisms raised by advisers. And this is done explicitly with the readers in mind: to ensure that the stories are as accurate as possible, and that they get some sense of the to-and-fro of questions raised. Now, it’s a very particular style of journalism at Focus, and wouldn’t work for everyone; but I believe it is a very defensible policy.

The New York Times explained its "no show" policy in 2012, and it made a lot of sense: it seems some political spokespeople and organizations were demanding quote approval and abusing it to exert control over what was reported. Press aides wanted to vet everything. This was clearly compromising to pen and balanced reporting.

But I have never encountered anything like that in many years of science reporting. That's not surprising, because it is (at least when we are reporting on scientific papers for the scientific press) a completely different ball game. Occasionally I have had people working at private companies needing to get their answers to my questions checked by the PR department before passing them on to me. That's tedious, but if it means that what results is something extremely anodyne, I just won't use it. I've also found some institutions - the NIH is particularly bad at this - reluctant to let their scientists speak at all, so that questions get fielded to a PR person who responds with such pathetic blandness and generality that it's a waste of everyone's time. It's a dereliction of duty for state-funded scientific research, but that's another issue.

As it happens, just recently while writing on a controversial topic in physical chemistry, I encountered the extremely rare situation where, having shown my interviewees a draft, one scientist told me that it was wrong for those in the other camp to be claiming X, because the scientific facts of the matter had been clearly established and they were not X. So I said fine, I can quote you as saying “The facts of the matter are not X” – but I will keep the others insisting that X is in fact that case. And I will retain the authorial voice implying that the matter is still being debated and is certainly not settled. And this guy was totally understanding and reasonable, and respected my position. This was no more or less than I had anticipated, given the way most scientists are.

In short, while I appreciate that an insistence that we writers not show drafts to the scientists is often made in an attempt to save us from being put in an awkward situation, in fact it can feel as though we are being treated as credulous dupes who cannot stand up to obstruction and bullying (if it should arise, which in my experience it hasn’t in this context), or resist manipulation, or make up our own minds about the right way to tell the story.

There’s another reason why I prefer to ask the scientists to review my texts, though – which is that I also write books. In non-fiction writing there simply is not this notion that you show no one except your editor the text before publication. To do so would be utter bloody madness. Because You Will Get Things Wrong – but with expert eyes seeing the draft, you will get much less wrong. I have always tried to get experts to read drafts of my books, or relevant parts of them, before publication, and I always thank God that I did and am deeply grateful that many scientists are generous enough to take on that onerous task (believe me, not all other disciplines have a tradition of being so forthcoming with help and advice). Always when I do this, I have no doubt that I am the author, and that I get the final say about what is said and how. But I have never had a single expert reader who has been anything but helpful, sympathetic and understanding. (Referees of books for academic publishers, however – now that’s another matter entirely. Don’t get me started.)

I seem to be in a minority here. And I may be misunderstanding something. Certainly, I fully understand why some science writers, writing some kinds of stories, would find it necessary to refuse to show copy to interviewees before publication. What's more, I will always respect editors’ requests not to show drafts of articles to interviewees. But I will continue to do so, when I think it is advisable, unless requested to do otherwise.

11 comments:

Emily said...

The main issue with your practice, for me, is the "slippery slope" problem. Yes, what you're doing is probably not dramatically affecting the content of your articles (aside from fixing inaccuracies). But there are journalists for whom that would matter: those that cover science politics or scientific misconduct, for example. Then it becomes necessary to start drawing lines: it's okay to share copy on this type of story, but not that one. Or this reporter can share copy but not that one. Same goes for particularly controversial subjects, and that line is even harder to draw. So that's why a lot of outlets just have a policy that is straightforward: no sharing of copy at all. It makes sense to me why publications want to do that, or why individual writers choose to do that.

You make the point that the journalist can always say, "no, I will not make that change that you have requested." But then there is always the temptation not to anger a valuable source, or other complications that can lead to changing the story. I have heard of cases of science journalists sending copy to sources who then got angry, and threatened to sue if their work was described as written. That leaves the journalist's outlet in a difficult position that can end up with the story getting killed.

I'm not sure Physics Focus is a good example, as in my opinion, it is not strictly journalism. It is published by APS to promote APS papers. Although it is also certainly intended to be useful to its readers, you wouldn't see a full-on takedown of a PRL paper there. (Although there is outside comment that is sometimes a bit negative. Maybe that makes it borderline.)

Finally, I will also note that almost every editor I've spoken with about this has told me not to send copy to sources. I've heard this over and over again, enough that I got the sense that this really is a norm of the profession. (Also I'd like to remain employed and my employer tells me not to send stories to sources.)

Anyway, I'm glad you wrote this because scientists who want me to send them my articles sometimes tell me of other science journalists who agree to do so and I always wondered who these mystery journalists were and what their arguments were.

Andy Extance said...

I agree with you Phil. There are a lot of practical advantages in terms of accuracy, and I do often send pieces to sources to check for accuracy, unless my editors specifically tell me not to. Interesting that Mika McKinnon sees it as getting scientists to do free work - most of my sources are keen to do this kind of thing. But also not making it a blanket thing is sensible too - for example sometimes there's no point, if it's a drug company or government agency, they won't add anything further. On the matter of whether things get legal - don't you actually have an ethical responsibility to show them, or at least inform them, if you're publishing something that might provoke that kind of response to give them a right of reply? I believe the key is to show them at the last minute so they don't have the chance to bring out an injunction - if that were ever to happen in the scientific field. I could go on and on, and my thoughts aren't as well honed as yours, but one final point. On the subject of whether a journalist can stand up against changes, going to look at ethics codes has made me notice point of the NUJ ethics code (https://www.nuj.org.uk/about/nuj-code/):
[A journalist] Resists threats or any other inducements to influence, distort or suppress information and takes no unfair personal advantage of information gained in the course of her/his duties before the information is public knowledge.
So if we don't stand up to our sources, we're not doing our job.

Peter said...

I don't write much nowadays but I found the following approach useful when I was at Physics World and often interviewed people over the phone:
1) Type up the notes of the interview straight away, ideally in the form of two or three paragraphs that could be dropped into the story, including facts (such as names and numbers), direct quotations, and other relevant information
2 Email the notes to the interviewee, asking him/her to send corrections/comments by a deadline. This also allows you to send any follow-up questions that you didn't ask over the phone
3) Write the article on the basis of these notes, which have been fact-checked already. I did not send a draft of the whole article to anyone before publication.

Philip Ball said...

Emily,
Thanks so much - this is really useful to know. We're mostly in complete agreement: yes, writing on science politics/policy, misconduct and controversies undoubtedly creates situations where it would be highly advisable not to send any copy to sources. And I can see why some science publications would implement a blanket policy for that reason, so that the rules are clear. But in my experience that could well act to the detriment of accuracy of much straightforward science reporting, which describes much of what I do. So I'd prefer the writer to be allowed some discretion.
As for not wanting to anger a valuable source, or risking being sued - well, won't both those things happen anyway if the comments are published without the source seeing them first? And personally, if one of my sources were going to get bolshy about some unreasonable (in my view) request for a change, I would have no worries about losing their good will as a source.
Agreed that Physics Focus is a grey area!
And yes, I'm prepared to do as I'm told by editors too! But I'm a bit concerned that, as this approach becomes more and more the standard practice, there may be unsufficient debate about the pros and cons - it could end up as just "That's bad journalism!"

Andy: "If we don't stand up to our sources, we're not doing our job." Absolutely. My preference would be to confront them if they are being unreasonable and manipulative, not to duck the issue by avoiding it. Though I agree it gets tough if they start getting litigious...

Peter: That's a good scheme. But I'm not sure how, short of telling them exactly how you're going to describe the work, you can then be sure that your technical explanations are going to be correct (which is a different matter from the accuracy of quotes). That's the aspect I want them to check.

Jacob Aron said...

I wrote something this morning but I think it may have been eaten by internet gremlins. So, my perspective as a New Scientist editor:

There are two fairly practical reasons for not sharing copy which you've not really touched on. The first is simply time: sending copy to a scientist, after it has been written, edited and subbed (and why would you bother before?) adds yet another stage, introducing a delay that is rarely worth it when it comes to news stories.

The second is, while I may trust New Scientist staffers to stand up to attempts to change their copy, I'd be less confident that all freelancers, some I will know better than others, can do the same. A blanket rule against sharing copy helps protect those freelancers, particular people who are just starting out and may not have the confidence to face off with a source. After all, they can always blame the evil editor.

That said, I'm very happy to run an analogy for a tricky concept by a researcher, though normally I'd do this on the phone and paraphrasing, rather than sending copy.

Ehsan Masood said...

Hi Phil (and colleagues),

I don't know why I always find this subject endlessly fascinating, so here's a bit of stream of consciousness (though more from the perspective of an editor).

Policies are important so that both writer and source/interviewee know the boundaries. This did not need to be the case so much in the past but is now needed as more scientists begin to be represented by their universities' and government department communications and strategy teams. What was once more common in large companies is now pretty widespread in the public sector too.

I was recently at a small event at the Broad Institute at MIT. After we did a round of introductions it occurred to me that the numbers of in-house communications/impact/strategy staff sitting in the periphery of the room were almost as many as the journalists sitting around the table. In some respects that is understandable, as the Broad is potentially sitting on a lot of lolly and it is entirely in its interests to influence the message. I would totally do the same were I in their position.

Also, scientists in these institutions are encouraged to take media training and can be advised in these courses not to deal directly with journalists (especially with journalists they don't know).

As writers and journalists who talk to scientists, we are essentially dealing with employees of large organisations now. Hence I've found that creating a fairly tight policy and explaining it up front is important so everyone knows where they stand.

Inspite of this, I have lost count the number of misunderstandings, arguments, cross words etc that can quickly happen given the tight deadlines for daily and weekly news and the numbers of folks involved.

I accept that one policy for every type of article can end up becoming a blunt instrument. Checking technical accuracy on a single source story is very different to checking the accuracy of competing claims to a discovery or invention.

But once you've set the policy, as an editor/editorial team you have every right to interpret it according to the merits of the situation.

I think we can all agree that discretion is everything!

Best wishes, Ehsan

Unknown said...

I'm with Phil and Andy on this. I don't see the problem with sending a draft to check quotes and accuracy of scientific descriptions. The articles I'm writing are mostly highlight pieces where the aim is to translate some pretty complex science for a more general audience. And I regard that more as a fact checking process - they certainly don't get full copy approval (I would overrule any attempt to change something that wasn't a factual error).

I agree it would be different if the article was of a critical nature. And if there were quotes from another source (critical or otherwise) I'd likely leave them out of this process. The only reason I do this by sending a draft rather than on the phone is that it's easier than trying to schedule a call with busy people (who may also be in a different time zone).

Amy Charles said...

Part of my job involves being a slice between university chemists and journalists, university-news and otherwise. Bidirectional prep and translation.

There's a problem with fact-checking by running descriptions and explanations past a scientist (which seems the obvious and sensible thing to do) when the scientist isn't terribly media-savvy and doesn't know what the journalist's job is, so doesn't have a sense of the precision/accuracy tradeoff in translating things for broader consumption. From their perspective, the world is Disciplinary Journal World, so every translation gets struck down as inaccurate, the analogies are whittled away to nubs, and everything's replaced with paper language -- and then they're bothered (usually over something no one else would notice) when the story still doesn't come out sounding like it belongs in Journal of Sub-Subdiscipline B. My experience is that while most scientists will come around given enough time and trust, and accept that things will necessarily go out of focus as you try to talk to a broader audience about the work (also that no, most people really don't know what X is), it takes time to build the trust and find the limits of that scientist's tolerance. More time than someone on tight deadline likely has. (Another problem: recognition of the importance of story, as opposed to bare explanation. I had an episode once in which a scientist and I were working on a short NPR piece about his Science paper, and I'd put story in, and he'd take it out, and we went around like that a few times before I realized that the story was an irrelevance to him, and that I'd have to explain why it was important there. Happily, he was game and we'd built a lot of trust, so explaining wasn't expensive.)

So part of the question is also about *which scientists* you're going to for the run-by -- is it someone who understands what you're doing, who's a sort of partner in sci comm? And then how are you going to talk with a less savvy colleague who's heard that you give that other person copy to look over, and that you accept their changes?

One of the things I've been doing, intermittently, is to encourage and help the chemists (incl. grad students) I work with to develop their own news-ready analogies and pocket explanations, even when there aren't any journalists emailing. Just as a matter of course: you're writing up something new, how would you explain it to someone who's not a scientist, or not a chemist? Some of them are very good at it, and a nice byproduct's that what's essentially science writing starts turning up in grant proposals, especially private/foundation proposals, where it's usually very well received. While being handed the prose like that might be restrictive for a really good science writer, on the whole the response from journalists has been gratitude. (1/2)

Amy Charles said...

(2/2)

Our university absolutely wants scientists talking to journalists, but the institutional media training I've seen is aimed at local-news pitches, useful mainly in putting across to scientists how few words they'll have, how very simple the story has to be, how little the reporters are likely to know, and how much goggling at kit there's going to be. At some point I'll probably put something together about recognizing what kinds of stories can be put across well that way, and what stories belong to other forms and media.

I appreciate the point of having a no-run-by policy when you're a large, fast-moving news organization, but I suspect that a landscape like that further polarizes scientific communities -- scientists comfortable with sci comm and journalists are better able to defend themselves and steer their stories, becoming media-friendly, and those who aren't wind up feeling awfully banged around and maybe disinclined to try it again, even as their institutions push them to. I know this is not what publishers or institutional marketing divisions want to hear, but maybe the answer here is that science journalism ought to consider adapting to science, rather than the other way round, and slowing down. Going at longform pace, or even at book pace, also lets you avoid the problem of standing two inches from a pointillist painting, which sounds to me like a happy thing for readers.

Dana said...

Philip, I'm writing a piece for Undark about the recent Twitter poll and subsequent debate over whether journalists should show scientists their quotes and/or drafts before publication. Would you be open to chatting with me about your experience and opinion on the matter?

Philip Ball said...

Amy, these are great points - thanks so much. I guess I'd try to use judgement about when scientists are making valid points and when they're just being pedantic or stuck in their disciplinary box - but it's not always easy, and as you say, time is often short. I certainly wouldn't consider it a duty to keep the scientists happy! I reckon we need both quick news and longform in science reporting, and the former is bound to have to cut more corners, as in any journalism.