One of the few agreements about e-cigarettes is that there is still no consensus about the role they play in helping smokers quit.
This post reports on the conflicting (and missing) evidence on the effects of using e-cigarettes for smoking cessation. A subsequent post will look at the conflicting (and missing) approaches to promoting e-cigarettes for quitting that have been recently adopted by health authorities.
More evidence that reviewers disagree
At the beginning of this month, yet another review of evidence on e-cigarettes as cessation devices was published. This umbrella review of the health outcomes of e-cigarettes [1] was conducted by a team headed by Dr. Banks, and is the a journal version of a study commissioned by the Australian government. [2]
After reviewing 11 randomized controls and other reviews of these controls, this team concluded that:
+ there was limited evidence that using freebase nicotine e-cigarettes with clinical support was more effective than NRT or nothing;+ there was no evidence that these e-cigarettes worked outside of clinical settings;+ there was insufficient evidence that freebase nicotine e-cigarettes were effective when used outside a clinical context;+ there was no evidence on the usefulness of nicotine salt products (found in capsule and disposable cigarettes) for cessation;+ there was insufficient evidence that free-base nicotine e-cigarettes were more effective than those with no nicotine.
There are now more than a dozen systematic reviews and meta-analyses looking at a smaller number of randomized control trials comparing e-cigarettes with NRT or no intervention to help smokers quit. (A list of these reviews can be downloaded here.) In this case, the hierarchy of evidence looks more like a cube than a pyramid!
Generally these reviews concur that e-cigarettes in clinical settings (often with counselling or other support) have performed as well or better than NRT. They often disagree about how strong this evidence is, or its generalizability to real-world circumstances, or the usefulness of these experiments for clinicians, policy-makers or and smokers.
Among these, the greatest enthusiasm is expressed by the Cochrane Tobacco Addiction Group.[3] Where the Australians and others have concluded there is “limited evidence” that e-cigarettes outperform NRT, the Cochrane group finds “high certainty evidence”. (Our report on their review can be found here).
In addition to the Banks’ team review, the three other reviews published over the last year were much more cautious. A French research team noted that “there have only been few studies carried out, which does not allow for an affirmation and recommendation of practice.” [4] A Chinese research team characterized the evidence on the effectiveness of e-cigarettes as “low certainty.” [5]
Explaining the difference
How can similarly-trained researchers consider the same body of evidence and produce such different conclusions? Factors that may play a role are the independence of the reviewers, their choice of studies to include and exclude, the role of the tobacco industry, the preferences of publishers, and geography.
Independence from primary studies
Dr. Banks’ team explained its concerns with the Cochrane analysis: “We rated it less independent than other reviews because four of its authors were also investigators in the included trials … The review inclusion criteria were broader than ours, including self‐reported cessation outcomes and trials not published as peer‐reviewed articles, and the authors generally rated risk of bias in individual trials lower than we did.”
This polite wording points to concerns others have raised about the personal connection of the Cochrane reviewers to the original studies and also their inclusion of a study conducted by tobacco companies.
The Cochrane group selected only five studies for its meta-analysis, three of which they had conducted themselves and one of which was paid for by an e-cigarette manufacturer. Shown below is a table taken from the pay-walled full version of the Cochrane report which has been annotated to identify those studies which were authored by Cochrane reviewers and those which were industry funded .
Independence from tobacco industry
More than one-third of the results of the Cochrane meta-analysis are derived from two arms of a study conducted by the U.K.-based Centre for Substance Use Research (CSUR). This firm is funded by tobacco and nicotine companies. A core part of its business is supporting applications to the FDA related to market authorization for e-cigarettes.
The study in question has not yet been published in any journal. Its results are available only in the form of an abstract (PH 353) presented at the 2021 SRNT Conference.[6] The abstract acknowledges funding to the “E-cigarette/Alternative nicotine products Industry”. The device tested in the trial (myBlu) is manufactured by Fontem, which is a division of the world’s fourth largest tobacco company, UK-based Imperial Brands.
It seems probable that the study was conducted to support an application to the US FDA for permission to market myBlu. Shortly after the results were shared at the 2021 SRNT conference, the U.S. FDA issued a marketing denial order for myBlu. The letter sent by the FDA to Imperial Brands e-cigarette division (FONTEM) provides the reasons for denying the application, but does not identify any studies by name.
In addition to being industry-funded and unpublished, there is another red-flag attached to this study. CSUR failed to prospectively register its trial of myBlu. All those who conduct clinical trials are expected to register them in advance in order to reduce the likelihood that only positive results will be shared. The duty to register is no minor obligation – the World Health Organization describes it as a “scientific, ethical and moral responsibility”. The Cochrane organization (which oversees the Cochrane Tobacco Addiction Group) also expects prospective registration of all trials.
Systemic influences on favourable results for e-cigarettes
Mexican researchers recently reviewed almost 700 articles on electronic cigarettes, assessing the relationship between reported conflicts of interest and the research conclusions of each paper. Their results were published last summer in the Journal of the Pan American Health Organization. [7]. This paper did not consider results of unpublished research like the CSUR study included in the Cochrane review.
This study found that articles in 
which any conflict of interest was reported were almost 5 times more likely to find a positive result for vaping (OR 4.7; 95% CI 2.89-7.65). Papers that were funded by tobacco companies were 30 times more likely to have positive results (OR 29.95; 95% CI 9.84 – 90.98).
They also found other patterns related to geography and editorial selection: papers published in England had more than twice the rate of positive results as those published elsewhere. These results are shown in the table here.
And what about the unpublished results of other trials?
The number of RCT’s comparing the use of e-cigarettes for quitting smoking published by 2021 hovers around a dozen (see, for example, the list collected by the Irish Research Board).[8] But there are a great number of unpublished trials registered – some of which are newly started, but many of which have been underway for several years. The absence of any reports on these trials raises concerns about publication bias.
These unpublished results include trials conducted by the tobacco-industry funded team in Italy. (NCT01979796, registered in 2013; NCT02124187, registered in 2014), as well as a study by Cochrane Tobacco Addiction Group member Peter Hajek (NCT01842828, registered in 2013). A Canadian trial of e-cigarettes for inner-city substance users was registered in 2017 and has also seemingly dropped from view (NCT03249428).
Other large trials in Europe have been completed or are close to completion, but the results are not yet shared. These include the Swiss ESTxENDS trial of 1,246 would-be quitters (NCT03603340), the French ECSMOKE trial of 650 comparison e-cigarettes with varenicline (NCT03630614) and a similar trial of 450 middle-aged heavy smokers in Finland (NCT03235505). A study comparing e-cigarettes and NRT with telephone quitline support among 926 Australians is expected to finish data collection this summer. (ACTRN12619001787178).
And there is more to come. Many new trials have recently been registered. Included in these is a study with 800 Canadian smokers which will compare e-cigarettes with varenicline (NCT05715164).
The results of these trials may contribute to a consensus among independent reviewers of the efficacy of e-cigarettes to help people quit in the short term. They are not designed to address questions about long-term impact.
Clinical trials, real-world experience, professional concerns of health practitioners
E-cigarettes are sold worldwide as consumer products, and nowhere yet are they authorized as medicines or therapeutic devices. The RCTs discussed above focus only using these products in structured clinical settings. Other approaches (longitudinal studies, cross sectional studies) are designed to assess real-world outcomes outside the clinical setting or in the natural history of nicotine use.
Many real-world studies have been conducted and a handful of systematic reviews of these studies have also been published. [9, 10, 11] These reviewers generally concur with the findings of Banks et al, discussed above, that there is insufficient evidence to conclude that e-cigarettes are effective in non-clinical circumstances. The results of cohort studies, shown below, is taken from a meta-analysis conducted by a Swedish research team
Evidence based medicine relies on approved products
Using a system designed to assess therapeutic drugs to evaluate consumer product use creates additional complexities. In many countries, clinicians are bound by codes of practice which require them to recommend drugs whose safety and efficacy has been established by health authorities. This is not the case with e-cigarettes. (Australia’s requirement for e-cigarette prescription is based on that country’s special rules for “unapproved therapeutic goods”).
In short – the evidence in support of clinical use of e-cigarettes is of limited value to clinicians who are professionally obligated to recommend medications that have been evaluated and authorized and for whom evidence for different formulations and dosages is expected .
This may contribute to the assessment by the leading health agencies that e-cigarettes should not be recommended as smoking cessation interventions. This is the advice of the World Health Organization, the U.S. Surgeon General, the U.S. National Institute on Drug Abuse, U.S. Centre for Disease Control and Prevention, Direccion General de Salud Publica (Spain), Haut Conseil de la Santé Publique (France), the Instituto Nacional de Salud Pública (Mexico) and many others. A list of statements by these and the organizations shown below can be downloaded here. This week another organization – the Academy of American Pediatrics– joined the list.
Downloads:
Presentation: Evidence of e-cigarettes for cessation
Fact sheet: Systematic reviews and meta-analyses of electronic cigarettes for smoking cessation published since 2020
References
[1] Banks E AM, Yazidjoglou A, Brown S, Nguyen M, Martin M, Beckwith K, Daluwatta A, Campbell S, Joshy G. Electronic cigarettes and health outcomes: umbrella and systematic review of the global evidence. Med J Aust. 2023 Mar 20. doi: 10.5694/mja2.51890. Epub ahead of print. PMID: 36939271.
[2] Banks E, Yazidjoglou A, Brown S, Nguyen M, Martin M, Beckwith K, Daluwatta A, Campbell S, Joshy G. Electronic cigarettes and health outcomes: systematic review of global evidence Report for the Australian Department of Health. National Centre for Epidemiology and Population
[3] Hartmann-Boyce J, Lindson N, Butler AR, McRobbie H, Bullen C, Begh R, Theodoulou A, Notley C, Rigotti NA, Turner T, Fanshawe TR, Hajek P. Electronic cigarettes for smoking cessation. Cochrane Database Syst Rev. 2022 Nov 17;11(11):
[4] Vanderkam P, Bonneau A, Kinouani S, Dzeraviashka P, Castera P, Besnier M, Binder P, Doux N, Jaafari N, Lafay-Chebassier C. Duration of the effectiveness of nicotine electronic cigarettes on smoking cessation and reduction: Systematic review and meta-analysis. Front Psychiatry. 2022 Aug 4;13:915946. doi: 10.3389/fpsyt.2022.915946. PMID: 35990084; PMCID: PMC9386078.
[5] Li J, Hui X, Fu J, Ahmed MM, Yao L, Yang K. Electronic cigarettes versus nicotine-replacement therapy for smoking cessation: A systematic review and meta-analysis of randomized controlled trials. Tob Induc Dis. 2022 Oct 20;20:90. doi: 10.18332/tid/154075. PMID: 36339933; PMCID: PMC9582581.
[6] Russell C, McKeganey N, Katsampouris E, Satchwell A, Haseen F. A randomised community-based trial of a closed-system pod evapour product and nicotine replacement therapy for cigarette abstinence and reduction. Society for Research on Nicotine and Tobacco (SRNT) 2021 Annual Meeting February 24-27 2021 Virtual:PH-353 p230.
[7] Vidaña-Perez D, Reynales-Shigematsu LM, Antonio-Ochoa E, Ávila-Valdez SL, Barrientos-Gutiérrez I. The fallacy of science is science: the impact of conflict of interest in vaping articles. Rev Panam Salud Publica. 2022 Jun 10;46:e81. doi: 10.26633/RPSP.2022.81. PMID: 35702716; PMCID: PMC9186096.
[8] Quigley JM, Walsh C, Lee C, Long J, Kennelly H, McCarthy A, Kavanagh P. Efficacy and safety of electronic cigarettes as a smoking cessation intervention: A systematic review and network meta-analysis. Tob Prev Cessat. 2021 Nov 22;7:69. doi: 10.18332/tpc/143077. PMID: 34877438; PMCID: PMC8607936.
[9] Hedman L, Galanti MR, Ryk L, Gilljam H, Adermark L. Electronic cigarette use and smoking cessation in cohort studies and randomized trials: A systematic review and meta-analysis. Tob Prev Cessat. 2021 Oct 13;7:62
[10] Zhang YY, Bu FL, Dong F, Wang JH, Zhu SJ, Zhang XW, Robinson N, Liu JP. The effect of e-cigarettes on smoking cessation and cigarette smoking initiation: An evidence-based rapid review and meta-analysis. Tob Induc Dis. 2021 Jan 13;19:04
[11] Wang RJ, Bhadriraju S, Glantz SA. E-Cigarette Use and Adult Cigarette Smoking Cessation: A Meta-Analysis. Am J Public Health. 2021 Feb;111(2):230-246. doi: 10.2105/AJPH.2020.305999. Epub 2020 Dec 22
