The ad campaign received mixed analysis on its effectiveness. Using data from the Youth Risk Behavior Surveys, a 2015 analysis found that "when accounting for a preexisting downward trend in meth use, [Montana Meth Project's] effects on meth use are statistically indistinguishable from zero." A study was published in the
Journal of Marketing Research validating the effectiveness of the Meth Project's advertising in deterring substance abuse. The researchers tested the effectiveness of several advertisements—including the Meth Project's—and found that ads that relied on fear alone to convey their message did not lead to immediate changes in attitudes or behavior. However, according to the study, the Meth Project ads that incorporated an element of “disgust,” such as rotting teeth, skin sores or infections, did compel viewers to “undertake distancing behaviors,” such as deciding not to use illegal drugs. in 2010, the Meth Project was named the third most effective philanthropy in the world, up from #5 in 2009 on Barron's yearly rankings. In its efforts to effectively reach teens and change attitudes and behaviors toward meth, the MMP regularly conducts focus group research to refine its messaging and better understand how to connect with the state's youth.
HBO has also partnered with the MMP on a documentary as part of its Addiction series. Two surveys have been conducted that have investigated methamphetamine use amongst teenagers in Montana before and after the launch of the Meth Project's ads. The first survey is the CDC's
youth risk behavior survey (YBRS). The YRBS data are listed below. YRBS Data - Percentage of Montana Teens who have ever used meth: However, the YRBS data also shows that meth use was dropping for at least 6 years prior to the launch of the ad campaign. The other survey of teen meth use has been conducted by the Meth Project. In contrast, the 2007 and 2008 MMP data was weighted and compiled from 2,335 and 2,334 participants, respectively. In press materials, the Meth Project commonly cites YRBS figure of a 45% decrease in meth use between 2005 and 2007. However the
absolute drop for the period was 3.7%. In contrast, the Meth Project's own data for the same period show a 2%
absolute increase in meth use, or a
100% relative increase. since the inception of the program in 2005, there has also been a 72% relative decrease in adult methamphetamine use, and a 62% relative decline in methamphetamine-related crimes. Additionally, the percentage of teenagers who are aware of meth's dangers increased from 25% to 93%, and Montana's ranking among U.S. states in meth abuse fell from #5 to #39.
Office of National Drug Control Policy report In November 2006, the
Office of National Drug Control Policy (ONDCP) published a report,
Pushing Back Against Meth: A Progress Report on the Fight, highlighting the impact of recently enacted State and Federal laws, such as the Combat Methamphetamine Epidemic Act (CMEA) of 2005, that restricted transactions for the
over the counter drugs that can be used to manufacture methamphetamine. Based on the results of
Quest Diagnostics' preliminary review of workplace drug tests conducted during the first five months of 2006, the nationwide adult usage of meth declined by 12% when compared to the same period in 2005. "Montana’s methamphetamine precursor law went into effect July 1, 2005. The Montana law is stricter than the CMEA in several important respects." and the MMP in December 2008. The review examined the Meth Project's statistical methodology and
data reporting. The review found that the Meth Project had selectively reported their research findings, focusing on unrepresentative positive findings and ignoring data suggesting that the campaign may be associated with harmful outcomes. The review found that the Meth Project's data suggests that exposure to the graphic ads may lead to an
increase in the percentage of teenagers who believe that taking meth is socially acceptable and
not dangerous. Such 'boomerang' effects in response to persuasive, graphic ads are not uncommon, and are predicted by the theory of
psychological reactance. The review found that the selective reporting of results by the Meth Project has led the media, politicians and the public to form distorted and inaccurate beliefs about the campaign's effectiveness. The public believes that the ad campaign is far more effective than the Meth Project's research findings indicate. The main recommendation of the review was that public funding and additional roll-outs of the program should cease until its effectiveness can be scientifically examined. The review concluded: "Politicians, the media, and prevention researchers also need to ensure that in future they critically evaluate any research released by the MMP, rather than assuming the organization’s press releases (and reports) are presenting data in a fair and balanced way. It is recommended that any future reports documenting the results of MMP’s use and attitudes surveys include complete statistical analyses for every question in the survey. This is because researchers and policymakers making decisions about MMP-style graphic advertising campaigns need access to all evidence, rather than a subset of findings that portray the MMP in a positive light.” ==Government funding and expansion==