When can you trust your campaign numbers?

Your QR campaign dashboard reports a 12% scan-to-conversion rate. Good news, until you notice it came from 25 scans and 3 conversions. Would that rate survive if 2,500 people scanned instead? That is the question hiding behind every optimization decision: when can you trust your campaign numbers? Get this wrong and you kill a working campaign over random noise, or you scale a loser because a tiny sample flattered it. This article walks through the three levers that decide whether a number is real (confidence level, margin of error, and sample size), how big your sample actually needs to be, and the analytics trap that silently distorts the figures you already report.

Why Small Numbers Lie to You

A single scan can swing a small campaign's headline metric by several points. If 20 people scan a table tent and 2 redeem the offer, that is a 10% redemption rate. One more redemption and you are at 15%. One fewer and you are at 5%. Nothing about your campaign changed. The audience did not get warmer or colder. Only the dice landed differently.

This is the core reason early campaign numbers feel volatile: with few observations, chance dominates. The pattern you see is mostly randomness wearing the costume of a trend. Marketers who react to it end up rewriting copy, swapping colors, and reprinting flyers to chase movement that was never real.

Here is an obviously hypothetical worked example to make the point. Say you run two QR flyers. Flyer A gets 40 scans and 6 signups (15%). Flyer B gets 38 scans and 3 signups (about 8%). It looks like A is nearly double B. But at those sample sizes, the gap could easily reverse next week. You are looking at coin-flip variation, not a winning creative. The fix is not a better gut feeling. It is knowing how much data you need before the difference counts as real, which statisticians call statistical significance.

The Three Levers That Decide If Your Campaign Numbers Are Real

Whether a number is trustworthy comes down to three settings you choose in advance, plus the underlying rate you are measuring. Practitioners on MarketingProfs lay these out clearly for anyone sizing a survey or a test, and the same logic governs scan and conversion data.

The first lever is the confidence level: how sure you want to be that your result reflects reality. Ninety-five percent is the standard, meaning if you repeated the campaign many times, the true value would fall inside your range 95 times out of 100. The second lever is the margin of error: how much slack you accept around the number. A result of 12% with a 5% margin really means somewhere between 7% and 17%. The third lever is sample size: the number of scans, clicks, or responses you collect. Larger samples shrink the margin of error and raise your confidence.

There is a fourth factor most marketers miss. According to the MarketingProfs discussion, the required sample also depends on the probability of the outcome you are measuring, and the worst case (the largest sample requirement) sits at 50%, the coin toss. Because you rarely know the true rate before you measure it, designing for 50% is the safe default. If the real rate turns out higher or lower, your error will be smaller than planned, not larger.

How Big Does Your Sample Actually Need to Be?

The reassuring news is that required sample sizes are smaller than most people expect, and they stop growing once your audience gets large. According to the MarketingProfs research discussion, a customer base of 10,000 needs about 370 responses for 95% confidence with a 5% margin of error. That is not 370 per thousand. It is 370 total to speak confidently about all 10,000.

Sample size climbs slowly as the population grows and then flattens. A second MarketingProfs answer puts the ceiling in perspective: the largest sample required for 95% confidence with a 5% margin of error is 385, and that holds for populations at or above 468,000. In other words, whether your reachable audience is half a million or fifty million, roughly 385 clean observations gets you to the same confidence. Scale does not keep raising the bar.

The rate you are measuring nudges the number too. The same source notes that at a 65% response probability the required sample drops to about 350, versus 385 at the 50% worst case. And your tolerance for error matters most of all. Loosen the standard to a 10% margin at 90% confidence and, per MarketingProfs, the sample size for a 10,000 population falls to just 68. That is the practical trade you are always making: tighter certainty costs more data, looser certainty gets you a fast read you should hold loosely.

• Population 10,000, 95% confidence, 5% margin: about 370 observations (MarketingProfs).
• Population 468,000 or larger, 95% confidence, 5% margin: about 385 observations (MarketingProfs).
• Population 10,000, 90% confidence, 10% margin: about 68 observations (MarketingProfs).

What a Confidence Interval Really Tells You

A single percentage on a dashboard is a point estimate. The honest version of that number is a range. When the European Commission's SFC portal defines a reliable representative sample, it does not report a bare figure. It requires values with a margin of error no greater than 3 percentage points at a 95% confidence level, which produces a confidence interval 6 percentage points wide. So a measured 40% is really reported as roughly 37% to 43%.

Apply that discipline to your own campaign numbers. If your scan-to-redemption rate reads 9% with a 3-point margin, the true rate lives somewhere between 6% and 12%. That range is the number you should act on, not the tidy 9% in the center. When two variants have overlapping ranges, you do not yet have a winner. You have two results that could be the same thing.

This reframes the trust question in a useful way. You are not asking whether the number is exactly right. You are asking how wide the range is and whether that range still supports your decision. A wide interval on a cheap test can be fine if both ends of the range clear your break-even threshold. A narrow interval matters most when the decision is close and the reprint bill is large. Report the interval, not just the headline, and every stakeholder conversation gets more honest.

The Google Analytics Trap: Sampled Data

Even after you collect enough data, the tool reporting it can quietly distort the picture. Many analytics platforms use sampling to keep reports fast, and Google Analytics is the one most QR marketers rely on for the landing-page side of the funnel. According to Moz, sampling means the platform analyzes a small, randomly selected subset of your data rather than every event, then extrapolates to the whole.

Moz identifies three common attitudes toward this, and calls all three misguided: marketers who fear sampling and demand unsampled reports for everything, marketers who trust the statistical logic without question, and marketers who have no idea it is happening at all. The takeaway is not panic. It is awareness. As Moz puts it, sampling is not something to fear, but in Google Analytics in particular it cannot always be trusted.

The practical risk shows up on your smallest, most interesting segments. High-level totals across a big date range usually survive sampling fine. But drill into one QR campaign, one city, or one device type over a short window and the subset behind that number can get thin enough to wobble. Before you make a call on a narrow slice, check whether the report was sampled, widen the date range, or pull the raw scan counts from your QR platform where every scan is counted rather than estimated.

Your Sample Must Look Like Your Audience

Sample size answers how many. Representativeness answers who. A large sample drawn from the wrong slice of your audience is still a misleading sample. The SFC portal is explicit on this point: a representative sample must reflect the characteristics of the population it describes across defined variables, in its case employment status, age group, and education level. The method of selection must be documented, not improvised.

Campaign data has the same failure mode. Suppose your QR codes run on both a downtown storefront window and a rural flyer drop. If 90% of your scans come from the downtown window, your blended conversion rate mostly describes downtown foot traffic, not your full audience. It can be perfectly precise and still answer the wrong question. This is why unique codes per location or per channel matter: they let you see whether each segment behaves differently before you average them into one number that represents no one.

Timing skews samples too. Scans collected only during a launch-day promotion capture your most eager audience, not your steady-state customer. A weekday-only sample misses weekend behavior. Before trusting a rate, ask whether the scans behind it were drawn across the mix of places, times, and devices you actually care about. If they were not, segment first, then judge each segment against its own sample size.

When to Trust the Number and When to Wait

Put the pieces together and the trust decision becomes a short set of checks rather than a gut call. The question is never simply is this number good. It is: is this number backed by enough representative observations, reported with an honest range, from a tool that counted rather than estimated?

Use these thresholds as a working guide. They favor caution on expensive, hard-to-reverse decisions and allow faster reads on cheap experiments where a loose margin is acceptable.

• Trust it for a big decision: you have roughly 350 to 385 clean observations per variant, the confidence intervals do not overlap, and the data was not sampled by your analytics tool (MarketingProfs, Moz).
• Trust it loosely for a cheap test: you have around 68 observations at 90% confidence and a 10% margin, and both ends of the range still clear break-even (MarketingProfs).
• Do not trust it yet: fewer than a few dozen conversions, overlapping ranges between variants, or a single location or day standing in for your whole audience (SFC representative-sample standard).
• Verify before acting: any narrow segment pulled from Google Analytics, where sampling can distort small slices; confirm with raw scan counts from your QR platform (Moz).

A Simple Pre-Trust Checklist for Your Dashboard

Before you scale, kill, or reprint anything, run the number through five questions. First, how many conversions (not just scans) sit behind this rate? A rate built on single-digit conversions is a headline, not a finding. Second, what is the margin of error, and does the decision still hold at both ends of that range? Third, was this figure sampled by the analytics tool, and would it change if you widened the date range?

Fourth, is the sample representative of the audience you are deciding for, across location, channel, time, and device, or is one segment dominating the average? Fifth, have you set your confidence level and acceptable margin before looking at the result, so you are testing a hypothesis rather than rationalizing whatever the dashboard happened to show? Answer those five honestly and you will trust the right numbers and ignore the noise that fools everyone else.

Trustworthy campaign numbers are not the ones that look best. They are the ones backed by enough representative observations, reported as a range, and counted rather than estimated. Roughly 350 to 385 clean conversions per variant gets you to serious confidence, and even a large audience rarely demands more. This week: pull your top-performing QR campaign and write down the raw conversion count behind its headline rate, not the percentage. If it sits under a few dozen conversions, or if the scans came from one location or one day, pause the decision and keep collecting for seven more days before you scale or reprint anything.

Start measuring your own campaigns

QRlytics tracks every scan with location, device, and browser data, so you can see which printed pieces actually pay off. The free plan covers your first codes at no cost, and Pro unlocks unlimited dynamic codes and full scan history when you are ready to scale. See all options on the plans page.