In the Compass series, I often return to a simple idea. Communities do not fail because they lack data. They struggle because they are asked to trust data that is not fully understood.

This week, I was struck by a piece from Doug Dawson reflecting on how we interpret surveys in the broadband industry. It raises a question that sits at the center of my work as a data scientist. How much of what we present as insight is actually supported by the strength of the underlying data, and how much is confidence layered on top of uncertainty? [1]

The Quiet Risk Behind Small Samples

One of the most counterintuitive truths in survey work is that relatively small sample sizes can appear statistically valid. Under standard assumptions, a few hundred responses can represent populations in the tens or even hundreds of thousands within a defined margin of error. [1]

That technical truth often becomes a narrative shortcut.

In practice, especially in the kinds of community and infrastructure contexts we work in, small samples rarely behave as cleanly as the math suggests. The assumptions behind those margins matter just as much as the numbers themselves. Respondents must be randomly selected. The population must be well understood. Nonresponse must not introduce systematic bias.

As Dawson points out, many surveys do not even explain how respondents were selected. [1] That alone should give us pause.

From my perspective, the danger of a small sample size is not simply that it might be wrong. The greater risk is that it can be precise in presentation and misleading in reality. A single subgroup that is underrepresented, or a small cluster of similar respondents, can disproportionately shape the outcome.

In broadband and digital equity work, that often means the people we most need to hear from are the least likely to be counted.

When Statistical Validity Meets Community Reality

I spend a lot of time working at the intersection of data, infrastructure, and lived experience. Whether it is a feasibility study, a grant application, or a regional planning effort, survey results frequently become the evidence base for major decisions.

Yet many of those datasets come with blind spots.

Dawson highlights three critical elements of a reliable survey. Who is surveyed, how many respond, and how the questions are constructed. [1] All three introduce potential distortion.

A survey distributed through an online platform will miss households without reliable connectivity. A voluntary response survey will overrepresent those who are already engaged. A poorly worded question can nudge respondents toward a preferred answer, even if unintentionally. [1]

Individually, each of these issues is manageable. Together, they can fundamentally reshape what the data appears to say.

This is why I increasingly view survey results not as measurements, but as signals. They point us in directions. They do not define the landscape.

The Ethics of Presenting What We Do Not Know

There is another dimension to this conversation that often goes unspoken. It is not just about statistical rigor. It is about ethical communication.

My grounding in this comes directly from my time in the University of Michigan School of Information MADS program, where courses in experimental design, communicating data science results, and data science ethics emphasized the responsibility that comes with interpreting and presenting data.

Experimental design reinforced how sensitive outcomes are to sampling choices, assumptions, and structure. Communication coursework pushed beyond accuracy into clarity, teaching that how results are framed can shape decisions as much as the results themselves. Ethics coursework made the stakes explicit. Data is never neutral when it is used to inform policy, allocate resources, or define community needs.

Those lessons surface in moments like this.

Dawson notes that many published surveys do not disclose the full methodology, the total universe of respondents, or even the exact questions that were asked. [1] When that happens, the audience is left with a conclusion but not the context needed to evaluate it.

From my perspective, that is where the ethical line begins.

Presenting data without its uncertainty does not make it stronger. It makes it less honest. And in community-facing work, where data informs funding decisions, policy direction, and long-term infrastructure, that lack of transparency can have real consequences.

From Precision to Trust

What I have learned over time is that decision-makers are not actually asking for certainty. They are asking for confidence they can understand.

That is a different obligation.

It means explaining not just what the data shows, but how stable that result is. It means being clear about who is missing from the dataset. It means acknowledging where assumptions may not hold. It also means resisting the urge to overstate conclusions for the sake of alignment or momentum.

In the Compass framework, this connects directly to capacity. A community’s ability to act on information depends on how well it understands both the insight and its limits.

When we are transparent about uncertainty, we are not weakening the narrative. We are strengthening trust.

A Better Way Forward

Surveys remain one of the most valuable tools we have. They can surface trends, capture sentiment, and help prioritize action. Dawson makes an important distinction here. Even non-random surveys can be useful for tracking changes over time, even if their point estimates should not be taken at face value. [1]

The opportunity in front of us is not to use surveys less. It is to use them more responsibly.

That means:

  • Designing outreach strategies that intentionally include underrepresented groups
  • Pairing quantitative results with qualitative insight from direct community engagement
  • Treating percentages as approximations rather than definitive facts
  • Clearly documenting methodology, assumptions, and limitations

Most importantly, it means shifting our mindset from proving a point to informing a decision.

Closing Reflection

If there is a single takeaway from reflecting on Dawson’s piece, it is this. Data does not speak for itself. We speak for it.

And with that comes responsibility.

Small sample sizes, biased questions, and incomplete methodologies are not just technical issues. They shape the stories we tell and the choices communities make. When we present survey results without fully representing their uncertainty, we are not just simplifying. We are distorting.

In the work ahead, especially as we continue building digital infrastructure and community capacity, our goal should not be to eliminate uncertainty. That is not possible.

Our goal should be to make it visible, understandable, and actionable.

Because in the end, trust is not built on precision alone. It is built on honesty about what we know, what we do not know, and how we choose to move forward anyway.

Sources