Summary

  • NANOG 55 reported 525 registrants and a 14 percent survey participation rate; later notices often reported response counts or targets without the same denominator.
  • Survey feedback can improve a conference without being representative. Programme-facing claims become more useful when the invitation population, response unit, question base and resulting decision are recorded.

The archive already shows the difference

In 2012, NANOG’s account of NANOG 55 gave readers both 525 registrants and a 14 percent survey participation rate. It summarized reasons for attending and operational preferences, said comments were shared with speakers, and noted suggestions for later meetings. The figure was still meeting-specific, but it had a visible frame.

The NANOG 93 closing message used an approximate comparison: just over 100 responses against more than 700 attendees. It also disclosed that forms mixed speaker feedback with questions for staff planning and that respondents could skip questions. That matters. A submitted form is not automatically an answer to every item.

NANOG 96 then reported 97 responses across three days and hoped to reach 200. NANOG 97 reported 147 with the same goal. These are useful operational updates, yet the cited notices do not say whether “responses” means unique people or daily forms, how many people received an invitation, or the base for each question.

A programme trace is visible, but not a method

The NANOG 98 call for content makes the governance relevance concrete. It published topics drawn from recent surveys and used plus signs to mark repeated requests. Survey material had reached a programme-facing decision surface. The notice did not publish question wording, respondent composition, the count behind each plus sign, or which request changed which agenda choice.

That is not an accusation of bad polling. Event surveys are service tools. The narrower point is that a number should not acquire a representative meaning merely because it becomes visible in programme planning.

A lightweight methods note is enough

For each meeting, NANOG could preserve the invitation population, fielding window, unique respondents, daily-form rules, completion by question, optional participant type, free-text coding method and a short decision ledger. Missing fields should remain missing. Registration, attendance, invitation, response, answer and programme change are separate events.

Heng Lu’s doctrine is only an analytical lens for keeping constituency, evidence, decision and outcome apart. It is not evidence about NANOG’s practice.

Sources and limits