Summary

  • NANOG’s public material establishes a real feedback channel: daily meeting surveys, post-event prompts, an open community-meeting invitation, and a Program Committee call that displayed examples of requests drawn from recent answers.
  • The record does not establish that a response total represents all attendees, that the displayed topics were ranked preferences, or that a particular request caused a particular agenda decision. The missing public layer is a privacy-preserving feedback-to-programme ledger.
  • A proportional remedy would publish collection windows, eligible audiences, response totals by attendance mode, questionnaire versions, aggregate themes, and an action, non-action, or deferral record—without publishing names, raw comments, employer data, or small cells.

A small word with a large institutional consequence

“We asked” is not the same proposition as “the community decided.” “One hundred people answered” is not the same proposition as “one hundred people represent everyone present.” And “these requests appeared in a call for presentations” is not the same proposition as “the programme adopted those requests.”

Those distinctions can look pedantic until an organisation begins to use feedback as evidence of openness, responsiveness, or agenda relevance. At that point, the public record needs enough structure for readers to understand what feedback was collected, how much of the available audience it reached, how it was interpreted, and where it did or did not change a decision. The aim is not to turn a technical meeting into a referendum. It is to prevent a survey from acquiring a mandate it cannot bear.

NANOG’s published material supplies the beginning of that record. It shows repeated invitations to answer daily and post-event surveys. It distinguishes feedback intended for speakers from feedback relevant to staff planning future events. It shows the Program Committee publicly presenting examples of requests while asking for future presentation proposals. It also contains a rare contemporaneous comparison between survey answers and meeting attendance. Those are meaningful facts. They show that feedback is sought and can move into the public framing of a programme call.

They are not, however, a complete accountability trail. The relevant fields are not published in the reviewed source set: a recurring denominator by attendance mode, questionnaire wording or version, aggregate response distribution, coding method, ranked topics, or a dated explanation of which programme choices followed, did not follow, or were deferred after feedback was reviewed. That absence should be stated carefully. It is an absence from the reviewed source set, not proof that NANOG lacks internal analysis, ignores comments, or makes decisions in bad faith.

The distinction matters because a feedback system can be useful even when it is incomplete. An open comment box may surface a problem that a carefully sampled questionnaire misses. A daily prompt may catch an operational issue while it can still be fixed. Professional programme review can properly reject a popular request if the proposal pool, technical quality, speaker availability, time, and room constraints point elsewhere. The question is not whether every answer should control an agenda. It is whether the public can see enough of the route from solicitation to decision to evaluate claims about listening.

The clearest denominator is also the most limited one

The most concrete snapshot in the reviewed material comes from a NANOG 93 follow-up message. NANOG asked attendees to complete a survey and said that, at the time of the message, it had received just over 100 responses while the meeting had over 700 attendees. The message separated questions that give feedback to speakers from questions relevant to staff planning future events, said that no one had to answer every question, and set a closing time. [S01]

This is better evidence than a generic assertion that a survey existed. It establishes an active feedback channel, a time-bound response count, a larger time-bound attendance count, and more than one intended use for answers. It also demonstrates a useful disclosure instinct: the organisation did not present the response total without any contextual audience number.

But the comparison is not a final response rate. Both quantities are approximate. The message supplies no final total after the deadline, no count of people who received an invitation, no count of people eligible to answer a particular form, and no breakdown between those present in the room and those using a signed-in web platform. It does not tell readers whether the same person could answer more than one daily form, whether a response was partial or complete, or whether any particular question was optional. It does not describe the makeup of the respondents.

It does not state what share of them attended remotely, in person, or through both channels.

The only safe conclusion is modest but important: at the time stated, the number of answers was materially smaller than the number of attendees stated. That is a signal, not a census. It tells a reader that survey evidence existed; it does not establish the views of the whole meeting.

Why insist on that boundary? Because denominator errors quietly change the political meaning of a number. A response count can become “the community’s view” in a slide, a meeting conversation, or a programme justification without anyone ever saying the words. Once that happens, nonresponse is treated as silent consent, and a useful input becomes an implied authorisation. Neither inference follows.

There is an opposite error as well. A small or incomplete count does not make feedback worthless. A response can identify a broken room setup, an inaccessible scheduling conflict, a subject that deserves a better call for proposals, or a recurring concern about event design. It can do so without representing every person in the building. The correct public posture is neither dismissal nor inflation. It is to describe the measurement honestly and to retain the route by which it was used.

Daily collection is a practice, not a measurement design

The NANOG 93 notices show that daily surveys were not a single afterthought. Monday, Tuesday, and Wednesday notices directed attendees to daily surveys. The same meeting had in-person attendance and a signed-in web platform; one notice also described webcast chat and live questions. [S01]

That structure has practical advantages. Daily feedback can be closer to an experience than a post-event recollection. It can separate a problem on one day from a general opinion about the meeting. It can give operations staff time to adjust a later day. It can also give speakers a narrow channel for learning what their sessions delivered to those who chose to answer.

Yet daily collection raises questions that a link alone cannot answer. Were the forms the same each day? Did a daily link go to every attendance channel in the same way? Were people asked about a session they watched, a room they entered, the online experience, the general event, or a mixture? Could one person answer repeatedly? Were reminders sent, and if so, to whom? Did the collection window remain open long enough for remote viewers in different time zones? Were free-text comments read alongside ratings, and how were duplicate or incomplete answers handled?

None of those questions presumes a defect. They are the ordinary questions that determine what a number means. A daily tally without an eligible audience may be a useful operational count, but it cannot be read as a day-specific response rate. A daily score without a published questionnaire version may measure something real, but readers cannot know whether its wording changed. A theme extracted from free text may identify an emerging issue, but without a coding description readers cannot know whether it was a single vivid comment, a recurring phrase, or a staff-created grouping.

NANOG 94 makes the recurring collection pattern more visible. Its survey page labelled separate Monday, Tuesday, and Wednesday links with their opening dates and said that feedback was critical to the success of NANOG’s peer-reviewed programme. [S02] This supports two limited propositions: feedback was solicited by day, and NANOG publicly connected feedback to programme quality. It does not, in the reviewed source set, publish the forms’ contents, their totals, their denominators, their results, their coding rules, the proposals considered, or the programme changes made afterward.

The distinction is especially important because “critical” can be an aspiration, an operational description, or a public-value statement. It need not mean that a score mechanically changes a schedule. A careful public record should let readers see how the word is being used. Is feedback one input in a programme review? Is it primarily an event-service instrument? Does it trigger a discussion only above a stated threshold? Does it inform the next call rather than the current agenda? Each answer can be reasonable. What is difficult to assess is a broad claim paired with no traceable route from question to disposition.

A topic example is not a preference ranking

The strongest public connection between answers and future programming appears in an April 2025 Program Committee call for NANOG 94 presentations. The call invited proposals for in-person or live-remote presentations. It said that at every NANOG conference people are asked what they would like to hear at a future conference, then reproduced examples from recent answers: hardware, career development, AI, network automation, routing, residential broadband, IPv6, and performance. It directed prospective speakers to the Program Committee Tool and gave a proposal-and-agenda timeline. [S03]

That disclosure is more useful than a statement that “we listen.” It gives potential speakers a window into topics that recent respondents raised. It may help widen the proposal pool by telling knowledgeable people that their expertise has an audience. It may also help readers see that feedback is not confined to room logistics or speaker ratings.

But the list must be read as a list of examples, not a frequency table. The call does not say how many answers produced any one theme. It does not say whether the themes were chosen because they were most common, most urgent, most technically timely, easiest to describe, or simply illustrative. It does not disclose whether the responses came from one meeting or several, which attendance modes they represented, whether themes overlapped, or how open-text answers were combined.

It does not state that a cited theme became a session, that a proposal under one of those headings was received, or that an accepted talk was selected because an answer mentioned it.

Those limits are not semantic trivia. They protect both sides of the relationship. A reader should not infer that a theme absent from a call was ignored; it may have appeared in a different public channel, fit a different programme format, lacked a viable proposal, or been deferred. A prospective speaker should not infer that using a displayed phrase confers an advantage. And the Program Committee should not be boxed into defending every agenda choice as a direct poll result when its published process calls for proposals and review.

The useful claim is therefore specific: NANOG publicly carried examples of recent requests into the framing of a call for proposals. The unsupported claim is causal: that a survey answer produced a particular session. An accountable record should preserve the first claim while refusing to manufacture the second.

Feedback has more than one route into an organisation

Survey links are not the entire feedback landscape in the reviewed record. NANOG 83’s event page invited feedback on recent organisational developments in a Community Meeting described as open to all. The listed topics included current and future programming by Program Committee leadership. [S07]

That matters because some questions are better addressed in a public meeting than in a form. An open session can expose disagreement, allow a follow-up question, and reveal that a concern has multiple interpretations. It also changes the kind of evidence created. A survey can produce aggregate counts, structured responses, and optional comments. A community meeting can produce an invitation and perhaps a discussion, but not necessarily a durable result unless the organisation records one.

The reviewed event page proves an invitation, not a transcript, attendance count, vote, agenda-change record, or consensus. It would be a mistake to turn “open to all” into a claim that every interested person attended or that a speaker’s statement represented everyone present. Openness of invitation and breadth of contribution are distinct variables.

The same caution applies to older daily notices. A NANOG 90 attendee notice included daily surveys among meeting and agenda information. [S05] It supports the proposition that the daily-survey practice predates the NANOG 93 and NANOG 94 materials reviewed here. It does not supply a result distribution, an eligible population, or a feedback-to-decision path. Repetition of an invitation is evidence of institutional practice; it is not evidence that every collection round produced a comparable measure.

Looking across these sources, the appropriate description is plural: NANOG has used survey prompts, daily meeting notices, post-event follow-ups, a public community-meeting invitation, and presentation-call framing. Each route may collect a different kind of input and may carry a different burden of disclosure. The public record is most useful when it says which route is being invoked, what it can show, and what it cannot.

A forward-looking promise is not a retrospective record

In an October 2025 NANOG 95 follow-up, NANOG asked attendees to complete surveys and said that their input drives improvements seen at future events. [S04] As a statement of intent, that is understandable. Event organisers need ways to improve operations, programming, and experience between meetings. People who spend time answering a form reasonably want to know that someone reads it.

Still, “drives improvements” names a relationship that readers cannot evaluate without a record of the intervening steps. Which input was reviewed? What improvement was considered? What other evidence mattered? Who was responsible for deciding? Was an action taken, rejected, delayed, or referred elsewhere? Was the change a room adjustment, a communications change, a programme-call change, a format decision, or something unrelated to the current article’s scope? Without an aggregate record, the phrase remains a commitment rather than an independently testable account of outcomes.

This is not a demand to expose raw comments. Free text can contain identifying details, workplace complaints, vendor disputes, or small-group information. Publishing it may create a privacy problem and may make people less candid. It is also not a demand to make every minor operational adjustment a public governance event. A meeting team needs room to fix ordinary problems quickly.

The relevant public standard is lighter: publish the category of issue, the collection window, the aggregate signal where safe, and the disposition. A line saying that a concern was deferred because it required a later budget decision, or that a programme request was not pursued because no suitable proposal was received, can explain more than a slogan while revealing far less than an unfiltered comment archive.

The value of such a record is reciprocal. It protects NANOG from an unfair inference that an unadopted request was buried. It protects respondents from the equally unfair inference that answering a form guaranteed a result. It also makes it possible to see when a decision had several valid inputs: technical quality, timeliness, speaker availability, format balance, budget, room constraints, event safety, and the programme process itself.

The historical count shows both ambition and restraint

NANOG’s 2019 annual report offers a different, organisation-wide view. The report recorded 82 respondents and six survey winners. It described those surveys as NANOG’s first community- and member-wide surveys, intended to help guide the organisation’s continuing evolution and the resources and experiences that matter to its community. On adjacent programme pages, the report listed 186 presentation submissions and 93 accepted talks for that year. [S06]

The annual report is valuable for what it makes visible. It connects the survey exercise to a stated organisational purpose, reports an aggregate response count, and separately reports programme activity. It therefore avoids one common clarity failure: it does not collapse the respondent total into the submission and acceptance figures.

The document’s restraint should guide later interpretation. Eighty-two respondents is an aggregate count, not a response fraction, because the reviewed source set does not give the eligible survey frame. The figure does not show who answered, what questions they saw, how answers were distributed, or which recommendations arose. The nearby count of presentation submissions and accepted talks does not create a causal bridge from the survey to programme outcomes. It may be tempting to place the figures side by side and draw a story of influence; the source does not establish one.

That limitation does not diminish the survey’s possible value. A modest number of thoughtful answers may reveal a useful concern. A separate programme process may work well without being traceable to a survey. The point is that the public should not have to choose between silence and overclaiming. It can have a compact record that states both: here is the feedback signal, and here is what was done with it, alongside the other inputs that mattered.

Who controls what should be legible

A credible feedback account begins by separating functions that are easy to blur in public prose.

Event and communications staff can determine when a survey is sent, how reminders work, what collection channel is used, where answers are retained, and how an event-level concern is routed. Respondents supply observations, requests, and ratings. Their answers can be useful evidence without becoming instructions. The Program Committee controls the framing of a presentation call and the review of proposals under the process it publishes. Meeting operations and Board functions may have responsibilities for event delivery or budgets, but the reviewed sources do not establish them as programme selectors.

These distinctions are not accusations; they are the minimum conditions for locating responsibility. If a survey question is unclear, the organisation needs a visible owner for wording and versioning. If comments are grouped into themes, readers need to know whether staff, a committee, or an external service performed the grouping. If a topic is displayed in a proposal call, the Program Committee needs room to explain that it was an example, a priority signal, or one input among several.

If a change requires money or a later event decision, the relevant function should be able to record a deferral without suggesting that the feedback disappeared.

Without role separation, every positive outcome can be credited to “NANOG” and every gap can be assigned to a vague institution. That makes praise and criticism equally hard to evaluate. A ledger is useful not because it creates a new hierarchy, but because it attaches each step to a function: collection, analysis, programme review, operational action, or public reporting.

The minimum useful ledger

The remedy need not be elaborate. A meeting-level feedback-to-programme ledger could fit on a compact public page or a dated report. Its value comes from the chain it preserves:

survey version and collection window → eligible audience and invitations → completed responses by attendance mode → question or coding method → aggregate request themes → programme action, non-action, or deferral → stated reason and review date

Each field has a distinct job.

Survey version and collection window. A reader needs to know whether Monday’s form and Wednesday’s form asked the same questions, whether a post-event form added a new question, and when answers were accepted. A version label can be simple. It does not require publishing every internal drafting exchange. It makes later comparison possible and allows a reader to recognise that two totals should not be treated as the same measurement when their questions differ.

Eligible audience and invitations. This field should distinguish the best available audience frame from the count of completed answers. It could say, for example, whether invitations were sent to registered in-person attendees, signed-in platform users, both, or a broader mailing list. It should not invent precision where the system does not support it. If the eligible audience is not measurable for a particular form, the ledger should say so rather than presenting a response total as a rate.

Completed responses by attendance mode. The purpose is not demographic exposure. It is to prevent an in-person response count from silently standing in for every remote viewer, or vice versa. Where a count would create a small-cell privacy risk, the page can suppress or combine it. The essential thing is that readers can see whether a result reflects one channel, multiple channels, or an unknown mix.

Question and coding method. Closed questions need their wording or a stable question identifier. Open text needs a plain explanation of how answers were grouped: whether themes were manually coded, whether more than one reviewer checked the grouping, whether one answer could appear under several themes, and what the minimum publication threshold was. This does not require exposing comments. It explains how a private set of answers became a public aggregate.

Aggregate request themes. A programme call can continue to use readable examples. The ledger should say whether they are examples, common themes, emerging issues, or items selected for illustration. If safe and meaningful, it can provide a count or band. If counts are too small or too easily misread, it can say that a theme was noted without asserting rank. The point is to prevent a curated list from being mistaken for a poll result.

Action, non-action, or deferral. This is the field that turns feedback into an accountable route rather than a promise. It need not say that a request “won.” It can say that a request informed an invitation to propose, that it was addressed through operations, that no suitable proposal was received, that it was deferred to a future meeting, that it duplicated an existing channel, or that it was not pursued. Where a programme choice had several inputs, the ledger should say so.

Stated reason and review date. A short reason lets readers distinguish neglect from a legitimate constraint. A review date prevents a deferral from becoming an unexamined graveyard. It also creates a place for a later correction: a decision initially deferred may become feasible once a speaker, room, budget, or programme cycle changes.

None of these fields requires names, employer data, raw comments, or publication of small cells. In fact, the ledger should expressly suppress them. The appropriate unit of transparency is the decision path, not the individual respondent.

What the ledger would let readers test

The ledger would not settle every dispute. It would make several narrower questions answerable.

First, it would let a reader distinguish an invitation from a completed-response measure. A page that lists a survey link tells us that a channel was offered. A ledger that adds the collection window, eligible audience, and response count tells us what the observed signal actually covers.

Second, it would let a reader distinguish a topic excerpt from a ranked preference. The NANOG 94 call’s examples could remain useful while a linked aggregate record made clear whether they were illustrations, recurring themes, or only part of the response set. That improves the usefulness of the call for potential speakers without converting it into a demand schedule.

Third, it would let a reader distinguish feedback from a programme decision. A request can inform a call, a call can attract a proposal, a proposal can be reviewed, and a session can be scheduled for reasons that include but are not limited to feedback. Those are separate steps. A public disposition log makes the chain visible without pretending that one answer determines the final agenda.

Fourth, it would help identify where a problem belongs. A daily concern about the web platform may be an event-operations issue. A recurring request for a topic may be relevant to a later call. A concern about the general direction of the organisation may belong in a Community Meeting or another governance channel. The appropriate response differs, but the public should be able to see that the issue was routed rather than lost.

Finally, it would create a basis for proportionate challenge. If a reader believes a displayed theme was misleadingly framed, the reader could point to the theme definition and the collection window. If a respondent believes an issue was deferred too long, the reader could ask about the stated review date. That is more useful than demanding that every private comment be revealed or treating every unfulfilled request as proof of disregard.

The strongest case against a heavier disclosure regime

There is a strong counter-case to this article’s proposal, and it should be taken seriously. Survey comments can be personal, commercially sensitive, or unfairly accusatory. A tiny response group can be identifiable even without names. Publishing topic counts may produce false precision, invite lobbying, or encourage people to game a form. Programme committees must preserve room for judgement. They have to consider technical quality, timeliness, speaker availability, format balance, budget, room capacity, and the coherence of a meeting.

A transparent record can itself create administrative work that competes with the work of improving an event.

All of that is plausible. It is why the recommended ledger is not a raw-data release, an automatic agenda mechanism, or a vote. It does not require a committee to accept a proposal because a theme appeared in an answer. It does not ask NANOG to publish a comment that could identify a person or a company. It does not claim that an incomplete response pool can settle a question of representation.

The proposal is intentionally smaller. It asks for a privacy-safe description of the measure, the aggregate signal, and the decision path. If a field cannot be published safely, the ledger can record that a privacy, contractual, or operational constraint applied and describe it at the appropriate level of generality. The point is not to erase judgement; it is to show where judgement entered.

A heavier system could backfire by turning feedback into theatre. If people believe every answer must be visible and every displayed theme must have a public victory or defeat, they may write strategically rather than candidly. Staff may spend more time defending classifications than fixing issues. Programme reviewers may avoid experimentation because any departure from a theme list looks suspect. A compact disposition record avoids these traps by making the boundary clear: feedback informs; responsible functions decide; the public can see a defensible account of how the two met.

What would change the conclusion

This article’s thesis is deliberately falsifiable. It should narrow if NANOG publishes a stable recurring record that contains the eligible audience and completed-response denominator by attendance mode, the questionnaire version and collection window, an aggregate privacy-safe thematic result, an explanation of coding, and a dated action, non-action, or deferral log that identifies other decision inputs where relevant.

Such a record would not need to prove that every attendee agreed with every choice. It would accomplish something more achievable and more important: it would show what the feedback evidence was and how it entered a decision process. If NANOG already has such a public record beyond the reviewed source set, the appropriate conclusion is not that the organisation needs to invent a new system. It is that the public account should be located and assessed against these modest criteria.

The recommendation should also narrow if a proportionate reason makes one field unsuitable for public release. A small attendance mode, a sensitive question, or a contractual obligation may justify aggregation, suppression, or delayed publication. In that case, the ledger should say enough to explain the limitation without turning privacy into a blank space where no accountability is possible.

Conversely, the public record would become weaker if it repeatedly invoked feedback as proof of a broad mandate while declining to disclose even the basic measurement and disposition context. The issue would still be interpretive, not automatic evidence of wrongdoing. But the distance between a useful feedback channel and a representative claim would remain unmarked.

From encouragement to evidence

NANOG’s reviewed material already contains the ingredients of a better public account. It has daily survey practice. It has a post-event message that pairs a response count with a larger attendance count. It has a Program Committee call that lets recent requests be seen by potential speakers. It has a forward-looking statement that feedback contributes to later improvements. It has an open Community Meeting invitation. It has an earlier report that publicly recorded a respondent count while describing a community- and member-wide survey effort.

Those are not negligible signals. They show an organisation making room for feedback in more than one setting. The mistake would be to dismiss them because they are incomplete. The equal and opposite mistake would be to treat them as proof of representation, consent, or a one-to-one programme trail.

The practical next step is not a grand governance redesign. It is a meeting-level ledger that lets readers move from invitation to measure, from measure to aggregate theme, and from theme to action, non-action, or deferral. It should be candid about unknowns, protective of people who answer in good faith, and explicit that professional programme judgement remains necessary.

That is the standard a survey can meet without becoming something it is not. It can be a signal: timely, useful, and sometimes corrective. It need not be a mandate to deserve a clear public record.