Summary

  • The Library of Congress announced in April 2010 that it would acquire public tweets from Twitter's beginning and keep receiving them. Its FAQ two weeks later excluded private account information, deleted tweets, linked sites and pictures. The promise was broad; the material taken was defined.
  • By December 2017, the Library said the collection held only text while tweets had become more visual and longer. It would preserve the 2006–17 text archive and collect future posts selectively from January 2018. The public announcement kept the existing collection under embargo while access questions remained unresolved.

The 2017 switch was about what a post had become

In December 2017, the Library of Congress changed the rule for a collection it had once described as comprehensive. At the end of the year it would stop taking the entire stream of public tweet text. It would still take selected tweets about events and themes, including elections and public policy. The 2006–17 collection would stay in its care. A large cultural memory was not thrown out; the institution changed what it would add next.

The Library's decision paper gave several reasons. The number of tweets and related transactions had grown. Posts were becoming longer. The Library ordinarily did not collect every item in a field forever; this had been an exception made when the future of social media was unknown. One reason exposed a sharper problem than size: the Library received only text. It did not receive the image, video or linked page attached to a post. As posts became more visual, the archive's unit of acquisition captured less of the thing a reader had seen.

A sentence that points to a photograph is not the photograph. A short reaction to a video is not the scene that prompted it. The collection could store the words faithfully while leaving the object of those words beyond its boundary. This is why the 2017 decision was more than a storage calculation. It was a judgment about whether the incoming record still represented the communication the Library meant to preserve.

The boundary was written down at the beginning

The April 2010 announcement gave the Library's gift an irresistible headline: the entire public Twitter archive, starting with the first tweets in 2006, with new public tweets to follow. It offered historians a record of ordinary communication, news and social change. The optimism was sincere; so was the boundary that appeared in the Library's FAQ of 28 April.

Twitter, not the Library, would determine which tweets were public. Private account information and deleted tweets would stay out. Pictures and websites reached by links would stay out too. The Library said it would not reproduce the service's own functionality. It planned an archive for research, with tools and access rules still to be built. The six-month delay before any tweet could become available to researchers was a contractual floor, not a working search box on opening day.

Those limits matter when the collection is later described as “all tweets.” The donor controlled the public-data feed. The Library controlled preservation and, within the agreement, research access. Linked content lived elsewhere. None of these parties could make a text-only copy turn into a preserved picture by naming it a tweet archive. The original FAQ told readers exactly which seams to expect.

A machine for intake arrived before a door for readers

The Library's January 2013 technical report shows that the intake was real, not a press-release promise. The historic 2006–10 shipment contained about 21 billion tweets in three compressed files. By 1 December 2012, more than 150 billion further tweets had arrived: about 170 billion in all, occupying 133.2 terabytes for two compressed copies. Daily receipts had risen from 140 million tweets when ongoing transfer began in February 2011 to nearly half a billion in October 2012.

The transfer agent grouped the stream into hour-long files. The Library retrieved each file, checked it, counted its tweets, copied it to tape and cleared temporary space. Two tape archives in different locations held the preservation copies. This was an operating pipeline, with a custody trail and redundancy. It solved the job of receiving a growing feed of text.

It had not solved the different job of letting a scholar interrogate it. The same report said researchers had not yet been given access. On the Library's then-current systems, one search of the fixed 2006–10 archive could take 24 hours. The agreement allowed access to bona fide researchers after a six-month delay, with restrictions on commercial use, redistribution and a large downloadable web copy. Contractual permission, a physical copy on tape and a useful reading room were three different achievements. In 2013, the first two were more advanced than the third.

That access gap has its own history; it does not explain away the media gap. A faster search could find the archived text but could not retrieve an image the Library never received. As collection grows, the cost of preserving a unit and the value of that unit must be considered together. The 2017 paper did exactly that when it named the visual turn as one of several reasons to stop treating comprehensive text intake as the only route to a historical record.

The record remained; the rule changed

The 26 December 2017 announcement said the Library had secured tweet text from 2006 through 2017. From 1 January 2018, it would collect selectively. Its decision paper said the existing collection would remain embargoed while the Library worked through access questions at a sustainable cost; it gave no date for public access. It also identified deletion transactions and user-access issues to resolve with Twitter.

A selective archive makes the curator's choices more visible. Which event deserves capture? Which theme remains in the collection when the text stream no longer enters wholesale? The 2010 arrangement could hide those choices behind the phrase “every public tweet,” even though the photograph beside a sentence had always been outside the gate. By 2017 the Library put the selection decision in the open. Its twelve-year text record remained a substantial primary source. Its missing media remained missing. The new intake rule acknowledged that the boundary of a post had moved while the archive's feed had stayed put.