DonkBoy
Internet |
Chapter 2
THE EVOLUTION OF USENET: THE POOR MAN'S ARPANET
By Ronda Hauben
rh120@columbia.edu
Part I
--------
In Fall of 1992, an undergraduate college student had a term project
to do. The assignment required that the project be the result of using
resources beyond research from books. His professor proposed that
students consider interviewing people, sending letters, etc.
The student had done some reading and found a source that claimed
computer networks had become "the largest machine that man has ever
constructed -- the global telecommunications network"(1)
The student decided that he would do his research on this
international computer network that now spans the globe and that
many computer users have access to. He planned to use the network
as much as possible to conduct his research.
After reading some of the few books and articles that he
could find to describe the global computer network, he gathered a
few significant quotes and wrote a brief introduction stating
that he was trying to determine the subject for a term paper. He
asked if the quotes seemed accurate and if readers had any
advice. Some of the quotes were from a journal article
discussing how the disintegration of Eastern Europe was in part due
to the lack of free speech impeding computer development.(2)
Also, the student asked if there was any evidence that the Berlin Wall
had fallen because of new developments in computer communications.
He raised several other questions and included quotes from his
reading.
Then he took this research proposal and posted it on the computer
network news system called Usenet. Posting is the word used
to indicate that one has sent an article to be propagated around
the world.
Usenet is like an electronic news magazine or a world
town meeting. It has various newsgroups organized by different
topic areas, in a variety of languages and on many different subjects.
And the number of all is continually growing. Net users can post articles
to any of the newsgroups, can respond to someone elses article or can
send a message in response to the author of an article via e-mail.
The student posted his questions to a number of newsgroups. In
his post, he wrote:
"`The Largest Machine': Where it came from and its importance
to Society"
He explained:
"I propose to write a paper concerning the development
of `The Net.' I am interested in exploring the forces behind
its development and the fundamental change it represents over
previous communications media. I will consult with people who
have been involved with Usenet from its beginnings, and
the various networks that comprise the Computer Network
around the world. I wish to come to some understanding of
where the Net has come from, so as to be helpful in figuring
out where it is going to."(3)
Within a few hours, responses began to arrive via electronic
mail to him. Among the responses he received was one from a
teacher in Russia who explained how pressure for the free flow of
information was a force for change in Russia.
"Hello," the teacher wrote, "I would also consider another
side of the coin: the world is divided on people who use the
possibility of computer-mediated communications and the ones who
do not. But I am not a specialist in your field."
"And as one from the East," he continued, "I know well that the
Internet is the first and only connection to the rest of the world
for us in Russia. But unfortunately," he reported "to get it
there is not too easy.... If you have some questions you think
I could answer - please send me email...."(4)
The student sent the Russian teacher some excerpts from an
article about the lack of free speech in Eastern Europe and its
effects on computer development. And he asked the teacher some
questions about the excerpts. In response to the questions, the
teacher from Russia answered:
"[The] first time I saw a computer was in 1985, when our
institute got one. It was [an] Apple II. At that time, I had no
idea about what a network is, and it was a time when PCs just
started to appear in the environment of [the] `normal Russian'...
As you can see, it was already Gorbachev's time, and communists
were stopping (or already were unable) to keep a very strong
control on information flow in the society. So it was easier to
access the PC in our institute than to get permission to use
photocopy devices. Then the number of PCs was fast growing, and
now we have more then 15 PCs, but still no Internet
connection....Most scientific institutes now have access to
the net. But usually it is restricted to the possibility of using
the electronic mail system."
"I would say," observed the teacher, "that in the past
networks had no direct effect on the life of people there, and
now they become more and more important. One of the points is
that it is practically the only way to communicate with the
West. Telephone lines are so bad that to send a FAX message is
almost impossible, conventional mail will reach the address with
[a] probability of 50% and it will take at least one month," but the
teacher wrote, "e-mail...will be received in 12-24 (!) hours. I
have used it for the last year and never had any problems. I am
lucky that one of my relatives has e-mail! I guess, you
understand how the possibility to communicate is important for
[the] scientific community...," the Russian teacher ended.
The student received many other responses. One from a German
student who described how the Berlin Wall had fallen because of the
increased communication made possible by computer networks. The
German student pointed out that accurate information about events like
the Chernobyl nuclear power plant explosion had become available to
people who were no longer dependent upon government channels
as their only source of information. Also, there were responses
from a teacher in Australia, a businessman in California, a net
pioneer, and many others.
The student decided to prepare another post. He had become
interested in how far and wide the network reached and who would
have access to the post he was making. He posted the following
message:
Subject: I want to hear from the four corners of the Net -
that means YOU!
I would like to hear from EVERYONE on the Net-Frontier.
If you think you are weird or abnormal (or special) in terms
of net-connections or usenet connections, please tell me
about it....
To the further expansion of the Net! :)
He received answers from over 50 people around the
world from France to India and Africa. A response from Japan
explained: "Yes, I believe I'm connected through some sort of
hokey mechanism, but that's just because I'm in Japan.
Connectivity doesn't register highly on the importance scale
here. Takes a few hours for mail to get from one side of Tokyo to
the other.
"So what makes me so `special'," wrote the correspondent
from Japan, "as far as net connections go? A few things. I can
not receive most newsgroups and can not post to any. Yet a friend
of mine in the same building as me (on another floor) receives a
mostly different set of newsgroups and can post to a few. The
interesting bit about any group we both get is that we don't
always get the same articles. Japan," he explained, "the
`leader' of technology, doesn't know a thing about actually using
computers. Just my opinion, of course -- my company won't listen
to me anyway! Hope this adds to your research ..."
The student received a response from an employee in a
large American company. The writer explained: "Not too strange,
but I work for a big company that leeches off two small `service
providers' for free mail and news feeds. Kind of funny,
really....Hey, Usenet broke... and I can't receive mail from the
Internet anymore, although I can send it." He described how the
company told him `Sorry... the problem is with our feeds. We'll
try to get them to fix it.' "Strange enough," the writer
sarcastically reported "these small services...[a medical school
and a public access usenet site] wouldn't drop everything to
fix our problem. How dare they! Of course MY suggestion," he
explained, "PAY THEM SOME MONEY," was completely ignored." He went
on to explain that he had been told that his company "won't let
us have a direct connection to the Internet for security
concerns. I understand, but it doesn't make me happy," he ended.
A response from Krakow, Poland explained that their site in
the Department of Physics at Warsaw University was one of the
first four sites in Poland to have access to Usenet.
A response from a French user explained how the government
charged a lot of money for an Internet connection in France and
thus discouraged usage: "It's cheaper to send a `hello' to
someone in the US than to someone 5 kilometers from my desk!,"
the French user wrote, "If you have a `stupidity chapter' in your
paper, this could fill a few lines."
From Wellington, New Zealand, the student learned that there
was a "burgeoning Net Community in Wellington, as there were
two Internet connections, one by a private net.enthusiast, and
another run by the Wellington City Council on an old PDP-11
computer." They offered the citizens of Wellington "free ftp,
telnet, IRC, archie, gopher, E-mail, and Usenet - and all the
1935 locally carried newsgroups."
A response from someone who works for Bell Telephone Laboratories
wrote: "Some people say that many of us at Bell are on the fringe, but
we're probably in the core of things in the Internet. :-)"
Other responses came from university students and hobbyists
in the U.S., from Net users in Germany, Italy, India, etc.
The student also got offers of help finding information,
suggestions of books to look at, and offers to send him articles
or reports that would be helpful with his research.
Also many people wrote describing the unusual or
interesting net connections they used to connect to Usenet,
A user in South Africa told how he distributed news and
e-mail and was trying to gain access to a satellite in order to
set connections up with the interior of Africa that lacked the
otherwise needed infrastructure. There were many other stories
of unusual or pioneering efforts to make connections possible for
people to Usenet.
Many people wrote asking for a copy of the paper when
it was written. In response to requests that he post the draft of
the paper before it was completed, the student wrote a draft and posted
it on Usenet. He received several helpful comments. He wrote
the final draft and handed it in to his teacher and posted it on
Usenet. A lively discussion ensued because the student's paper had
maintained that the ability of users to post on Usenet was a
sign that Usenet was democratic, whereas a document describing
Usenet to new users which was posted in the new users newsgroup,
maintained that Usenet is an anarchy. The discussion raised the question
of whether there can be an official statement maintaining that something
is an anarchy.
Also, a number of people wrote the student asking him if
they could distribute the paper more broadly, quote the paper in
their upcoming book, etc. Another student who was writing a
proposal for his Master's thesis cited the paper as an important
source in his proposal. This all occurred within 2 weeks of the
paper being posted.
This experience is but one example of the important
educational possibilities represented by Usenet and the
worldwide communications network it is part of. Yet there are
many people who still know nothing of Usenet, and many who
are on Usenet do not realize the important potential it makes
possible. In a similar way, many people do not know how Usenet
developed or the obstacles the pioneers were continually faced
with in their efforts to create and nourish Usenet. Since the
details of how Usenet was created can provide helpful insight
into how to deal with the problems one encounters today, following
is an account of how Usenet was created.
Part II
---------
Usenet was born in 1979 when Tom Truscott and Jim Ellis, graduate
students at Duke University, conceived of creating a computer network to
link together those in the Unix community. They met and discussed their
idea with other interested students, including Steve Bellovin, who was a
graduate student at the neighboring University of North Carolina at
Chapel Hill. Using homemade auto dial modems and the Unix to Unix copy
program (called UUCP), the Unix shell and the find command that were
being distributed with the Unix operating system, Version 7, Bellovin,
wrote some simple shell scripts to have the computers automatically call
each other up and search for changes in the date stamps of the files.
If there were such changes, the changed files were copied from one
computer to the other.
Soon three computer sites, "duke" at Duke University, "unc" at
the University of North Carolina at Chapel Hill and "phs" at the
Physiology Department of the Duke Medical School, were hooked together
and a simple program was running connecting the three sites.
Gregory G. Woodbury, a Usenet pioneer from Duke University,
describes how "News allowed all interested persons to read the
discussion, and to (relatively) easily inject a comment and to make
sure that all participants saw it."(5)
The program was slow so the students enlisted Stephen Daniel,
also a graduate student at Duke, to rewrite the program code in
the C programming language. Stephen Daniel writes that "a news program
written I believe by Steve Bellovin as a collection of shell scripts
was already working, but it was slow, taking upwards of a minute of
time on an unloaded PDP 11/70 to receive an article. I got involved
when I happened to drop in on a conversation between
Tom Truscott and Jim Ellis who were complaining about how slow this
news program was. I suggested that if it was so slow it could easily
be rewritten in C to run faster. I soon found myself volunteering to
do just that."(6) Daniel agreed to write the program in C with
help from Tom Truscott. This became the first released version of
Usenet in the C programming language, which came to be known as A News.
Other people at Duke and the University of North Carolina took
part in getting the network debugged. Once the program was functioning
on their respective machines, Jim Ellis went to a meeting of what
was then the academic Unix users group known as Usenix. In the
following account, Tom Truscott describes what happened:
"James Ellis (jte) gave a short talk and handed out a 5 page
`Invitation to a General Access UNIX Network' at the January
1980 Usenix Conference in Boulder Colorado. We made up 80
copies and they were gobbled up (not surprising, there were a
record-smashing 400 attendees)....Afterwards, jte mentioned
that the audience particularly enjoyed his description of
Duke's two home-built 300 baud autodialers."(7)
The invitation they distributed explains:
"The initially most significant service will be to provide a
rapid access newsletter. Any node can submit an article,
which will in due course propagate to all nodes. A `news'
program has been designed which can perform this service.
The first articles will probably concern bug fixes, trouble
reports, and general cries for help. Certain categories of
news, such as `have/want' articles, may become sufficiently
popular as to warrant separate newsgroups. (The news program
mentioned above supports newsgroups.)"(8)
The Invitation urged:
"This is a sloppy proposal. Let's start a committee.
No thanks! Yes, there are problems. Several amateurs
collaborated on this plan. But let's get started now. Once
the net is in place, we can start a committee. And they will
actually use the net, so they will know what the real
problems are."
Several months later, the software for the A News program for Usenet
was put on the conference tape for general distribution at the Delaware
Summer 1980 Usenix meeting. The handout distributed at the conference
explained, "A goal of USENET has been to give every UNIX system the
opportunity to join and benefit from a computer network (a poor man's
ARPANET, if you will)...."(9)
Daniel explains why the term "poor man's ARPANET" was used.
He writes, "I don't remember when the phrase was coined, but
to me it expressed exactly what was going on. We (or at least I)
had little idea of what was really going on on the ARPANET, but
we knew we were excluded. Even if we had been allowed to join,
there was no way of coming up with the money. It was commonly
accepted at the time that to join the ARPANET took political
connections and $100,000. I don't know if that assumption was
true, but we were so far from having either connections or $$
that we didn't even try. The `Poor man's ARPANET' was our way of
joining the Computer Science community and we made a deliberate
attempt to extend it to other not-well-endowed members of the
community. It is hard to believe in retrospect," he concludes,
"but we were initially disappointed at how few people joined us.
We attributed this lack more to the cost of autodialers than
lack of desire."(10)
The ARPANET that Daniel is referring to pioneered the networking
technology that serves as the foundation of today's global
Internet. The first host connected to the ARPANET was the SDS
Sigma-7 on Sept. 2, 1969 at the UCLA (University of California in
Los Angeles) site. It began passing bits to other sites at SRI
(SDS-940 at Stanford Research Institute), UCSB (IBM 360/75 at
University of California Santa Barbara), and Utah (Dec PDP-10 at
the University of Utah). There were many unexpected problems and
obstacles, but through the collaborative work by the participants
using the network they were creating, the number of sites
steadily expanded and by 1977 the ARPANET extended to more than
50 sites from Hawaii to Norway. Originally funded under the US
Department of Defense's (DoD) program for Advanced Research Projects
Agency (ARPA), however, only those academic computer science departments
with DOD funding had the possibility of access to the ARPANET.
Usenet, however, was available to all who were interested as long as
they had access to the Unix operating system (which in those days was
available at a very low cost to the academic and research computer
community.) And posting and participating in the network was
possible at no cost to the individuals who participated, besides
the cost of their own equipment and the telephone calls to
receive or send Netnews (as Usenet was called). Therefore, the
joys and challenges of being a participant in the creation of an
ever expanding network, the experience available to an exclusive
few via the ARPANET, was available via Usenet to those
without political or financial connections -- to the commonfolk
of the computer science community.
As Daniel notes, Usenet pioneers report that they were
surprised at how slowly Usenet sites expanded at first. But when
the University of California at Berkeley (UCB) joined Usenet,
links began to be created between Usenet and the ARPANET. The
University of California at Berkeley was a site on the ARPANET.
At first, it is reported, mailing lists of discussions among
Arpanauts (as ARPANET users were called by those on Usenet) were
poured into Usenet.(10) This first connection between the ARPANET
and Usenet, Daniel reports, only contributed to "the sense
of being poor cousins." Daniel explains: "It was initially very hard
to contribute to those lists, and when you did you were more likely
to get a response to your return address than to the content of
your letter. It definitely felt second class to be in read-only
mode on human-nets and sf-lovers, which were two popular ARPANET
mailing lists."(12)
Daniel also clarifies the different philosophy guiding the
development of Usenet as opposed to that of the ARPANET. He explains that
"Usenet was organized around netnews, where the receiver controls what is
received. The ARPANET lists were organized around mailing lists, where
there is a central control for each list that potentially controls who
receives the material and what material can be transmitted. I still
strongly prefer the reader-centered view," he concludes.
With the increasing connections to the ARPANET from Usenet,
the number of sites on Usenet grew.
A map from June 1981 shows the number of different sites on
Usenet during this early period:
!- Uucp links
: Berknet links
@ Arpanet links
pdp
(Misc) ! (NC) (Misc)
decvax sii reed phs--unc--grumpy duke34 utzoo cincy teklabs
! ! ! ! ! ! ! ! ! !
! +--+----+-----+-+--+-------------+-------+------+ !
! ! !
! duke !
! ! !
! +------+---+-----------------------+--------+ !
! ! ! ! ! !
ucbopt ! hocsr--mhtsa----research mh135a harpo-----chico
: ! ! ! !
ucbcory ! ! eagle ihnss vax135 (Bell Labs)
(UCB) : ! ! ! ! !
ucbvax--++----------+--+--+-----+--+------+--------+
: @ ! ! ! (Silicon Valley)
ucbarpa @ (UCSD) sdcsvax ! menlo70--hao
: @ sdcattb-----+ ! ! !
ucbonyx @ +-----ucsfcgl sytek sri-unix
@ phonlab-----+
cca-unix sdcarl
There are many stories of frustrations as Usenet developed,(13)
Despite such frustrations, there were many who helped Usenet
to grow and develop. Unix enthusiasts and pioneers at some large
organizations like AT&T's Bell Labs did whatever they could to provide
support for Usenet. At one point, AT&T realized that it would save
millions of dollars if it worked out the bugs to have internal e-mail.
In the process it gave support to the pioneers of Usenet who were
trying to develop more efficient e-mail routing programs. Also, Digital
Equipment Corporation (DEC) supported Usenet in various ways and the
spread of Usenet and Unix encouraged the sale of the Unix based computers
from DEC. And, Usenet newsgroups provided much needed technical
help for the folks using Unix and Unix based computers.
By 1982, the continuing explosion of Usenet surprised
even its most dedicated fans. One of those active on Usenet from
its earliest days to the present, Gregory G. Woodbury, describes
the shock that was experienced when the pioneers realized how
Usenet was taking a totally unexpected course of development.
He writes: "I do not recall that anyone was quite expecting the explosion
that followed," recounts Woodbury, "What developed took everybody by
surprise. When the direction of evolution took an unexpected
turn, and a continental network emerged, spanning the continent
from California to North Carolina, and Toronto to San Diego, it
was sort of a shock to realize what had happened."(14)
Statistics another Usenet pioneer, Gene Spafford presented
at an IETF (Internet Engineering Task Force) meeting in 1988
showed the tremendous growth and development that Usenet experienced.
Usenet developed from 2 articles a day posted at three sites in
1979 to 1800 articles a day posted at 11,000 sites by 1988.(15)
Year Number of Sites Articles/Day Megabytes/Day
1979 3 ~2 -
1980 15 ~10 -
1981 150 ~20 -
1982 400 ~50 -
1983 600 ~120 -
1984 900 ~225 -
1985 1,300 ~375 1+
1986 2,500 ~500 2+
1987 5,000 ~1000 2.5+
1988 11,000 ~1800 4+
Today Usenet continues to grow in the number of sites participating
and in the number of posts it carries and in the number of newsgroups.
Usenet is transported by uucp connections and via nntp (Netnews
Transfer Protocol) along the Internet, which is the child of the
old ARPANET.
Many times, pioneers of Usenet have been convinced that the
load of posts or the number of sites was becoming too great and
that further growth couldn't be sustained. The fear is now
facetiously referred to as the "imminent death of the net is
predicted." Although each time the problems have seemed
insurmountable, they have been investigated and solutions found to
deal with them through the hard work of many net participants
(referred to on Usenet as "net.citizens" or netizens).
In the past few years a system of Free-Nets and community networks
has begun to develop, many utilizing the Netnews software to make Usenet
available to community people for free or at a very low cost. Cleveland
Free-Net, sponsored by Case Western Reserve University and other
community organizations in Cleveland, Ohio, was the first Free-Net. It
used the Netnews software to create a set of local newsgroups reflecting
the different community services in the Cleveland area like the
hospitals, public schools, public libraries, museums, etc. Cleveland
Free-Net users also have access to the worldwide newsgroups of Usenet.
The software used by such community networks makes it relatively easy to
read and post on Usenet and in a variety of local discussion groups. A
number of community networks have come online around the US and in
Canada. Many others are in the planning stages. Also there is a Free-Net
in Erlangen, Germany and in Finland. And people with telnet access can
join many of these free of charge and thus have e-mail and Usenet access
through these community networks.
The ARPANET pioneered important breakthroughs in computer
networking technology. It also pioneered the ability to collaborate
and to utilize dispersed resources -- both people and computers. Usenet
represents the continuation of this tradition by making access to these
collaborative research relationships available to the commonfolk. The
extension of Usenet has also required a great deal of pioneering effort
and technical development, but the folks participating in Usenet have
been there to solve the problems.
Writing in 1968 before the ARPANET network began, J.C.R
Licklider, who has been called the Father of the ARPANET, and
Robert W. Taylor predicted the challenge that would face society
with the development of computer networks.
"First, life will be happier for the on-line individual
because the people with whom one interacts most strongly will be
selected more by commonality of interests and goals than by
accidents of proximity. Second, communication will be more effective
and productive, and therefore more enjoyable. Third, much
communication and interaction will be with programs and programming
models, which will be...both challenging and rewarding. And, fourth,
there will be plenty of opportunity for everyone (who can afford a
console) to find his calling, for the whole world of information,
with all its fields and disciplines, will be open to him, with
programs ready to guide him or to help him explore."
"For the society, the impact will be good or bad depending
mainly on the question: Will `to be on line' be a privilege or a
right? If only a favored segment of the population gets a chance to
enjoy the advantage of `intelligence amplification,' the network may
exaggerate the discontinuity in the spectrum of intellectual
opportunity."
"On the other hand, if the network idea should prove to do for
education what a few have envisioned in hope, if not in concrete
detailed plan, and if all minds should prove to be responsive,
surely the boon to humankind would be beyond measure."
"Unemployment would disappear from the face of the earth
forever, for consider the magnitude of the task of adapting the
networks software to all the new generations of computers coming
closer and closer upon the heels of their predecessors until the
entire population of the world is caught up in an infinite crescendo
of on-line interactive
debugging."(16)
Their vision of an ever growing part of the population of the
world being needed to participate in the debugging and development of the
network that will make a new world possible is still a helpful vision.
Thus I want to invite you all to help to contribute to the spread of the
Wonderful World of Usenet news. It is a world that needs and will reward
your participation.
---------------------------------
Notes for CHAPTER 2
(1) From Ithiel de Sola Pool, Technology Without Boundaries,
edited by Eli Noam, Cambridge, Massachusetts, 1990, p. 56.
(2) See for example "The Information Technologies and East
European Society" by Gary L. Geipel, A. Tomatz Jarmoszko, and
Seymour Goodman, in East European Politics and Society, vol 5
no 3, pp. 394-438.
(3) Michael Hauben, post November, 1992.
(4) E-mail correspondence from Dima
(5) Gregory G. Woodbury, "Net Cultural Assumptions," reprinted in
Amateur Computerist, vol 6 no 2/3, Fall/Winter 1994-5.
(6) E-mail correspondence from Stephen Daniel.
(7) From Usenet History Archives, October 12, 1990,
ftp://weber.ucsd.edu/pub/usenet.hist/nethist.901012.Z
(8) From "Invitation to a General Access Unix Network" by Tom
Truscott, Duke University, Durham, North Carolina. (Copy made
available by B. Jones.)
(9) Copy from Usenet History Archives. ftp://weber.ucsd.edu/pub/usenet.hist/
(10) Usenet History Archives, January 26, 1993. See nethist.930126.Z.
Also by 1979-80, UCB was under contract to ARPA to provide a version
of Unix (Berkeley Systems Distribution) for the ARPA contractors that
were going to be upgraded to VAX computers.
(11) E-mail correspondence from Stephen Daniel.
(12) E-mail correspondence from Stephen Daniel.
(13) One of the most outstanding is told by Amanda Walker who
remembered how it was necessary to send an e-mail message across
the continent twice using three networks to get it from the
Computer Science Department to the Computer Center on the
Case Western Reserve University campus. Usenet History
Archives, October 16, 1990. See nethist.901016.Z.
(14) Gregory G. Woodbury in "Net Cultural Assumptions."
(15) From Gene Spafford, Usenet History Archives, nethist.901011.Z
from the Mailing List, Gene Spafford, October 11, 1990 based on
data from Adams, Spencer, Horton, Bellovin and Reid. Updating
these statistics in 1993, David Lawrence estimates that in a 2 week
period ending March 9, 1993, about 26,000 articles per day were
posted to 4902 groups for 65 total megabytes (52 without headers).
(16) In Memoriam: J.C.R. Licklider 1915-1990, Aug. 7, 1990, p. 40,
reprinted by Digital Research Center; originally published as
"The Computer as a Communication Device," in Science and
Technology, April, 1968.
--------------------
Special thanks to the Usenet pioneers and to Bruce Jones
for the history materials they have gathered and made available.
Last Updated: June 5, 1996
===============================================================
This article is a draft chapter from Michael Hauben's
<hauben@columbia.edu> and Ronda Hauben's <rh120@columbia.edu> netbook
titled "Netizens: On the History and Impact of Usenet and the Internet."
*Commercial use is prohibited*
Please send us any comments about this draft. Send comments to
both hauben@columbia.edu and rh120@columbia.edu.
Friday, January 12, 2007
All Rights Reserved