NSSDC Uses World Wide Web To Deliver Shoemaker-Levy 9 Data In Near Real
Time
Volume 10, Number 2, September 1994
by Syed S. Towheed with Dave Williams
Table of Contents
Thousands of scientists and non-scientists alike were fascinated
by the recent spectacle of Jupiter being bombarded by the Shoemaker-Levy 9
comets. The National Space Science Data Center, located at Goddard
Space Flight Center, made Shoemaker-Levy 9 images from a variety of sources
available via World Wide Web in near real time. The result was record-
breaking usage of the National Space Science Data Center's World Wide Web
service, as nearly 400,000 accesses were logged from July 18-29, 1994. The
author, systems programmer and World Wide Web coordinator, along with Dave
Williams, planetary scientist, was responsible for creating the Shoemaker-Levy
9 system and tells their story here in detail.
Return to Table of Contents
On Saturday, July 16, 1994 at about 4:00 PM EDT, the first fragment of
comet Shoemaker-Levy 9 (SL-9) struck Jupiter. The first image, from
the Calar Alto Observatory in Spain, was made available soon afterwards,
and at 10:00 PM a press conference was held and the first Hubble image was
released. By the morning of July 18, 1994, both the Jet Propulsion Laboratory
(JPL) and Space Telescope Science Institute (STScI) World Wide Web (WWW)
servers offering SL-9 images were laboring under the enormous loads and
turning away many users. Out of frustration and excitement, users turned to
the NSSDC hoping to find SL-9 observations. We already had a well used WWW
service with information about the comet, but our ambitions about providing
data were rather modest - we would put up a few of the better observations.
When I logged on at work on Monday July 18, 1994, I was greeted with a hail of
e-mail from around the world asking where "the" images were. Dave had also
reached the same conclusion that morning independently, so we decided that
a change of plans was in order.
By midday both myself and Dave had put together a dozen of so of the first
observations.We tested the system and made it available through our existing
WWW services on our primary server "Bolero". Much later that night I made an
announcement on several network news groups that the National Space Science
Data Center at NASA Goddard Space Flight Center was now providing SL-9
observation through it's WWW service.
Return to Table of Contents
Next morning news had traveled that a third major source was now providing SL-
9 data. By 9:00am the load on the server "Bolero" was sufficiently high for
us to consider closing down the service. Bolero was spawning up to 30 httpd
daemons (server software that handles requests from client software)
simultaneously and not able to service any one of them. It went into a kind of
"palpitation" spawning and closing down daemons. Some of our key resources
were on this machine and we couldn't afford all of them
to go down simultaneously.
The decision was made to bring up a secondary and more capable server and
migrate the SL-9 system over to the new computer. We decided to share part of
the load with "Bolero" by leaving all the top level menus there and move off
all the data components to the new server "NSSDC." For the next two hours we
worked feverishly with Jim Williams, our resident UNIX guru, to migrate the
SL-9 system and test it in the new environment.
Return to Table of Contents
The initial shock taught us a few lessons right away. The SL-9 system was
going to be very heavily used and we needed to optimize the pages for rapid
transmission. Online images would have to be shrunk to a 100x100 pixel
"browse" version, textual information should be on request only, i.e. no text
on the page but a link to a caption, and the whole system would have to be
only two layers deep so as to minimize browse activity. We were interested in
wanting the user to come in, go to the observatory or fragment they are
interested in, view the browse image, click on the image and caption to get the
originals, and leave. We consciously avoided including animated data and
motion pictures realizing that their immense size would create a massive bottle
neck for users who just wanted the latest observations.
The initial server usage statistics confirmed that our design criteria were
paying off handsomely. The server "NSSDC" was transferring data at 4.6 times
faster than Bolero had done at it's peak before the crash! We reached peak
activity on "NSSDC" on Thursday July 21, 1994 when we logged close to 6,000
accesses between 1:00 - 2:00pm. By the end of that day we had logged 73,557
requests, and transferred a little over 1,200 megabytes!
One of the interesting aspects of this experience was how communicative our
users were. We had "regulars" from around the world who would be constantly
reviewing the system to find bugs and errors. Dave Williams was equally
active in scouring sources around the world very early each morning (when
network traffic was the lowest) to ensure that we had the most recent
observations up before the morning rush of activity began. We made at least
three updates each day to ensure that we were as close to a real-time system
as possible.
Return to Table of Contents
Between Monday July 18, 12:00am and Tuesday, Jul 27 1:00pm, 1994 we had a
total of 361,132 accesses for just the SL-9 system. (An access is defined as
a single file transfer to a client. The file may be a HTML page, some image
element on the page, or a data file. The access ratio between image file and
non-image files is 6:1 for the SL-9 system.) NSSDC WWW services in general
experienced an all time high of 441,091 access for a similar period.
The total data transfer for the SL-9 system was 5,274 megabytes while the
total transfer for all NSSDC systems was only 787 megabytes. This is partly
due to the fact the nascent NSSDC system currently provides information only
with the exception of a CD-ROM based data system. However, this transfer rate
proves the applicability WWW as a data system.
The number of unique hosts which accessed the SL-9 system was
approximately 7,000. (Unique hosts is the number of actual TPC/IP addresses
(i.e. the number of different users) that accessed the server.) This
approximates to 51.6 accesses per user (they were browsing after all!) with
an average data transfer rate of 771.5 KB per user. The most requested image
was that of an early fragment A impact taken by Hubble Space Telescope. Over
2,200 requests were made for this image alone.
The most requested image from NSSDC's SL-9 WWW service shows Jupiter's cloud
tops after
the impact of the first fragment (A) of Comet Shoemaker-Levy 9 on 16 July. A
violet (410 nm)
filter of the Wide Field Camera 2 of the Hubble Space Telescope was used to
make the image,
which was taken at 5:32 EDT on 16 July 1994, 1.5 hours after the impact.
Image was provided courtesy of the HST Science Team.
The four pie charts below show the demographics spread of the users by
different categories.
Return to Table of Contents
Fortunately or not, the World Wide Web in particular, and the Internet in
general, has come of age. We must now learn to share our "space" with the
newcomers. This is probably the first time users outside our immediate
community have been so intimately involved with the excitement of discovery.
Since the SL-9 impacts began, we have been inundated with e-mail of sincere
appreciation and thanks. Our correspondents wrote how much they enjoyed being
part of the process and how much pride they felt for the scientists,
engineers, and NASA in bringing this news and data from the edge of the Solar
System to their computer within hours of the event. The SL-9 experience was
truly a fitting celebration of that other great event twenty-five years ago!
Return to Table of Contents
The actual images of the comet impacts hold great scientific value.
Infrared, ultraviolet, and visible images were made available by
various Earth- and space-based observatories. The visible images
showed plumes extending upward over 1000 km from some impacts, then
spreading out and falling back to the planet's surface. Visible
images also recorded the evolution of the cometary debris in the
atmosphere after the impacts, debris which left large dark "scars" at
the impact sites, and which remained visible for weeks. The
ultraviolet images showed the impact plumes and debris too, but also
allowed for the identification of acoustic waves in the atmosphere
which spread out from the impact sites. The infrared images were
particularly spectacular, showing explosions at the impacts and
intense heating of the atmosphere around the impact sites. The impact
sites remained hot and distinct in the infrared images for weeks,
causing the appearance of a bright ring of spots around the southern
hemisphere of Jupiter after all the impacts had completed. Scientists
will be using these images for years to come to study these impacts
and attempt to understand the composition, structure, and dynamics of
both the comet and Jupiter. In fact, at a seminar at Goddard soon
after the impacts, scientists working at observatories around the
world reported that the "real-time" images obtained over the World
Wide Web at their remote observing sites allowed them to see what
they could expect and prepare for their own observations accordingly.
Return to Table of Contents
Use your favorite WWW client software and open an URL to the address below
for SL-9
http://nssdc.gsfc.nasa.
gov/sl9/comet_images.html
for other NSSDC WWW services, open an URL to
http://nssdc.gsfc.nasa.gov/nssdc/nssdc_home.
html
NASA
home page
GSFC home
page
GSFC organizational page
