NSSDC Uses World Wide Web To Deliver Shoemaker-Levy 9 Data In Near Real Time

Volume 10, Number 2, September 1994
by Syed S. Towheed with Dave Williams
Table of Contents


Thousands of scientists and non-scientists alike were fascinated by the recent spectacle of Jupiter being bombarded by the Shoemaker-Levy 9 comets. The National Space Science Data Center, located at Goddard Space Flight Center, made Shoemaker-Levy 9 images from a variety of sources available via World Wide Web in near real time. The result was record- breaking usage of the National Space Science Data Center's World Wide Web service, as nearly 400,000 accesses were logged from July 18-29, 1994. The author, systems programmer and World Wide Web coordinator, along with Dave Williams, planetary scientist, was responsible for creating the Shoemaker-Levy 9 system and tells their story here in detail.

Return to Table of Contents

WWW: An Instant Data and Information System

On Saturday, July 16, 1994 at about 4:00 PM EDT, the first fragment of comet Shoemaker-Levy 9 (SL-9) struck Jupiter. The first image, from the Calar Alto Observatory in Spain, was made available soon afterwards, and at 10:00 PM a press conference was held and the first Hubble image was released. By the morning of July 18, 1994, both the Jet Propulsion Laboratory (JPL) and Space Telescope Science Institute (STScI) World Wide Web (WWW) servers offering SL-9 images were laboring under the enormous loads and turning away many users. Out of frustration and excitement, users turned to the NSSDC hoping to find SL-9 observations. We already had a well used WWW service with information about the comet, but our ambitions about providing data were rather modest - we would put up a few of the better observations. When I logged on at work on Monday July 18, 1994, I was greeted with a hail of e-mail from around the world asking where "the" images were. Dave had also reached the same conclusion that morning independently, so we decided that a change of plans was in order.

By midday both myself and Dave had put together a dozen of so of the first observations.We tested the system and made it available through our existing WWW services on our primary server "Bolero". Much later that night I made an announcement on several network news groups that the National Space Science Data Center at NASA Goddard Space Flight Center was now providing SL-9 observation through it's WWW service.

Return to Table of Contents

Bolero Crashes at 9:00am

Next morning news had traveled that a third major source was now providing SL- 9 data. By 9:00am the load on the server "Bolero" was sufficiently high for us to consider closing down the service. Bolero was spawning up to 30 httpd daemons (server software that handles requests from client software) simultaneously and not able to service any one of them. It went into a kind of "palpitation" spawning and closing down daemons. Some of our key resources were on this machine and we couldn't afford all of them to go down simultaneously.

Graph showing Bolero activity

The decision was made to bring up a secondary and more capable server and migrate the SL-9 system over to the new computer. We decided to share part of the load with "Bolero" by leaving all the top level menus there and move off all the data components to the new server "NSSDC." For the next two hours we worked feverishly with Jim Williams, our resident UNIX guru, to migrate the SL-9 system and test it in the new environment.

Return to Table of Contents

New Server "NSSDC" On-line by 12:00pm

The initial shock taught us a few lessons right away. The SL-9 system was going to be very heavily used and we needed to optimize the pages for rapid transmission. Online images would have to be shrunk to a 100x100 pixel "browse" version, textual information should be on request only, i.e. no text on the page but a link to a caption, and the whole system would have to be only two layers deep so as to minimize browse activity. We were interested in wanting the user to come in, go to the observatory or fragment they are interested in, view the browse image, click on the image and caption to get the originals, and leave. We consciously avoided including animated data and motion pictures realizing that their immense size would create a massive bottle neck for users who just wanted the latest observations.

The initial server usage statistics confirmed that our design criteria were paying off handsomely. The server "NSSDC" was transferring data at 4.6 times faster than Bolero had done at it's peak before the crash! We reached peak activity on "NSSDC" on Thursday July 21, 1994 when we logged close to 6,000 accesses between 1:00 - 2:00pm. By the end of that day we had logged 73,557 requests, and transferred a little over 1,200 megabytes!

Graph of Bolero and NSSDC activity

One of the interesting aspects of this experience was how communicative our users were. We had "regulars" from around the world who would be constantly reviewing the system to find bugs and errors. Dave Williams was equally active in scouring sources around the world very early each morning (when network traffic was the lowest) to ensure that we had the most recent observations up before the morning rush of activity began. We made at least three updates each day to ensure that we were as close to a real-time system as possible.

Return to Table of Contents

Server Logs Reveals Unprecedented Access

Between Monday July 18, 12:00am and Tuesday, Jul 27 1:00pm, 1994 we had a total of 361,132 accesses for just the SL-9 system. (An access is defined as a single file transfer to a client. The file may be a HTML page, some image element on the page, or a data file. The access ratio between image file and non-image files is 6:1 for the SL-9 system.) NSSDC WWW services in general experienced an all time high of 441,091 access for a similar period.

The total data transfer for the SL-9 system was 5,274 megabytes while the total transfer for all NSSDC systems was only 787 megabytes. This is partly due to the fact the nascent NSSDC system currently provides information only with the exception of a CD-ROM based data system. However, this transfer rate proves the applicability WWW as a data system.

The number of unique hosts which accessed the SL-9 system was approximately 7,000. (Unique hosts is the number of actual TPC/IP addresses (i.e. the number of different users) that accessed the server.) This approximates to 51.6 accesses per user (they were browsing after all!) with an average data transfer rate of 771.5 KB per user. The most requested image was that of an early fragment A impact taken by Hubble Space Telescope. Over 2,200 requests were made for this image alone.

Most popular image

The most requested image from NSSDC's SL-9 WWW service shows Jupiter's cloud tops after the impact of the first fragment (A) of Comet Shoemaker-Levy 9 on 16 July. A violet (410 nm) filter of the Wide Field Camera 2 of the Hubble Space Telescope was used to make the image, which was taken at 5:32 EDT on 16 July 1994, 1.5 hours after the impact.

Image was provided courtesy of the HST Science Team.

The four pie charts below show the demographics spread of the users by different categories.

Pie chart of top 10 domains
Pie chart of top 10 commercial sites Pie chart of top 10 educational sites
Pie chart of top 10 government sites

Return to Table of Contents

Public Shares the Process of Scientific Discovery

Fortunately or not, the World Wide Web in particular, and the Internet in general, has come of age. We must now learn to share our "space" with the newcomers. This is probably the first time users outside our immediate community have been so intimately involved with the excitement of discovery. Since the SL-9 impacts began, we have been inundated with e-mail of sincere appreciation and thanks. Our correspondents wrote how much they enjoyed being part of the process and how much pride they felt for the scientists, engineers, and NASA in bringing this news and data from the edge of the Solar System to their computer within hours of the event. The SL-9 experience was truly a fitting celebration of that other great event twenty-five years ago!

Return to Table of Contents

Scientific Interest in Comet Shoemaker-Levy 9 Collision with Jupiter

The actual images of the comet impacts hold great scientific value. Infrared, ultraviolet, and visible images were made available by various Earth- and space-based observatories. The visible images showed plumes extending upward over 1000 km from some impacts, then spreading out and falling back to the planet's surface. Visible images also recorded the evolution of the cometary debris in the atmosphere after the impacts, debris which left large dark "scars" at the impact sites, and which remained visible for weeks. The ultraviolet images showed the impact plumes and debris too, but also allowed for the identification of acoustic waves in the atmosphere which spread out from the impact sites. The infrared images were particularly spectacular, showing explosions at the impacts and intense heating of the atmosphere around the impact sites. The impact sites remained hot and distinct in the infrared images for weeks, causing the appearance of a bright ring of spots around the southern hemisphere of Jupiter after all the impacts had completed. Scientists will be using these images for years to come to study these impacts and attempt to understand the composition, structure, and dynamics of both the comet and Jupiter. In fact, at a seminar at Goddard soon after the impacts, scientists working at observatories around the world reported that the "real-time" images obtained over the World Wide Web at their remote observing sites allowed them to see what they could expect and prepare for their own observations accordingly.

Return to Table of Contents

How to Access our WWW Service

For NSSDC WWW services, open an URL to

http://nssdc.gsfc.nasa.gov/nssdc/nssdc_home. html

Return to NSSDC News Table of Contents

NASA home page GSFC home page GSFC organizational page

Author:Miranda Beall
Curators: Erin Gardner and Miranda Beall
Responsible Official: Dr. Joseph H. King, Code 633
Last Revised: 21 Nov 1996 [EDG]