« PreviousContinue »
gence techniques may be brought to bear. We propose a greater emphasis on intelligent sensors to perform data reduction and selective data transmission, and the development of knowledge data bases to aid in experimentation and planning.
portion of them have been automated. Over time, more and more of these steps will be automated and the programmer's role will become more supervisory. For the first time, the programming process will have been rationalized and recorded, open for examination and analysis. This will enable programs to be produced which are guaranteed to be consistent with their specification. It will eliminate the need for program testing and the cost and unreliability associated with undiscovered bugs. In addition, as automation increases, costs and effort will plummet. Besides the obvious advantages these reductions offer, a very important side benefit will occur. We know from instrumentation studies that large systems are not efficient when first implemented. Unanticipated bottlenecks always occur. The drastically lower costs of implementation will afford the opportunity for people to experiment with alternative implementations. These experiments wil broaden their experience base and enable them to develop better intuitions about how such implementation should be constructed. Furthermore, once people have gained this knowledge, it can be incorporated as a further automation of the programming process.
Current and future planned missions within NASA are oriented heavily towards the acquisition, dissemination, and analysis of data transmitted from space. The amount of such data is currently voluminous and will become larger by an order of magnitude in the 1980s. An estimate of the problem in the 1980s indicates that some 1010 bits of data/day will be generated for non-imaging data, while some 1012 bits/day will be generated for imaging data. The magnitude of the data acquisition and dissemination problem is staggering. When one adds the increased sophistication in data processing needed to convert raw data to information and to make it accessible to the users one has a major problem in managing such data.
All of this paints a very rosy picture about automatic programming. The catch, of course, is that these capabilities don't yet exist. The field is in a quite formulative stage. Impressive work is being done in a number of research labs, but none of these systems is close to practical use by an external user community. A period of research support followed by specialization to particular applications is needed if NASA is to reap any of these potential benefits. Since each of NASA's missions require similar, but different, software, a number of such specialized automatic programming systems could be constructed to cover a large percentage of NASA's total software effort. Recommendations:
The present NASA data management system has evolved in an ad hoc manner. Continuation of an ad hoc approach will neither be resource-effective nor meet the needs of the scientific user community for the post-1980 time frame. Greater reliance must be placed upon computers playing a greater role in space. The heavy density of data, instrument sophistication, and miniaturized microprocessors in space mandate that resource effectiveness be achieved on and between missions by end-to-end management of data. This will involve policy, management, software, and hardware. It is extremely important to have careful planning or central management planning for data. To achieve resource-effectiveness, the management of data must become a controlling force in the development and plans for any mission. In the following sections, we shall briefly describe the flow of data as it exists now, and the end-to-end data management concept that will be necessary to meet the demands of the 1980 era and beyond. We shall also discuss the steps required by NASA to meet the major challenge of the data management problem.
1. That NASA develop a research and development plan, in
conjunction with experts in automatic programming, for the creation of automated tools for the design and implementation stages of the software development process.
2. That NASA identify its major areas of software concen
tration and that specialized AP systems be developed for these as the field matures.
7.2 State of the Art: Flow of Data Within
7. Data Management Systems
The flow of data from an instrument to a principal investigator in today's technology goes from the instrument onboard to data processing on the ground and then is transmitted to a principal investigator or to facility instrument team members.
This section briefly outlines a proposal for a coherent data management system which would control data acquisition, reduction, analysis, and dissemination. We discuss NASA's NEEDS effort highlighting those areas where machine intelli
Future missions will require that, instead of a oneinstrument to one or many-instrument users, it will be necessary to have the outputs from many instruments onboard is limited in that the images are not keyed to a specific map projection.
the spacecraft undergo data processing and provide outputs for many users. For example, weather and climate information, spacecraft thematic data, and hydrological data obtained from many instruments are combined with data obtained through non-space observations to prepare food and fiber production forecasts.
Current Data Control. The management of data as it is obtained from the spacecraft is currently provided by onboard control management. They specify the data to be sensed, conditioned, handled, and transmitted by the instruments on the spacecraft. They have available to them flight direction data, and can make adjustments during the flight. Investigators who desire changes, must negotiate with the management team. Data obtained from a mission must undergo processing, sorting, and distribution. Further reduction, extraction, and analysis of the data takes place to transform the data into useful information. The transformed data and the raw data are stored in central repositories and distributed to principal investigators for further processing, analysis, and use.
Future Control: NASA End-to-End Data System (NEEDS). Projected mission data requirements exceed the present system capabilities to handle them. The increase in data volume can only partially be met through engineering technology improvements, as there promises to be a concomitant increase both in the number of users and complexity of sensor-related tasks. New demands continually arise for more complex instruments, better interfaces between instruments, and more sophisticated data processing. Many experiments and applications tasks in the near future will require a direct user/sensor coupling on a non-interference basis. This should require the development of dedicated, distributed microprocessors on board the spacecraft. Other applications in space will require large centralized processing on the ground to effectively integrate information provided by several satellites. For both instances, data management adminstration prior to launch of each mission is needed to assure coordinated information acquisition and integration.
An end-to-end data system will consist of the following elements:
This flow of data is illustrated by the LANDSAT project. LANDSAT data is currently transmitted to line-of-sight ground stations located at Beltsville, Sioux Falls, and Goldstone in the United States and in three foreign countries. The data is now in the planning stages to be transmitted to several other foreign ground stations. It is then retransmitted over the Space Tracking Data Network (STDN) or mailed to the Goddard Space Flight Center. In either case a three or four day delay results in the transmission receipt at Goddard.
1. Instruments Aboard Spacecraft which sense data,
provide attitude and position information, Greenwich mean time, and provide control information on the downlink to ground. They are provided uplink or control feedback to specify information to the sensors as to where to look, when to look, and how to look. Such control may emanate from mission operations or directly from users who can access the sensor remotely from terminals.
The raw data is assembled at Goddard where it must be spooled-up waiting for other data related to the flight, such as orbital information and fine attitude of the spacecraft. Some processing is performed on the data to account for such factors as the curvature of the Earth. Goddard then cuts a tape and transmits the processed data to the EROS Data Center run by the Department of the Interior in Sioux Falls. EROS catalogs the data, stores it in its data base and distributes data to users on a payment basis. Upon request, EROS produces high quality enhanced data. However, no LANDSAT data conforms to any particular map.
2. Operations – monitors system performance, develops
housekeeping information, coordinates activities, maintains a directory of the system, provides user assurance, and accounting information of the downlink. On the uplink to the spacecraft operations allocates resources, modifies the real time configuration, specifies flight maneuvers, coordinates user needs to the spacecraft, and provides housekeeping and repair for the spacecraft. A data base is maintained and updated on space flight information.
The LANDSAT data system for the U. S. should experience considerable improvement when a Master Data Processor (MDP) becomes available at Goddard. Such a MDP will provide space oblique mercator projections analogous to those obtained from ortho-photo aircraft images. Furthermore it is able to use selected ground control points for each frame to permit sequential frame overlays from several passes over a particular area. The master data processor can solve the problem of making the digital images look right, and can provide temporal registration. However, the MDP
3. Data Staging – receives data from operations on the
downlink through commercial networks and packages the data by operating on the output of many instruments required by a user. The output of instruments may require calibration, and the packaged data must be distributed to multiple users. Such information must be amassed in short periods of time (near real-time) to permit the user the ability to control and change instrument settings and programs. Data staging is a downlink operation. A data base is maintained at the data staging area.
4. Distributed Users – provides near real-time data and
investigates and screens output from instruments on the downlink. The data may be received directly from operations via commercial networks or be transmitted via commercial lines from the data staging area. On the uplink the users provide planning, scheduling, and control information directly to sensors that they control and which are independent of other instruments onboard the spacecraft. The user maintains a specialized data base.
images and by measurements taken by the robot itself. Item 2 can also be obtained partly on the ground and partly by the robot. General rules and axioms, on the other hand, needed to devise and effect plans, must be provided by the ground. Regardless of where the data and information arises, it is essential that capabilities exist within the robot to store, retrieve, and manipulate large amounts of data. Hence, a data management system will be necessary as a part of the robot itself. A non-conventional data management system will be required-one that can do deductive searching and plan formation. Hence, it will require an extensive knowledge base system, a semantic network, and an inference mechanism.
Even if the spacecraft is to be used to transmit data to Earth, where a data management system exists, there is considerable planning that can be accomplished so as to improve the efficiency of data acquisition. For example, the following topics should be addressed on every mission:
Need for data compression to minimize the number of bits transferred and to conserve communication channels.
5. Knowledge Centers - maintains data and knowledge on
specialized topics. The data from a mission is transmitted via commercial networks to one or more knowledge centers. The knowledge centers provide services to the distributed user community. They maintain not only mission supplied data, but data from other sources. knowledge center concerning weather data would maintain temperature, barometric pressure, and other weather data obtained by ground observation and measurement. The knowledge centers maintain archival records as well as data records. Knowledge centers will be linked together through commercial networks so that they may access one another. Users may access data in the knowledge centers through their remote terminals, and may thus perform operations on the data either at their own facility, or through the knowledge center facilities.
Data needed by the investigator as obtained by his instrument and other instruments. For example, if an image is to be transmitted to Earth, related data needed by the user should be transmitted at the same time. If, for example, the spacecraft is orbiting Earth, the precise location of the space vehicle, the angle of the Sun, and the angle of the camera tilt would be required as a minimum. Incorporating such data will save considerable effort on the ground by a small amount of processing in space.
Data Onboard the Spacecraft. Decisions must be made concerning the management of data within the spacecraft itself. These decisions will be a function of the particular mission, and whether or not there is ready access or interaction required with the user. For example, on a robotics application on Mars, because of the distance involved and the attendant time lag, it will not be possible to direct the robot from the ground, except to provide it with general goals to be achieved. This will require that the robot contain a large amount of data and information to permit it to maneuver on Mars. It will have to have the following, as a minimum:
Use of data by the scientist. Knowing the use to which the scientist will make of the data can determine the needs for archival data, and whether or not processing of the data either on the spacecraft, at mission control, or at a central repository would be of help to the scientist.
Data management planning for the spacecraft is, therefore, one important element of a data management plan for a mission.
1. Information as to the location, size, and composition of
Operations. Operations plays a central role on each mission. Hence, the data management system requires careful attention here. Design of the command system to the spacecraft must include consideration of integrity constraints. Such constraints assure that commands given to the spacecraft are legal ones, and that inevitable ground errors are minimized. Users who have sensor control on the spacecraft should have their commands passed via commercial digital networks to the operations group where the user command may be overridden if deemed necessary by operations, and if not overridden, the
2. A model of the terrain.
3. General rules about the relationships between objects.
Item 1 can be supplied partially from ground analysis of
command is scheduled for transmittal to the spacecraft, logged, and the user notified automatically as to when the command is to be transmitted. The delay between user transmittal and override should be in the order of a few minutes at most as user/sensor control should take place automatically only when the sensor is independent of other sensors onboard the spacecraft.
Operations will require a sophisticated message switching system to determine where space data is to be transmitted. It must also have a data base management system to be able to store and retrieve data retrieved from a mission. The amount of data for storage and retrieval at the operations facility will depend on the mission. Data for a few days receipt can be maintained while data of more than a few days can be retained at knowledge centers and retrieved through the network as needed.
The range of needs for the user cannot be anticipated in advance with respect to data processing functions. However, some users will require conventional data base management systems. In the former, there should be a major effort to standardize the data base management systems so that each user does not build or buy his own system. User proposals should be scrutinized carefully by a data base management system organization. Knowledge base systems will become prevalent in the 1980s. One can incorporate a knowledge base capability with a conventional data base management system (primarily relational data base systems), or build a special purpose system using one of the artificial intelligence languages. Such systems will be needed to do image analysis and picture processing, and to extract new data relations from given relations. Knowledge base systems can be general, but will require specific details based on the particular application. For instance, knowledge base systems could contain many of the features required for robots to accomplish their jobs on remote planets.
Knowledge Base Centers. Knowledge base centers will require three different types of data base management systems:
Data Staging. Data staging provides many-instruments to many-user capabilities. Data from many instruments are transmitted via commercial lines to the data staging area. The data undergoes data processing to transform it into usable modules for many users remotely connected to the data staging area. Capabilities should be provided to allow user access to raw and processed data. The users should be able to specify to the data staging area the operations to be performed on the data. The results can be placed into operational data sets consisting of processed data. All users should have access to all operational data sets. It will not be unusual for the many users to want common data. Making available the operational data sets to all users could save duplication of processing.
2. Generalized Data Base Management System.
3. Knowledge Base System.
The management of the data staging facility should assure that the same function is not applied many times and should recognize common requests by multi-users for the same processing functions. The data staging area should transmit processed data not only to users on an automatic basis, but to the knowledge centers for archival purposes. Data staging may be viewed as a switching and processing center. Its primary function is to operate upon spacecraft data and transmit the results to the user. It will have to maintain data base directory services, as required by the user.
Knowledge base centers should contain archived data relating to missions and related data. They must be able to retrieve requests and transmit responses to users who can access the knowledge centers through remote terminals. Requests can be specific – such as to retrieve the tape for the fifth orbit of data transmitted on a certain mission. They can be general, such as to send all tapes where the infrared reading was between certain limits on a mission. Thus, knowledge centers will have to maintain indexes of the data, and must perform some classification of data as it is stored in the system.
Users. The purpose of a mission will be to supply instrument information to the user population. The data staging area and operations can supply processed data and raw data to the user. Neither operations nor data staging can be expected to perform all the processing required by the user. Users will require their own processors and a means for storing and retrieving large quantities of data. One would not anticipate that users would require their own archival system as such a function can be provided by the knowledge centers.
Knowledge centers should also contain conventional data base management systems used to store and retrieve data conveniently contained in records. Whereas user data base management systems should fit on minicomputers, large scale computers and sophisticated data base management systems will be required. A distributed network of data base management systems should be investigated to iterate the various knowledge centers.
Sophisticated knowledge base systems which contain specific facts, general rules, and world models (e.g., a map containing roads that will be used as a template to match against images and be used to detect roads in images) will be required. By a general rule is meant a statement of the type, “If object 1 is to the left of object 2 and object 2 is to the left of object 3, then object 1 is to the left of object 3.” Complex knowledge base systems may also have to interface with sophisticated mathematical models. The area of knowledge base data systems is important and will require additional research support.
It is estimated that substantially less than 10% of all data received from space is ever used. By decreasing the amount of useless data by introducing intelligent sensors, and by providing better data management facilities to store, retrieve, and manage real-time and archival data, substantially greater use of data may be anticipated.
7.3 Opportunities and Recommendations
The NEEDS effort provides the potential for improving NASA use of data. Part of the improvement can come about by developing intelligent sensors and digital computer systems onboard spacecraft. Another part of the improvement can come about by developing an efficient ground communication/data processing system.
Although the NEEDS effort could yield considerable benefits for NASA, the efforts being conducted do not appear to be promising. NASA is taking a bottom-up approach to NEEDS. That is, rather than developing a comprehensive systems engineering approach to achieving such a system, a piecemeal approach is being taken. Various technologies are being investigated in an attempt to develop NEEDS. Although new technologies are clearly necessary, there is scant investigation into how they are to be brought to bear in a final system. To achieve an end-to-end data system that will provide users greater control over sensors, and will enhance the acquisition and dissemination of information from space, requires a systems approach in addition to a technology approach. The work will require significant planning at the management level, sophisticated software developments, and matching hardware capabilities. At the present time there appears to be no appreciation of the complexity of the NEEDS effort and the importance of engineering an entire system. Not only are intelligent sensors and reliable microprocessors needed in space but the management and flow of data from the spacecraft to the users and archival stores is essential. The following are some specific recommendations.
Intelligent sensors and digital computer systems onboard spacecraft can:
Send back processed, rather than raw data.
Obviate the need for documenting data on the ground as attitude, Greenwich mean time, and other information can be sent back to Earth with the sensor data as it is collected.
- Decrease the data flow as only relevant data need be
returned to Earth (for example, if image data of Earth is to be sent and the scene is obstructed by cloud cover, the space computer should detect this occurrence, and not send the useless cloudy image).
Respond to changes as to what should be collected as received in commands from the users.
Compress data so that needless or redundant information does not overload the communications channel.
Data Management Plan Coordination Group. A centralized group within NASA consisting of computer scientists will be necessary to provide overall plans for managing mission data. The group is needed to assure coordination between missions, to minimize duplication, and to determine the general tools that should be provided to user scientists by NASA. They should be concerned with assuring that there is a costeffective, appropriate data management plan on all missions. They should further be concerned with the acquisition and development of equipment and software.
Allow direct user-remote control.
An efficient ground communication/data processing system can:
Permit near real time processing of data.
Provide enhanced user services.
Mission Data Management Plan and Coordination Group. The mission-oriented group should provide plans as to how end-to-end data management will be achieved on a mission. They should be concerned with how to integrate the mission objectives with current and planned data management systems within NASA. The group should also consist of computer scientists and should review all data management plans with the centralized group.
– Transmit processed multisensor data to users.
Retrieve archival data more readily.