« PreviousContinue »
There could be, for example, several groups competing by applying different locomotion schemes to the same exploration job. Similarly, several groups could explore a variety of shuttle-based, large-structure assembly ideas.
NASA conducts large imaging programs which produce enormous volumes of images. NASA programs are studying means of making image data more available and more useful for users (NASA End-to-End Data Management System program). Those activities are largely for presentation of images to humans for human perception. Those NASA projects with large potential benefit to society which involve machine visual perception include:
1. Construction of large space structures, particularly
communications systems and antennas.
2. Remote sensing and agricultural resource evaluation.
We should begin thinking along these lines because spacecraft computer hardware is becoming more and more out-of-date and something simply must be done about it. Today it takes too long to “qualify” space computers. There seems to be no mission-independent way to do this. Individual missions have to use computers qualified by previous, almost accidental, qualification incidents. Memory sizes, in particular, are much too small. This leads to weak programs with minimal versatility and to doing things in hardware that might be lighter and more reliable in software. Therefore, at the very least, NASA should have a continuing program to spacequalify larger memories and more capable computers, as they evolve. We know enough about computation, today, to be able to assert that there is little reason to suppose that the computers will have to be adapted to particular missions much, except in regard to overall capacity parameters.
Advances in computer vision would enable increased effectiveness of the proposed Mars rover mission. These applications require a compact area of vision science and technology. NASA's vision applications are sophisticated. Suggestions are made to advance NASA objectives by:
Recommendations for Rover Research. Given the need to take advantage of imminent opportunities on Mars, we believe the design, construction, and systematic evaluation of a functional reconfigurable rover should be undertaken to:
1. Collaboration with other organizations which have an
investment in applications requiring similar technology and which support research in this area of image science.
1. Determine optimal configuration alternatives from the
standpoint of stability, maneuverability, and clearance with weight as a major, if not the major, consideration.
2. Involving the most advanced research groups in research
program formulation and implementation.
3. Evaluation of current NASA imaging programs.
2. Evaluate alternative wheel/tracking/leg concepts as a
function of terrain classes with respect to speed, steering, obstacle climbing ability, weight, and reliability both in the laboratory and out in the field.
3. Serve as a test bed for the development and evaluation
of alternative vision/sensor/calculational/guidance control systems applicable separately to long-range, mid-range, and short-range path-planning levels and to integrated systems ultimately. There is a corollary need to develop additional sensors to provide real-time sensory feedback with a much broader range of spatial and temporal resolution.
Automated imaging and mapping systems are planned to meet objectives of NASA application programs and missions. Earth resources surveys include crop production, water resources, land use, forest resources, ocean resources, and oil spill monitoring. Meteorological prediction, monitoring, and climatic studies already make an impact in daily life. Geological studies include crustal dynamics and a world geological atlas. Large space structure construction is likely to be important for communications. Automated sensing has a role in these applications.
2. Smart Sensor Technology
This section comments on NASA programs which use vision science to make an impact in its applications and mission programs. It summarizes the state of the art in the required technology areas, summarizes research recommendations, and suggests a structure which will encourage required research.
These activities overlap responsibilities of other organizations such as USGS, Forest Service, Defense Mapping Agency, etc. Capabilities necessary for NASA functions enable NASA to contribute significantly to development of automated imaging for civilian purposes. A major part of these NASA programs requires innovative and high level research to develop required technology. NASA can lead in development of this technology. Organizations with similar responsibilities have little resources to sponsor and direct research. A major emphasis of this section is that it is important for NASA to get “leverage” in research and applications, that is, to work with existing research programs and to work with potential users of the technology.
higher proportion of research and more innovative research should be supported, and that research results be incorporated into development systems continuously, with little lag. It appears that a production system was built with obsolete and inadequate technology.
NASA performs two functions in this area, data distribution and information extraction. Most effort has gone into data distribution. Much work remains. Current and planned imaging missions provide volumes of data beyond existing abilities to catalog, distribute, and assess the images. Smart sensors for data compression, automated image handling facilities, and high performance computers for imaging are needed. These needs are recognized by the NEEDS (NASA End-to-End Data Systems) program which addresses smart sensors, special purpose imaging computers, and image handling facilities.
A likely requirement for the application of structural pattern recognition and scene analysis techniques is imagery having much higher resolution than LANDSAT. Eighty meters resolution is probably too crude to use structural relations. High resolution imaging may make use of aerial photography, which is part of NASA's domain. A crop survey using structural analysis at high resolution is perhaps feasible now, and will be feasible in a few years. A scenario is outlined below which would require about 3.4 years to do a world-wide crop survey at 108 ops/second. The ultimate resolution is about 2 cm per pixel. Estimates are based on a two-stage analysis. For the first stage, a coarse sampling at 2 m/pixel is probably adequate. Alternatively, a coarse grid of linear scans would require about the same computation cost. The first stage is intended to separate major field boundaries. The second stage would use structural analysis at 2-сm resolution on limited parts of the fields. The use of smart sensors (for example, edge operators under development in the DARPA Image Understanding program) would be useful in this program. Smart sensors would cut computation cost significantly.
A greater need exists in information extraction. Here, NASA's objectives require sophisticated vision science which has not yet been achieved. That need is recognized within NASA. The Space and Terrestrial Applications program is soliciting proposals for new technology in remote sensing and terrain topography. The content of the recommendations of this report is that efforts to develop new technology should be intensified, and that they should be strengthened by strong participation of major research groups outside NASA and by cooperation with other organizations with similar needs. The balance between research and production systems should be evaluated; a heavy research component is essential. Careful examination should be made of current and proposed production systems to evaluate whether they are founded on an adequate technology base.
The Earth's area is 2 X 1019 cm2. About 1/4 is land and of that, half is arable. If we sample 10% at 2 m/pixel, there are about 5 X 1012 pixels. Assume about 1000 ops/pixel for reasonably sophisticated processing, and 108 ops/second. Then the required computation time is 5 X 107 seconds. There are 3 X 107 seconds per year. A single computer would require 1.7 years now. An equal amount of computation would be required for the second stage, for a total of 3.4 years. If we assume that semiconductors will increase density at the rate of a factor of 2 every two years (a factor of 2 per year is the current rate) and if we assume a factor of 4 speed increase in 5 years (the historical rate), then in about 10 years, a single computer will be able to make a world-wide sampling in four days. The vision science and software technology should be developed now to make use of that computing power.
2.2 The State of the Art
Remote Sensing and Crop Survey. An examination of crop census systems reveals that their performance is very limited. In summary, those systems do not seem to have met expectations of lowered cost and increased repeatability from automated classification. In these systems, humans make decisions, aided by computer clustering. The overall system accuracy is about 90%. Their computerized classification is not that good. What humans currently contribute to classification is use of spatial context. Both structural pattern recognition and scene analysis offer techniques to use spatial context in identification. Structural pattern recognition experiments indicate significant improvements in performance. Our evaluation of the mix between development and research indicates that a
Cartography. The production of maps by traditional means is labor intensive. Partial automation of elevation contour mapping has been in use for years by DMA, with analog stereo correlation systems. It is often thought that automated stereo mapping is a solved problem because there are production systems; however, these systems require a great deal of human intervention. Typically, they are interactive systems in which the operator redirects the system whenever it gets lost and patches up errors. There are problems when tracking over water and over uniform surfaces such as concrete. They do badly at surface discontinuities such as edges of buildings and cliffs. In trees, picking out the ground surface is beyond the capability of the system. The extent of human intervention required is enough to decrease mapping speed and limit mapping output.
shutdowns in nuclear electric power stations have highlighted the technical problems of servicing reactors. Work is currently done in a radioactive environment by humans. Advanced capabilities for remote operation with man in the loop offer opportunities to reduce hazards to workers, lower the cost, and increase the level of maintenance. On-line monitoring and maintenance are other possibilities. A high payoff is expected for a partially automated system. In this type of system, the teleoperator system takes over a set of limited operations, using sensing and knowledge of parts. Once the operator has positioned the manipulator to approximately the right orientation, the system completes the action itself. The payoff is in speed and ease for the operator.
The DMA has made a major study in automating cartography in a largely digital system. DMA studies revealed extensive requirements for advanced techniques in computer science with an emphasis on machine intelligence. There is a strong relationship of many DMA concerns with related issues in NASA particularly in the area of scene analysis and understanding, large database management, and information retrieval.
Research in stereo vision, some of it supported by NASA, has produced stereo systems which work in a research environment and has produced advances in our scientific understanding of stereo vision. A model is emerging of the stereo vision process from which newer high performance systems are being designed and developed. Preliminary research in linear feature tracing has been carried out and the results indicate that interactive systems using tracing aids are feasible for features such as roads and rivers. There is a growing body of research on edge finding systems which will support development of such aids to linear feature tracing. Building large data bases for cartographic applications requires the integration of research in vision, machine intelligence, and general systems issues in computer science.
Technical requirements for this application require the development of manipulator hardware, control systems, software, and sensor systems, in addition to a vision system. The vision system required for the simplest of teleoperator systems needs the ability to present multiple views, and could benefit from stereo if satisfactory stereo systems can be developed. For partially automated systems, stereo vision and the use of multiple views are highly important. Even when the views are separate (i.e., wide angle views which cannot be fused), the sort of modeling which is involved in stereo vision is important for autonomous vision in these contexts. Considerable use can be made of knowledge of the design of parts and joints, for model-based vision systems.
Teleoperators. This issue is shared between the Study Group's vision and robotics subcommittees. This section will address only the vision part of teleoperator work in space.
The building of large space structures for communications systems and possible experimental stations appears likely. The cost of maintaining a human worker in orbit, including life support systems and shielding from radiation, is estimated at $1.5M per year. It is hard to assess the difficulty of maintaining a crew of highly trained workers in this hazardous environment. Possible space power stations and space industrialization projects would involve large construction efforts. Development of teleoperator manipulation offers the possibility of increasing the productivity of human workers, while lowering their risk. Operation with large objects, such as the ShuttleAttached Manipulator, imposes another requirement for advanced teleoperator systems.
Mars Rover. A proposed Mars rover mission requires considerable onboard autonomy if one expects to achieve the objectives of a few hundred meters navigation per day, with communication for a short time once per day and round trip signal times of twenty-five minutes. The minimal navigation device is a laser ranging device. Its two limitations are limited range and limited number of samples. These limitations put restrictions on its reliability and utility since such a sensor can do little in looking for interesting samples. Navigation using only this device can be only local, with little look-ahead and low resolution. Under these conditions, the rover is likely at some time to reach a dead end that it can't back out of, or waste excessive time in getting out of, because of limited search strategy options.
NASA does sponsor some research in stereo vision. This is on a small scale and should be expanded. Functionally, stereo vision with motion parallax offers capabilities to maintain orientation by navigating with respect to landmarks, and to allow depth ranging and maping of distant objects by making use of large baselines accumulated in motion. It is thus possible for the rover to avoid problems and to return to base locations.
This technology would contribute to electric power generation, to undersea oil drilling and mineral exploitation, and to rehabilitation of disabled people. Recent incidents with power
We recommend evaluation of NASA participation in the development of advanced automation in cartography for civilian purposes. Cartography and land use studies appear to be important applications areas. Progress in computer stereo vision makes possible major advances in cartography. The civilian organization with responsibility in this area, USGS, has limited facilities and limited research. Because of the strong relationship of the Defense Mapping Agency Pilot Digital Operations Project with NASA interests, it is recommended that NASA maintain strong liaison with the DMA and investigate possible collaboration with their efforts. NASA should evaluate the DMA planning process to aid in costing the development of detailed plans for implementing some of the related suggestions of this Study Group. A collaborative research program with DMA and USGS would have high potential benefit, and would be strengthened by research underway in DOD, particularly for cruise missile guidance.
development investment. It is suggested that NASA support research at centers of excellence in computer vision. This approach is cost-effective since it is not necessary to support whole programs; these centers have broad support and well-established programs. This approach provides a means of collaboration with related research programs. It is suggested that the emphasis be on innovative focused research, not on applied research. It is recommended that a vigorous program of evaluation by members of the research community be used for program formulation and proposal review, and that they be involved in a strong periodic program monitoring effort. NASA is involved in the forefront of computer vision since its intended applications probably are not feasible by old technology. Yet, NASA does not have a broad enough base of imaging science within its organization. A significant part of NASA vision effort should be outside of NASA-related centers. It is recommended that hardware development work on smart sensors and image processing computers be carried out in collaboration with DOD and with broad contact with the research community. The NEEDS program represents a step toward a systems approach to providing data to users. There is a need for a program which integrates this data system with the information processing that users actually perform on the data.
We recommend the support of research in computer stereo vision for teleoperators intended for remote construction and maintenance of large space structures for communication facilities in space. Antennas and communication systems in space appear to have economic benefits in a reasonable time scale. We recommend that a small investment be made which would increase productivity of remote operations as the cost per man-hour in space will be high. Advanced teleoperator technology would lower exposure of human workers in the radiation belts and increase their effectiveness. The technology would be equally useful for large space structures for space power stations
stations or space industrialization should NASA undertake them.
3. Missions Operations Technology 3
This section discusses NASA's current mission operations and attempts to identify several areas in which machine intelligence can be brought to bear to increase the automation of control operations and to replace humans in time-critical, repetitive, and routine decision-making roles. A proposal to automate the mission-independent aspects of data collection and to provide a uniform facility for embedding missionspecific systems in the basic support system is reviewed.
We recommend that NASA increase support of computer stereo vision for a proposed Mars rover mission. Current progress in stereo vision promises improved capabilities and increased scientific payoff.
We recommend that agricultural remote survey applications be reevaluated. It is urged that performance limitations of the current technology be evaluated. NASA should study the feasibility of using more powerful structural pattern recognition and scene analysis approaches, and that systems be built which incorporate new technology. Crude estimates indicate that high resolution structural analyses may be feasible soon for crop census.
NASA currently builds and rebuilds mission-specific software for each mission's control. Although this state of affairs reflects the natural evolution of NASA as a large complex organization, there are indications that, without immediate and global reorganization of the mission control procedures, both NASA's science and economy will begin to suffer. Specifically, the Study Group sees a pervasive need to centralize and standardize mission operations procedures. In this regard, the Study Group sees a clear need for the development of a modular, “reusable” nucleus of mission operations software.
We recommend that NASA support basic research in structural pattern recognition and scene analysis approaches.
We recommend that NASA diversify its research base in imaging research, that it evaluate the proportion of research to
The scope of the standardization and centralization should include all aspects of mission control, from the lowest levels of 3 that might be made more reliable, rapid, or economical if partially or fully automated via existing AI techniques.
Current Missions Operations. Mission operations is the control executive for a mission. As such it comprises the following specific activities:
sequencing and monitoring to the highest levels of planning and problem solving. Current problems at the lower levels relate not so much to lack of mechanization as they do to lack of organization of the existing mechanization. Hence, cleaning up the lower levels calls for improved software development and integration techniques. On the other hand, establishing procedures and capabilities to organize and extend the effectiveness of the higher levels of mission control seems to call for the infusion of AI techniques; the goals at the higher levels would be to increase the automaticity of mission control, replacing humans in time-critical, repetitive, and routine decision-making roles.
1. Telemetry and Command gathering the data transmit
ted from the payload, deframing and demultiplexing it, reconstructing the original raw telemetry frames, storing it, and transmitting commands and/or data to the payload.
2. Payload Navigation – determining actual payload orbital
parameters, comparing them with nominal values, and making minor orbital corrections.
3. Monitoring interpreting received data to ensure
integrity of the craft and performing preventative and diagnostic tests and maneuvers.
All indications are that NASA is in immediate need of a more centralized, modular, and automated mission control center concept. This need for a reusable, centralized mission control center has already been recognized by certain groups within NASA. Des Jardains' POCCNET concept, reviewed below, provides an excellent overview of how mission operations could be cleaned up and standardized at the lower levels, providing a modular software foundation into which the specific scientific and technological needs of each mission could be grafted. At the higher levels, there are some AI methods that the Study Group feels are ready for immediate technology transfer, and others that NASA should invest in for longer term payoffs. We suggest several of the immediate and eventual payoffs from AI in mission operations below, and have included a brief survey of the state of the art in AI problem solving and programming languages.
4. Sequencing devising, coding, and transmitting se
quencing instructions to the craft for executing science experiments and remedying problems.
5. Data Interpretation and Display – capturing, formatting,
and displaying scientific and technological data from the craft for convenient use by humans.
3.2 State of the Art: Mission Operations
The view of mission operations developed by the Study Group is that there are three categories of human activity in mission control during a mission’s lifetime:
6. Manpower Coordination sequencing ground-based hu
man activities relating to the successful monitoring and science gathering of the mission; this includes such things as gathering together appropriate subsets of the scientific community for judgmental decisions, coordinating routine staffing of the monitoring facilities, locating technical experts to deal with specific problems, and so forth.
1. Intimate control activities, where human intelligence and
expertise seem to be demanded.
2. Mid-level intelligence problem solving tasks (real-time
flight sequencing, resource scheduling, automatic conflict resolution) where humans are extensively employed because of their problem solving and modeling knowledge, but where no judgmental decisions per se must be made.
Although the Study Group saw a wide spectrum of detail across the various projects and missions within NASA, every project and mission seems to demand these core activities. Indeed, it appears that only a small fraction of a mission's cost in manpower and planning derives from the unique scientific aspects of the mission; without a doubt, the bulk of missions operations is common to all projects within NASA.
3. Repetitive monitoring and control activities, where
enough intelligence and human intervention is required that humans are presently essential, yet where the tasks are unchallenging and wasteful of human resources.
Nearly everyone in NASA seems to realize this. Yet there seems to be such great inertia from NASA's early days of rapid growth that no one seems able to initiate cross-mission technologies that would coalesce missions operations. We saw one notable exception, however; des Jardains' proposal for an automated, reusable Missions Control Center (POCCNETRTOP #310-40-40). Des Jardains' proposal is well-conceived; but, as he points out, even the most ambitious automation
In this subsection, we highlight what seem to be the most important aspects of mission operations from categories 2 and