 |
 |
|
||||||||||||||
 | ||||||||||||||
|
Developing Sensors That Give Intelligence to Robots
|
|||
|
|||
| Which have the better auditory perception, humans or robots? | |||
|
On September 23, 2005, when the Bio-Mimetic Control Research Center (BMC) of RIKEN was opened to the public, noise was reproduced randomly from any one of six loudspeakers vertically arranged in two rows of three loudspeakers. Visitors to BMC were asked to compete with the robot that Dr. Hiromichi Nakashima and his colleagues have developed to see which was more effective in identifying which loudspeaker was on. How did the competition end up? "You may turn round to look when someone calls to you. This is unconscious human behavior, but it is more difficult for current robots to perform than complicated mathematical calculations," says Dr. Mukai. The phrase "sound source localization" means locating the position of a sound source. The current mainstream approach in giving robots the ability of sound source localization is the "microphone array" technique with multi-microphones, which uses the differences in the sound signals picked up by multi-microphones to identify the direction of the sound. However, what Dr. Nakashima is trying to develop is a sound source localization robot with only two microphones. The robot tries to aim its camera at the estimated direction of sound, and repeatedly learns the most effective way to estimate the direction by confirming the difference between the real and the estimated directions. "Use of a larger number of microphones is, of course, advantageous to sound source localization. Strange to say, however, many animals have only two ears. There must be some benefits, such as easier data processing or learning, in the fact that they have only two ears. We want to know what the benefits are," says Dr. Nakashima. However, how can we identify the direction of sound with only two microphones? A sound source is localized in the lateral direction on the basis of the difference of the loudness (sound pressure) and the arriving time between the sound arriving at the left and right microphones. For example, when a sound comes from the left side, the sound arrives at the left microphone earlier than the right microphone. Furthermore, the sound pressure at the left microphone is larger than that at the right microphone. What is more difficult is how to locate the vertical direction of a sound source. For performance improvement, Dr. Nakashima and his colleagues tried to install "outer ears" (sound reflectors) around the microphones (photo on left side of cover page). With the introduction of the outer ears, direct sounds from the sound source to the microphones interfere with the sounds reflected by the outer ears, thus causing sound wave cancellation between the direct and reflected sounds. The resulting sound, picked up by the microphones, exhibits some amplitude dips in the frequency spectrum because of the sound wave-cancellation (Figure 1 B). The dip pattern varies with changes in the location of the vertical sound source, and is used to estimate the vertical direction of the sound source. Now coming back to the result of the competition at the open house, it was not the visitors but the robot that won the game. However, the sound source that was used in the experiment was what we call "white noise," which contains an equal intensity of all wavelengths, and exhibits no dips in the frequency spectrum (Figure 1 A). When the white noise is used as a sound source, the robot can easily detect the dips caused by the interference of sound. Thus the robot won the game in its strongest field. However, a sound source generally exhibits some dips in its frequency spectrum. "Since our robot is capable of analyzing a combination of four or five dips, it can distinguish the dips that are attributed to the sound source itself from those caused by the interference of sound," says Dr. Nakashima, referring to the features of the system. However at the open house, there was a case when the robot could not locate the sound source with accuracy. It was the moment when a lot of noise was generated as the event captured a larger and larger audience. Humans have the ability to identify a talker's voice in a noisy environment such as at a cocktail party. Thus one of the future challenges for the robot is to develop a technique that gives the robot the ability to sort out only necessary sounds from amongst other noise. Dr. Nakashima and his colleagues are planning to conduct original experiments where auditory information and visual information captured on camera are integrated for judgment processing. When we hear a sound, and when the situation of what we see in the direction of the sound is almost in agreement, we tend to think that the sound is generated at the position of what we see. This is how ventriloquism provides the illusion of a doll speaking. However, when the ventriloquist is separated from the doll, we can easily tell that it is the ventriloquist who is actually speaking. Humans learn to tell the reasonable and appropriate distance that is necessary for matching the auditory and visual information. Dr. Nakashima says, "We will try to make the robot learn the appropriate distance." Maybe the day will come when a robot can enjoy ventriloquism.
|
|||
| Creating a smell identification robot | |||
|
Dr. Mukai points out, "From among the five senses, the one in which robots are the most inferior to animals and humans is olfactory perception. The concentration of a known gas could be derived. However, if the kind of gas is unknown, the robot can neither tell the kind nor the concentration of the gas. The only thing the robot can detect is the existence of the gas. This is typified by the fact that some gas-leak detectors in kitchens are responsive to hot sake." "We are conducting research on how to identify the kinds and concentration of gases by using semiconductor gas sensors, which are considered to be the most durable, and used for gas-leak sensors," says Dr. Yo Kato, Research Scientist. When the semiconductor in a semiconductor gas sensor is heated to a high temperature, gases are absorbed or combusted (oxidative reaction) on the heated surface, and the semiconductor gas sensor uses the changes in electrical resistance to detect the concentration of gases. However, conventional gas sensors have been unsuccessful in identifying kinds of gases. Why do they fail to identify the kind of gas? Because the change in electrical resistance caused by a single type-A gas molecule absorbed at the surface can be the same as that caused by two type-B molecules absorbed at the surface. This leads to an inability to separate A from B. At present, the mainstream approach for gas identification is to prepare and arrange many kinds of sensors that have different relationships between the kind of gas and the resulting change in electrical resistance. However, only about 10 kinds of sensors are available now because manufacturing itself is limited. On the other hand, Dr. Kato, Research Scientist, and his colleagues are trying to identify the kinds and concentration of gases by applying the concept of "active sensing," which actively changes the state of a sensor, thus periodically changing the surface temperature of the semiconductor. The electrical resistance shows different time characteristics with different kinds and concentrations of gases, which enables gases to be identified (Figure 2). According to Dr. Kato, "When A burns at 80 ºC and B burns at 100 ºC, the difference in temperature can be used for identification. Our principle is based on this concept. So far, we have confirmed by experiment that just a single sensor is capable of identifying eight kinds of gases. We think it possible to identify further kinds of gases by changing the period of the temperature variation on the surface of the semiconductor sensor, or by changing the upper and lower limits of the variation." He and his colleagues are advancing a study on how to mount this gas sensor on disaster-relief robots. For example, these robots are expected to detect gas-leaks at disaster sites. However, they need to detect gases in real time in an ever-changing environment because they are expected to move about, sometimes under a strong wind at a disaster site. Dr. Kato and his colleagues have built a gas detecting system that is capable of detecting gases in real time by providing a semiconductor surface area of 1 mm2 with a heater that can change the temperature of the semiconductor from 80 to 320 per second. In the photo on the cover page, Dr. Kato (middle) is holding a robot that is equipped with this sensor. The robot has three sensors at each of the apexes of a triangle, and the robot finds its way to the region where a greater condensation of gas is detected.
|
|||
| Towards the birth of "RI-MAN" in 2006 | |||
|
At BMC, researchers are working on research into "bio-mimetic" control, which is a technology that mimics the highly-sophisticated control functions of living systems. Dr. Zhi-Wei LUO, Laboratory Head of the Environment Adaptive Robotic Systems Laboratory, is playing the leading role to integrate main achievements of the center to develop their robot, which will soon be completed. The robot is called "RI-MAN." "We aim to develop a robot that can directly contact humans, for example, a robot that is helpful for nursing care. Our short-term task with RI-MAN is to develop the capability to hold an infant of about 10 kg in its arms. This will be a world-first challenge," says Dr. Mukai. The robot cannot hold an infant in its arms without tactile sensors. When we hold an infant, we will try to feel the position where the pressure is located and control our arms accordingly. In the same manner, when the robot tries to hold an infant, it should control its arms by feeding back the information obtained from the tactile sensors. Without tactile sensors, the robot may hold a person in its arms so strongly that it may cause harm to the person. However, existing tactile sensors for robots can detect only simple tactile senses such as "struck" or "touched," and the accuracy of signals from the sensors are insufficient to use for feedback signals. Why has the development of tactile sensors for robots been left on the back burner? "Basically because tactile sensors have not been required for conventional robots, which have been used in stable environments in factories, and only required to accurately perform routine operations. There has been no idea of covering a robot with tactile sensors that produce information signals on which the robot does its mechanical work," says Dr. Mukai. He made up his mind to study the structure of the human skin so that he can develop curved-surface tactile sensors that can cover the entire body and accurately detect contact points and contact strength. The human skin includes a slightly hard layer of outer skin and a soft layer of inner skin. The complicated structure of the skin allows the contact pressure to be focused on the receptor organs of tactile sensation for accurate signal detection. Dr. Mukai and his team members have completed a structure where the contact pressure is focused on high-precision semiconductor sensors by combining elastic bodies of different hardness and hard prongs (Figure 3). However, RI-MAN would be covered all over by cables if cables were used to directly connect the signals from the numerous semiconductor sensors mounted around RI-MAN to the central computer, which plays the role as the brain of the robot. To cope with this problem, they installed a small computer at the proximity of a sheet of semiconductor sensors (8 x 8 sensors), thus building up a system that can sort out and compress the information from the sensors and send the processed information to the central computer. For example, when one of the arms or legs of the robot bumps into something, one of the computers responsible for that portion issues a directive to withdraw it. This is analogous to the reflex action in human beings, because when a person touches something very hot, they instantaneously draw back their hand. In this reflex motion, a directive is issued from the spinal cord, not from the brain. Dr. Mukai and his team members, using an experimental robot arm wrapped with these tactile sensors, are advancing collaborative research with the Environment Adaptive Robotic Systems Laboratory on how to hold a person properly in the arms of a robot (top of the cover page, the two black sheets are tactile sensors). RI-MAN has speech recognition capabilities and is equipped with a sound source localization system and gas sensors developed by the Biologically Integrative Sensors Laboratory. When a person gives RI-MAN an order by voice to hold the child in its arms, RI-MAN turns around and looks at the person. Then RI-MAN moves toward the child and uses its tactile sensors to hold the child in its arms. In the future, RI-MAN will be able to use a gas sensor to detect when the baby has wet its diaper. The development of RI-MAN must be a step toward practical application of an intelligent robot that can live side-by-side with humans. 
|
|||
|
|||
 | |||
|
|||
[ Top ] |
"I really hope to develop a robot that is immediately on hand, when necessary, to help us lift heavy weights but stays out of the way. We will feel comfortable with the robot at home, and the robot should also be useful for household chores, nursing care, and crime-prevention," says Dr. Toshiharu Mukai, Laboratory Head, relating his dream. He and his team members aim to develop a robot that has the ability to do whatever is necessary in our changing daily environment such as in the home, creating an intelligent robot that can live side-by-side with humans. Dr. Mukai points out, "Computers, which act as the brain of the robot, have advanced considerably in recent years. What is most required now in creating the intelligent robot, is to develop sensors that can respond flexibly to changes in the environment." In the Biologically Integrative Sensors Laboratory, researchers are trying to develop sensors that can respond to environmental changes in the same way as humans, thus giving intelligence to robots.
|
Visualizing Structural Changes in Proteins and Elucidating Their Functions
|
|||
|
|||
| Visualizing structural changes by the multiple superposition method | |||
|
"I wanted to develop a new methodology for protein crystallography," says Kunishima, looking back upon the days when he organized his team in 2001. "The base sequences of the genomes have been clarified in a variety of organisms, including humans. However, no more than amino acid sequences can be known from the base sequence data obtained. Proteins comprising folded sequences of amino acids function in the body. Their functions cannot be understood without structural data. Structural genomics is a discipline characterized by comprehensive and systematic analysis of protein structures to elucidate all the information written in the genomes and to make use of the results for drug innovation and the like. I came here with the aim of establishing a new methodology for structural genomics." The Oligomeric Protein Crystallography Team is now undertaking X-ray crystallography of proteins using SPring-8, RIKEN's large synchrotron radiation facility, as part of the Protein 3000 National Project on Protein Structural and Functional Analyses, a national project sponsored by Japan's Ministry of Education, Culture, Sports, Science and Technology. With the participation of a number of universities and other research organizations in Japan, this project commenced in fiscal 2002 with the aim of clarifying the structures and functions of about 3,000 kinds of proteins within five years, and RIKEN, as the core institute, is responsible for analyzing 2,500 of these. The team is targeting oligomeric proteins. A protein comprises a sequence of amino acids (polypeptide chain) folded to form a three-dimensional structure, and a few proteins can function with only a single polypeptide chain. Many proteins function with a number of polypeptide chains gathered together; these are generically referred to as oligomeric proteins, and each polypeptide chain is known as a subunit. "Why do such proteins function with multiple polypeptide chains gathered together? Although this has long been an important theme in the relevant field, the mechanisms remain unclear. We are working to systematically elucidate the relationship between oligomerization and functions by analyzing the structures of many oligomeric proteins." Proteins function by binding to biological molecules, including other proteins. The biological molecules bound are called ligands, and the places where the binding occurs are called active sites. Individual proteins have different ligands. "Although proteins and ligands are often compared to the relationship between a lock and key, the actual situation is much more complex. Most proteins bind to ligands and change structurally, in order to perform their functions. Protein functions cannot be understood unless not only the shapes of the lock and key, but also structural changes, are visualized." In many cases, however, the structural changes are indeterminable by conventional methods because of their minuteness and complexity. To observe the minute structural changes and correlate them with the functions had been a major issue in structural genomics. Against this background, Kunishima's team succeeded in observing the minute structural changes in an oligomeric protein known as acylcoenzyme A thioesterase PaaI, and elucidating their relationship to the protein's functions. The PaaI protein, a tetramer consisting of four subunits, acts as an enzyme to catalyze the degradation of biodegradable plastics and other processes. The Oligomeric Protein Crystallography Team first demonstrated that the ligand binds only to two of the four active sites (Figure 1). Only half the plurality of active sites are being used; this is called the 'half-of-the-sites reactivity phenomenon,' theoretically predicted more than 35 years ago. However, the PaaI protein became the first common enzyme for which the phenomenon was confirmed. Furthermore, Kunishima and others proceeded to more extensive analyses to examine the mechanisms and roles of the half-of-the-sites reactivity. "Once we felt that we would have to give up our project because of the enormous difficulty," says Kunishima, looking back upon his past experience. "But we continued on without losing our hope, and finally reached the multiple superposition method. As a result, we demonstrated that when two ligands bind, the subunits rotate slightly to alter the structures of the remaining active sites, thus preventing the binding of additional ligands." In the multiple superposition method, the crystalline structures of a protein with a bound ligand and another without are superposed and compared. Structural changes are divided into two elements: rigid body changes for the overall motion of subunits as a whole and localized changes within the subunits. By repeating the superposition several times, it became possible for the first time to visualize minute structural change as small as 2 degrees of angle. "Although the half-of-the-sites reactivity appeared to be wasteful, this idleness proved to be important." Acylcoenzyme A, the ligand for the PaaI protein, is relatively large for a ligand molecule. PaaI binds to a large ligand as if wrapping the ligand by changing its structure. As a result, two of the four active sites become no longer available. It should be noted, however, that this disadvantage is outweighed by the advantage of effectively binding to a large ligand, an important aspect for PaaI. Furthermore, PaaI is capable of controlling its functions allowing another small protein to bind to the remaining two active sites. These advantages may be shared by all oligomeric proteins that exhibit the half-of-the-sites reactivity phenomenon. Human type III thioesterase, a protein similar to PaaI, is known to become activated upon infection with HIV, the virus that causes AIDS. Although its crystalline structure remains to be determined, its activity is possibly controlled by half-of-the-sites reactivity. Elucidating the mechanisms behind this activation is expected to lead to the development of AIDS therapeutics.
|
|||
| Relationship between oligomerization and structural stability | |||
|
"We obtained another interesting finding concerning the relationship between protein subunit oligomerization and functions," says Kunishima. His team determined the crystalline structure of 2-deoxyribose 5-phosphate aldolase, a tetrameric protein derived from the highly-thermophilic bacterium Thermus thermophilus HB8 (left panel in Figure 2). The crystalline structure of this protein had been determined for Escherichia coli and an ultra-thermophilic archaebacterium. Because even where the same protein exhibits structural variation among different organisms, comparison is also of importance. We found that subunit contact areas increase as the occurrence temperature rises (right panel in Figure 2). For another protein, however, the reverse finding was obtained. Structural analysis revealed that the contact area is constant between dehydroquinate synthase, a dimeric enzyme derived from a highly-thermophilic bacterium, and that derived from an organism occurring at normal temperature. This enzyme proved to utilize a portion of the contact surface as an active site. "In cases where the contact surface is associated with functions, such as by providing an active site, the contact area is constant. On the other hand, where the contact surface has no association with functions, it is likely that the protein acquires thermal stability by increasing the contact area. Analyzing more samples would lead to the elucidation of a universal mechanism." The results can be fed back to technical development. X-ray structural analysis requires making single-crystals of good quality from solution, but some proteins do not permit making such crystals. A protein crystal comprises regularly-arranged, mutually-binding molecules. This is the same state as oligomers. According to Kunishima, high-quality crystals could be fabricated by artificially altering the contact surface, provided that the relationship between oligomer contact surface structure and stability is elucidated.
|
|||
| Technical development for 100-fold efficiency | |||
|
To date, the Oligomeric Protein Crystallography Team has determined the crystalline structures of more than 60 proteins. In fiscal 2004, seven researchers determined 38 structures. "I began analyzing protein structures at my graduate school in 1989, and I determined the crystalline structure of only one protein during the five years that followed. That's a common case. At present, every member of our team determines not less than five structures per year," says Kunishima in confidence. "Although structural genomics is being vigorously studied in the USA and Europe, ours and other laboratories participating in the Protein 3000 Project have seen much better achievements at world levels both quantitatively and qualitatively." Then, what is Japan's weak point? "Now, nothing," Kunishima answered definitely after a while. "However, negligence would soon result in other countries outrunning us. Emphasis must be placed on how we should maintain this lead in the future." He says that what is required for Japan to continue to be the top runner is technical development. "Because we began with easily-analyzable proteins, all remaining proteins are difficult to analyze. From now on, the key to successful analyses will be how to improve the quality of crystals. And we are now focusing on developing an automated system that will enable researchers with little experience to perform analyses through the right procedures. To date, structural analysis has been available only to highly-skilled researchers. In this situation, analytical efficiency never increases. We aim at increasing the overall efficiency to a level 10 times that with the currently-available automated system, or 100 times that by manual analysis, by streamlining the individual steps." (Figure 3) "Although this is our ultimate goal," Kunishima talked on the future prospects for his work. "The highly-thermophilic bacterium Thermus thermophilus HB8 has about 2,200 kinds of proteins. I want to elucidate the structures and functions of all these proteins and simulate all biological phenomena in a single cell using a computer." This project is called the 'Structural-Biological Whole Cell Project of Thermus thermophilus HB8' (http://www.thermus.org/), promoted by the Structurome Research Group at the RIKEN SPring-8 Center, in which the Oligomeric Protein Crystallography Team participates. They have already succeeded in producing about half the relevant proteins, and determining the crystalline structures of about 20% of them. "Further advances to elucidate the structures and functions of all human proteins and simulate all biological phenomena would dramatically change the methodology of drug innovation. This is called 'systems biology.' Although it is developing as a new research area, the situation appears to be stagnant, because of a lack of essential information. Systems biology does not go well without the information elucidated through structural genomics; to this end, research into structural genomics and technical development must be increasingly advanced-that's our mission."
|
|||
|
|||
| |||
|
|||
[ Top ] |
Researchers at the Oligomeric Protein Crystallography Team in the Advanced Protein Crystallography Research Group of the RIKEN SPring-8 Center are working to determine the crystalline structures of oligomeric proteins and elucidate their functions, and to develop relevant techniques based on the feedback of analytical results. "Proteins are often described as structures that cannot be elucidated unless their shapes are demonstrated. However, morphological data alone is not enough," says Naoki Kunishima, leader of the Oligomeric Protein Crystallography Team. "Proteins often undergo structural changes by binding to other proteins in order that they can perform their functions. The elucidation of structural changes must be emphasized." Kunishima is drawing attention since he established a new method known as 'multiple superposition,' which visualizes minute, complex structural changes. This article reports on the revolutionary developments in structural genomics, an emerging discipline aiming at comprehensively and systematically analyzing protein structures and functions to understand the information written in the genomes, and to make use of it for drug innovation and the like.
