|
||
|
||
|
|
|
|
|
|
|||
|
|
|
|
|
|||
|
 
|
|
|
Ramesh Jain is a world-renowned pioneer in multimedia information systems, image databases, machine vision and intelligent systems. He is the founder of three companies which he managed in their initial stages, then turned over to professional management. He is also an educator, having taught at several important U.S. universities, and is currently an Eminent Scholar at Georgia Institute of Technology in Atlanta where he is researching experiential computing. We are a people in transition, says Dr. Jain, in the process of moving from an information society to an experiential society, which will require systems and applications that allow us to immerse ourselves in rich sets of multimedia data and information that we can directly observe. Gartner Fellow and vice president, David McCoy, met with Dr. Jain in Atlanta to find out exactly what experiential computing is and how it is going to change the world in which we live and work. Interview conducted 31 July 2003
David McCoy: Ramesh, what is experiential computing? Would you define it? Ramesh Jain: Yes. When I was at the University of California, San Diego, I started thinking about how the information landscape is changing. It started to become more and more clear that because of Gutenberg's revolution, systems designers generally think in terms of alpha-numeric information, but in our lives, there are lots of different data sources — audio, visual, tactile. So if you can explore and find the information you are looking for on your own terms, combining audio, video and other elements, that's experiential computing. McCoy: You actually built a sports application as an example of an experiential computing environment. Tell us a little bit about what happened there, and what this thing actually did. Give people an idea of what it's like to work within an experiential computing environment.
The idea was to record a football game with many camera sources and once the game was over, to convert it to a time machine so that all the information about the game was available in any form a user wanted. Then a person could say, "Oh, I want to just see highlights," or "I want to see just the defensive plays." "I'm interested in this player. Show me only the interesting things that this player did." The system automatically created this huge database, and anybody could go and play, and see what they wanted to see. So the idea was to combine the audio of the commentators with the play-by-play and with video coming from different cameras so anyone searching could say, "This is what I want to see." When you start putting the person instead of the information in the center, that's when it starts becoming experiential. McCoy: Tell me more about how experiential computing would change what we do today? Jain: The way computing systems work right now, you go to Google and enter a query: "I want to know everything about George Bush." In point 36 seconds, Google sorts through billions of pieces of data and gives you 10 million answers listed in a sequence. But nobody goes beyond the third page; they say that less than 10 percent of users have ever seen the third page of Google. This is a query-centric — or what I call an information-centric — environment. You ask questions, you get the answers.
So Google is a system that uses the legacy information-centric environment. I love it, but I hate it. I love it because there is certainly nothing better, but at the same time it is so primitive that it is useless in most applications. On the other hand, if you are presented with and can explore a holistic picture, you are in an experiential environment. McCoy: Let's put you on the spot for a moment. What would Google look like in five years if it were designed as an experiential computing environment? Jain: That's a very good question. I've been thinking about this, and I really want to develop this system. The idea would be: I will ask the question. Number one, it will not be in the same form as it is now of inputting a query. It will be in a real exploratory form. So when I want to know about George Bush, I will be presented with a wired map, a time line, all kinds of things that have been written about George Bush or are about what Bush did. Let's say I'm interested only in one particular thing in Iraq about George Bush. I can expand on this, and as soon as I select Iraq, it will choose only the information about that. I can then say, "Okay, show me what they were saying before the war and what they were saying after the war." I can not only see those things, but I can also compare those things. So this way, the search becomes WYSIWYG — what you see is what you get. McCoy: One of the things it looks like from what you've described is that complex visualization is going to be a key part of experiential computing. I heard a talk a few years ago about how corporate board members could, in the future, review financial statements by wearing virtual reality goggles and walking through an immersive system where they look like they're in a cave. Financial performance for Asia-Pacific would be represented by stalactites on the roof. Stalagmites on the ground would represent performance in the UK region. And they could actually use this immersive system to analyze the financial performance through a 3D modeling technique.
There was a movie, Disclosure, which used that type of visualization. McCoy: Yes, and another more recent movie, Minority Report, used some stunning concepts. It seems like we've been looking for complex visualization for years. Is experiential computing getting us any closer? Jain: First thing, I think there is a very, very big difference between virtual reality and what we are trying to do, which I sometimes call "real reality." We are not trying to model the world in a computer and then let you navigate through this. We are trying to model the real phenomenon as it is happening in the real world, and allow you to explore that, as you do right now. When you go to a football game, you can turn your head and choose to watch the coach instead of the game, or observe fans in the bleachers or the cheerleaders. We are providing exactly that facility through experiential computing. The second thing is, we do not want to use any of the artificial gadgets of virtual reality. Human beings function very well in their natural environment, but as soon as you start adding constraints, the experience starts to become artificial. In experiential computing, we want people to use their senses in their natural setting. McCoy: One of the things I learned years ago is that we each perceive reality through a different set of lenses. An example you've used — I believe it comes from India — is the story of several blind men who come upon an elephant. One grabs the ear and thinks he is feeling a leaf. Another grabs the tail and thinks he's feeling a vine. And so on. The point is we all interpret what we see based on very personal contexts. How does that idea fit in with experiential computing? Jain: A very perceptive question, and in fact, a lot of people are a lot more aural than they are visual which is why I never call it a "visualization environment." I always call it a "presentation mechanism" because the presentation environment will decide, depending on the person, what kind of senses should be used, which senses will be the best in a particular situation. In some cases, it may be an abstract mechanism like text that is best. In other cases, it will be audio. And still others, video. The most dramatic way I came to understand that was when one of my friends asked why is that many people, when they see Harry Potter movies, are completely disappointed because they think the book was much better. On the other hand, some people like the movie much better than the book.
McCoy: Good point. So the idea of visualization, then, is that it's really not visualization; it is expression — complex expression — being targeted to the recipient's best mode. A blind person, obviously, is not going to care much for visualization; he or she would need a different kind of interaction. Jain: Yes. It's the presentation of data in the most useful form for each individual person. McCoy: Two other things you speak of in the context of experiential computing are Event Web and Folk Computing. Would you define these terms and tell us how they relate to experiential computing? It sounds like they are types of experiential computing implementations. Jain: That's right. And I find them very exciting, in fact, because they are slightly unusual. Let's talk about Event Web first. We all have been influenced by the Web quite a lot, but when you think about it, the Web today is still following Gutenberg's legacy; everything on the Web is a page. We talk about number of pages. We put documents on pages on the Web. Now think about this: What if you put events on the Web? That means that in place of writing an article about our meeting here and posting it to a page on the Web, we start by placing multiple cameras here, along with some other sensors, and you put the recording on the Web so that people can explore this conversation in the same way as the football game I described. So that at any time they could experience what you and I are doing, see how we are doing it, choose their perspective. This is the concept I'm trying to advance. It requires very interesting technological challenges because each search that is done becomes different. Time becomes the most important factor because you are more interested in what's going on right now, or what happened in the near past, etc. Cyberspace has brought some very powerful concepts to us, but for human beings, time and location are the most important things and cyberspace has removed them. When I send you an email, you have no idea if I'm sending it from Tokyo or Bombay or right here in Atlanta. Similarly, the time element becomes very different in cyberspace. How can we bring back those elements? In Event Web, the focus is, in place of a document about it, the event itself. Documents provide you information. Event Web gives you the experience. McCoy: There's something else you're working on called e-chronicles which is, if you will, a sort of digitization of your life history. But if you just let the camera run on the average part of anyone's life, it's not very interesting. I've got tons of videotape, but they are write-once memories. We record them and stick them on the shelf. How do we make use of this huge history we can amass with Event Web or with e-chronicles?
An interesting question. When you record events over time, lots and lots of events, how do you provide access mechanisms so people can see the events they want? McCoy: Yes. Maybe CSPAN can give us an example. All we know of George Washington and the early U.S. government is what we have in history books, and that's been filtered through someone's mind. What was written down may or may not correspond to reality, but it's all we have. Now, with CSPAN, we can run video tape of every second of every Congressional session and that allows us each to interpret an event as we see it. But how is experiential computing going to add value to that? Jain: This is one thing I love about e-chronicles. Let's step back and look at the way history is written, or the way our society evolved. Until recent history, writing was the only way to preserve things and writing, by definition, is subjective, as you point out. We all know, for instance, that history is written by the victors, from their perspective. Now we have added photographs and video bringing in the sensory element. And when you start doing that, you are reducing the subjective aspects. That's why video is more compelling and more powerful than writing. So now, coming back to e-chronicles, or the experiential computing aspect of it, in place of somebody writing history, you have a record — video, audio and the subjective element is minimized. Then people can see and analyze what has happened rather than relying on somebody else's filter. The indexing mechanism and the access mechanism will allow you to "see" for yourself and bring your own insight to it. McCoy: I wonder if the subjective element really does disappear though. If I show 20 people a video tape, there would be 20 different opinions. One would say, "Oh yeah, they were having a hostile debate." Another would say, "It was good, worthy competition." Jain: You mentioned the blind men and the elephant. Well, a lot of Indian philosophy is based on that idea — that the world is a set of data and when different people observe the data, depending on the observers' level of context and knowledge, they are going to see the same phenomenon differently. McCoy: What I believe we will be seeing here is less second-hand observation, fewer second-hand recipients of content and information, and more first-hand. The whole idea of experiential computing means that we will experience and interpret for ourselves as opposed to having it pre-digested and handed to us. Jain: That's exactly right, David. The computers and the environment become your assistant, but you will be doing the experiencing. The computer is just providing the data. McCoy: You also mentioned Folk Computing... Jain: That's one of my very favorite topics so I don't want to let that slip. To me, Folk Computing is very different from, but in some respects similar to, the kind of thing that happened in the late 1970s when Steve Jobs and Steve Wozniak began championing the idea that mainframe computers were not for everybody and that gave rise to personal computers. Today, about 700 million people in the world use computing and related things. The remaining 5.7 billion people are not helped by computing. What is the reason? The keyboard is an obstacle. Language is an obstacle. Available content is an obstacle. Education levels are an obstacle. I found it very interesting when I was in India working with some people in Dehli that when they tried to teach poor villagers about how a desktop works on a computer, the villagers had never even seen a desk. They didn't know what a file is. Or a trash can. So they cannot possibly understand the desktop on a computer. If we want computing technology to benefit many of the other 5.7 billion people in the world, we need to come up with devices, mechanisms, thinking about how the information should be stored, how should it be distributed, how should it be presented — all in a very different way. That is what I mean by Folk Computing. McCoy: Right - the metaphors we impose can be a limitation. So the key thing with experiential computing is that it lets you expose yourself to the content in the way you feel most comfortable. And that means a lot of computational change, a lot of interface design change, but I like the idea that it's not about modeling. It's not a synthetic world; it's just letting us do with computers that which we do in our daily lives. Jain: That's it. So there will be models, but the models will be in the background. And that's what Folk Computing is about — how can we come up with devices that are purely experiential in nature.
Ramesh, computer technology continues to speed up our lives. The average pace over the past ten years has eclipsed the pace of the previous ten years — you see that we now have more change in a year than some people saw in their entire lives. At what point, do you think, are we running so fast, becoming so wired, so connected that our world is no longer fun? Jain: I think people will control that. Because when it goes beyond human beings' limitations, they will stop it. I will share with you something very interesting that at one time bothered me, but later on I was impressed with. When my son was a teenager, he would come to my room where I would be watching TV. He would grab my remote control and being a sports fan, start flipping channels. He would go from one channel to another to another and I believed this was the impact of video games on his attention span. But then I started quizzing him and I understood that he knew on each and every channel what was going on. It was not that he had a poor attention span, but that he could get information and assimilate that information much faster than I. He was trying to find out where the most interesting things were. He was trying to maximize his knowledge of the games that were going on at that particular time. McCoy: I've always said that the business world of 2030 is going to be run by kids who are learning right now on Nintendo Gameboys, and they will take that mentality to work with them. Jain: That's exactly right. McCoy: I really believe we're going to see business run with a joy stick and with mouse buttons and with jumping characters representing stock transactions or representing inventory. You run, and jump up and down, and slay the bad inventory shortage. If you do, you gain a power pill that represents a portion of your annual bonus. It's going to be a video game reality. Jain: Whatever metaphors people are familiar with, they are going to use. Like when you are in the military world, you color-code everything. McCoy: It sounds like some of the things we've talked about are valuable and doable right now. But what extremely difficult thing can you imagine that would just be shocking if we could do it right now? What would be the proof case for experiential computing that would get everybody jumping up and down about it? Jain: I haven't given enough thought to this, but some of the applications that immediately come to mind are the ability to move to tele-presence-type applications. The kind where I am sitting here and if I want to interact with my mother, who is right now in India, I can work in a network environment so that she and I both feel that we are sitting in front of each other and talking. And including that I can feel the heat that she is feeling there or see the rain that she's seeing, and she can feel and see what's happening here. So I am working on how to bring all of those five senses that are required to feel the same environment. Can that happen in the next two years? No. Can that happen in 50 years? Possibly yes. McCoy: And it would be possible too, to see your mother in India but not feel the heat. To be able to turn that channel off. Jain: Exactly.  |
|
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
