Introduction

This article will discuss an idea for placing virtual reality into reality to form a shared-augmented reality, occupying the existing reality with which we are familiar.

The idea is built on various concepts that have been proposed before in some form. The key ideas are:

• providing a virtual reality which is overlaid on reality, so that objects in the virtual world have real world locations and can be viewed, or interacted with, by people moving around the real world.
• doing this without the need for expensive hardware to be installed locally to provide the virtual reality. I will not be proposing that some kind of virtual reality server is placed on every street corner to turn a country into a Star Trek type 'holodeck' [1, 2]: I will be proposing something more practical that we can create more easily and much more cheaply.
• allowing normal people to make changes to the shared-augmented reality. As an example, if I am sitting on a park bench then I should be able to make some kind of three-dimensional object and place it on the ground, or in the air, near me. When I have walked away, you should be able to see this object. This is very important to the proposal and dramatically affects the whole character and utility of the idea.

Existing Ideas and Technologies

The individual ideas on which this is based are not new: some of them are quite old. I hope to persuade the reader that the synthesis of all these parts into something else is radical and interesting. Before I describe the proposal, we will examine some existing technologies and ideas.

Creating three-dimensional simulations of environments which can be 'explored' using a computer that allows the user to change his/her point of view to interact with the reality. The earliest reference known to me to the modern idea of virtual reality is by Arthur C. Clarke in his science fiction novel The City and The Stars [3].

Virtual reality sometimes involves specific hardware. Specifically, headsets can be used that place different images in front of each eye. A computer can construct these images to provide a stereoscopic (three-dimensional) image to the user. Changes in the position and attitude of the headset can be sent to the computer so that it can update the 'scenery' accordingly, in real-time, to give the user the illusion that he/she is moving around a virtual world.

SecondLife^™ [5] is operated by Linden Research Inc. [6] and can be visited at www.secondlife.com. I recommend exploring it to get an idea of the purpose of this article. The connection between SecondLife^™ and some of the ideas in this article is quite strong: I could almost have called this article Putting Second Life Out There.

This is one description of SecondLife^™:

SecondLife^™ is an online virtual reality system with an interesting property: users explore a common space and can make modifications to it. This is extremely significant to what I have in mind. As an example, as well as users being able to see each other's 'avatars' and interact with each other, if a user places an object on the ground then other users will be able to see that object. This may not appear to be too significant: after all, interaction at this level occurs in many online games. What makes it much more significant is the extent to which the environment can be modified by users.

Users can use construction tools to build three-dimensional objects with which other users can interact. They can obtain the rights to hold virtual land and can create elaborate buildings and other structures on this land or in 'common' areas known as 'sandboxes'. They can use a high level language to create scripted objects that can even be let loose into the environment to behave autonomously.

This whole idea has extreme scope. SecondLife^™ is effectively a three-dimensional version of the existing internet. The existing internet has often been said to provide a 'cyberspace' - a kind of virtual environment in which, as well as interacting with each other, users can make their own pages which can be visited by other people. SecondLife^™ takes this idea and applies it to a three-dimensional world which is changed in real-time by the actions of the humans interacting with it.

One important concept of SecondLife^™ is that it does not have a specific use. It is a place, not a specific tool, just as the existing web forms a type of place. The 'inhabitants' of SecondLife^™, from the comfort of their homes, have done many things, just a few of which are:

• making houses in which to chat with friends [8]
• making communal spaces such as nightclubs [9]
• making shopping malls for the sale of virtual objects [10]
• fighting simulated battles in 'damage enabled' areas using scripted weapons that they make themselves [11]
• even meeting to conduct real-world business deals

What I am going to propose, however, will go far beyond all of this.

The facility provided by mobile phone networks to send short text messages [12] from one mobile phone to another. When the idea was first proposed many considered it unlikely to succeed commercially. What detractors did not reckon with was the sheer potential for ubiquitousness of this technology. The same technology that can be used to send information discretely into a business meeting can also allow drunk people to forward rude jokes to large numbers of friends. The fact that it can be used by business people and drunk people sums up what made SMS ingenious.

The SMS idea can be considered to be so successful because, like SecondLife^™, it does not provide a solution to a narrow problem, but instead meets the more general problem of providing a virtual space - we can think of it as 'SMS-space' if we like. Like SecondLife^™ and the web, 'SMS-space' is not specific to a particular purpose but provides an environment which a person can use for purposes of his/her own devising - in the case of SMS it can be used in almost any aspect of his/her personal or social life.

The phone network itself can be considered to provide this sort of ubiquitous virtual space, but SMS serves my purposes better as an example. What I want to capture from SMS is this idea of pervasiveness - a technology that builds a virtual place that fulfils so many communication needs that its use becomes a general part of life and a habit to many people while they move about in their daily activities.

I will not really be using the technological ideas of SMS in this article: rather I will be attempting to capture the idea of complete social ubiquitousness that has made it so successful.

Computers are now being developed to be worn by the user and used while walking around [13]. One component of these is typically a computer screen placed in front of an eye like a head-up display. This can be used to put computer data in the user's field of view while at the same time allowing him/her to see where he/she is going. Display devices to do this could be built into devices resembling spectacles.

Input devices for wearable computers could be novel and could include styluses and screens, voice control or gestures with the hands that are observed by cameras.

A network of satellites allowing the location of a device containing a GPS receiver to be determined to a tolerance of a few metres [14, 15]. GPS receivers are now reasonably cheap and are built into many handheld devices. It will not be too long before they are a standard feature of mobile phones.

GPS does have one limitation which we will need to deal with: it does not work inside buildings. I will ignore this problem for much of the article and simply refer to whatever means is being used of establishing the position of a device as 'GPS' in most cases. There are solutions to it and I will be mentioning them.

As I already mentioned, virtual reality headsets can be used to construct a false, three-dimensional reality by presenting different computer generated images to the left and right eyes.

This is usually done to immerse the user completely in a false reality, but it does not have to be like this: there is no reason why a virtual reality cannot be overlaid onto reality so that a user can experience virtual reality and reality at the same time.

How would this work? Imagine the helmet mounted head-up displays used by attack helicopter pilots that project computer graphics in front of an eye. The pilot is not blinded by this. He/she is still looking through transparent glass and sees a mixture of reality and computer graphics. This idea has been used in wearable computers and it is now possible to buy computer screens that can be mounted in front of your eyes. These are transparent, allowing you to look through the screen at the same time as seeing the graphics on it: the graphics appear superimposed on your view of reality.

Suppose now that we showed a wire-frame image of a cube to your left eye and, to your right eye, an image of a wire-frame cube viewed from a slightly different angle. If we got our mathematics right your brain would be tricked into seeing a three-dimensional object and you should see a cube floating in space. For a normal virtual reality headset you may see this cube floating against a blank background, but now we are using computer screens which you can look through: you would see the cube floating in front of you at the same time as seeing your existing surroundings. If you stood in a room, in front of a table, for example, you could see the room and the table, with the cube hovering above it.

If just this were done, the effect would not be all that realistic. When you moved around the cube would stay in the same place in your field of view - a clear reminder that it was not real. This could be dealt with though: imagine that we tell the computer that is controlling this that the cube is at a specific place in the room (above the table for example) and we allow the computer to monitor the movements of your head by means of sensors on the glasses that you are wearing to generate the illusion. In this situation, when you turned your head or moved from one place to another, the computer could sense your motion, determine your new position and adjust the graphics presented to your left and right eyes accordingly, so that the cube would appear to remain in the same place.

A computer and some glasses containing built-in display screens can be used to overlay virtual reality onto reality and make you see computer generated objects at specific places in reality that are not actually there, your movements being continuously monitored to update the images to maintain the illusion.

Glasses that overlay virtual reality onto a wearer's normal vision have been used in science fiction, but the idea is not too extreme. It has already been done and systems are already proposed to do this for specific uses. Examples of systems that have already been proposed are:

• a headset that allows a surgeon to view a patient's body in reality, while models derived from CRT scan images are superimposed onto his/her view of reality in a way dependent on where he/she is looking. The effect of this is to give the surgeon the illusion of X-ray vision, giving him/her a better intuitive idea of the surgery to be performed.
• a headset that allows people who maintain underground networks of pipes and cables to view images derived from three-dimensional models of these networks, superimposed on their actual view of the ground. As with the medical example, the idea is to give an illusion of X-ray vision, so that the person doing the work can 'see' the pipes that are hidden underground before he/she starts to dig.

These ideas are interesting, but they could be taken further than simply having reality augmented while you walk around in it. The idea that I will propose will be much more pervasive than this.

Siemens^™ and Location Specific Information

Siemens^™, a mobile phone manufacturer, has proposed this idea:

Users of mobile phones can already interact by sending pictures and text messages to each other, but what if users could leave items like this in places so that they could be detected and picked up by other users?

As an example, if you were in a nightclub you could take a photograph using the built-in camera in many mobile phones and tell your phone to 'leave' it where you were standing. For all practical purposes, the photograph's data would be dropped onto the floor. When somebody else with a similarly equipped mobile phone stood in the same place, or reasonably near it, their phone would 'detect' the photograph that had been left there and they would be able to view it on their own phone's display screen.

This is actually quite a profound idea. It allows mobile phone users to leave text, images and sounds all over the world to be picked up by other users.

As an example of how it could be used, one of the originators of this idea at Siemens^™ has suggested that you could visit a restaurant and write a review which you leave outside. Anyone about to enter the restaurant would then see the review and read it.

At first sight the idea may seem extravagant. If humans could leave data anywhere then does that not mean that computer hardware would be needed all over the place to store that data? For example, if I am able to leave a review outside a restaurant does it not mean that some sort of computer system would need to be installed there to receive and store my review and then transmit it to anyone else who passed by? Does this not mean that the idea would need a world saturated with computer devices for it to work?

In fact, there is a much more practical way of doing things. Mobile phones are now built containing GPS (global positioning system) receivers and this means that mobile phones can find out where they are. This could be used to store the data centrally.

When you chose to leave data - for example, a picture or text - at a specific location your phone would check the GPS coordinates and find out where you are. It would then send the data that you want to leave, together with the GPS coordinates, over the mobile phone network to a centralised server where it would be stored. This means that the data is stored centrally, even though it appears to be in all kinds of different locations. The server would contain all the data left by everyone else, no matter where they were when they left it. For each item of data there would be the GPS coordinates of the user (where he/she was on the map when the item was left) and the data itself (such as the text or picture).

When a user walked around his/her mobile phone would continuously determine its GPS coordinates and would send these to the central server. The central server would check for the existence of any items left by previous users that had similar GPS coordinates stored with them and, if it found any, it would send them back.

The result is that items left by users get sent to a central server and items received by users come from a central server, but the user is given the illusion that the items are being picked up from the environment around him/her.

The advantage of this sort of system is that the main infrastructure needed to do it is the mobile phone system itself, which is already in existence. The lack of any reliance on computers being placed everywhere means that all that would be necessary would be to produce phones with this sort of functionality and build some more functionality into the computing facilities of mobile phone networks. It would always work provided that you could get a mobile phone signal: nothing would stop you leaving a note in a field for any other passers by to pick up.

This idea has been used in fiction by William Gibson in his novel Virtual Light [16]. This book featured glasses that use a more exotic sort of input technology than we will be considering here to overlay information onto reality. One use of the technology was to show labels on objects in the real world.

These ideas are interesting, but they could be taken further than simply having reality augmented while you walk around in it. I am interested in just how far they could be taken. The idea that I will propose will be much more pervasive than this.

What is the Idea?

Siemens^™'s idea, while an interesting start, is only a small piece of what could be done. This is what I envisage:

A world in which the entire surface of the planet is turned into a three-dimensional hybrid of reality and virtual reality. A world in which people can construct three-dimensional objects, just as they can in SecondLife™, but in reality, not just on a computer system. If we do this we will have got virtual reality out of computers and into the real world.

Some readers may wonder what is new here: after all, have I not mentioned systems which overlay virtual reality data onto reality to let people 'see' buried pipes or to allow doctors to operate?

What I propose goes much further than this. I suggest this:

We should put the ideas of SecondLife^™ - those ideas of making a public space which is altered in real-time by its users - into such a system.

The effect of this will be dramatic and will be:

A shared, virtual space, overlaid on physical reality, which can be altered in real-time by its users, giving those users the illusion that they can actually edit physical reality while they are walking around in it.

This means that people will be able to construct three-dimensional objects and leave them anywhere they wish and that other people will see them and be able to interact with them. The idea of anyone being able to do it is really important. Without this feature it would simply be a system imposed from above. This idea of getting everyone to do it is what would establish the idea as a pervasive system like SMS.

This may be an awkward idea to grasp. To give a better idea of what I mean I will give some examples.

Scenarios

Let us imagine that it is a day in the near future and that technology of the type that I propose is in use. We will look at how it is impacting on the lives of some people.

Scenario 1: Amateur Geology

Darla is taking a part time course in geology and is visiting a quarry well-known for its interesting rock formations. After entering the quarry she puts on her SAR (shared-augmented reality) glasses.

These connect to her mobile phone, which is adapted to use the SAR system, using a short range Bluetooth^™ connection, though some of her friends have SAR glasses that contain the mobile phone as well so that they do not need to carry a separate phone.

With the glasses worn Darla's view is unobstructed - she can still see the quarry around her. When she presses the 'on' button other information is projected onto the glasses and appears superimposed on the scene in front of her.

Other people have been here before her and labelled the features of the quarry. In full three dimensions words hang in space, with arrows pointing to various types of rock, as if they had been placed in the air as solid objects, Schematic lines drawn on the rocks show different strata layers. A lot of people have been to the quarry, so Darla chooses to filter out some of the information. She notices that some of the information has been 'left here' by her course tutor and she filters out all the other data, so that she can more easily see the various notes, labels and outline sketches that he has placed over the scene.

The thought of the data being 'left here' amused her for a second - like everyone else she has fallen so easily into the illusion of data being 'left' in three-dimensional reality - in fact it is all on a server miles away - she does not even know where it is.

Before leaving Darla adds a few items of her own. She notices an outcropping of rock that nobody else seems to have labelled. She walks to the rock and points into the space in front of her. The glasses detect the hand movement and recognise that she wants to put a label here. She moves her finger in the space in front of her to write the letters and as she moves her finger she sees a green line being left in the path of her finger - a hand writing recognition program converts her scrawling into a neat looking typeface and another movement of her hand makes an arrow pointing to the rock.

Scenario 2: Fun in the Pub

Fred is at a table in the pub with a group of friends. Jim has just lost at pool again and is having some fun made of him about it.

Fred sees that some of his friends are wearing SAR headsets. People tend to leave 'graffiti' hovering in space in the pub and some of his friends are contributing to it. A lot of the graffiti is three-dimensional objects of various kinds scattered around the room. Some are a little.. adult… in nature. Various joking sketches and comments float near the dartboard.

It is not all graffiti though: a menu for the pub's bar food floats above their table.

With a malicious grin, Fred reaches for his mobile phone. Scrolling through a display of clip objects (the term has followed on from the earlier term of 'clip art' in desktop publishing) he selects a picture of a pool table and adds a frowning face. Shortly afterwards, Fred's friends are laughing as a three-dimensional model of a pool table, complete with a frowning face, is hovering above where Jim is sitting.

Jim is bemused - he has not bought a SAR unit - he never really got into the habit of using SMS when it came out either - even if they were not above his own head he would not be able to see the graphics.

Fred wishes that Jim would buy a SAR unit. He may be able to set the virtual objects to follow Jim around, though, of course, Jim could prevent this by locking the location of his unit from access by other people's objects, but Jim would probably not bother to do that.

Scenario 3: Ancient Ruins

Sandra is with a school trip visiting a Roman fort at Hadrian's Wall, an ancient fortification in England. The site is in ruins and little more than the foundations of buildings remains.

This is a popular site and a lot of objects and scripts have been 'left here' so that some filtering is needed to avoid being overwhelmed by information. Sandra hears her teacher asking the class to remove everything except an overlay prepared by the local tourist board. Sandra calls up the menu on her SAR glasses and sees it hovering in front of her. She removes all the objects and then selects just the tourist board's overlay. She gasps as computer generated walls rise from the ruins in front of her. Ghost-like figures of long dead people appear shimmer into existence in front of her and start going about their business in the fort. One walks straight at her, but seems to pass through her. It is, of course, all an elaborate illusion produced by computers. None of this information is even here - it is all on a remote server.

The rebuilt fort and its ghosts are impressive, but Sandra finds shared-augmented reality much more fun when normal people like her can make their own contributions. She allows her SAR unit to detect other people's work and she sees the project work of a previous school trip scattered among the ruins.

Scenario 4: Country Walk

Daniel Johnson's family are in the Lake District, an area of natural beauty in England. Using their SAR glasses, the Johnsons saw a note left in the car park of their hotel suggesting a walk and they have followed it. When they needed directions they have looked through their SAR glasses and seen a pattern of glowing footprints on the ground, left by the walk's creator, presumably during his own holiday.

The walk has taken them to the top of a hill with a spectacular view. People have left notes here, hovering in space, to record their visits. Gazing out over the landscape the Johnsons see that someone has put words above various landmarks in the distance to label them.

A table, which is also just a computer generated illusion, stands in front of them and a book, also an illusion, of course, appears to rest on it. It has been placed there by an enthusiast of Wordsworth. Daniel touches it, expecting words to appear, but, instead, a three-dimensional figure of Wordsworth appears and starts to recite a poem that seems to fit the spectacular scene.

None of this is real: the landmark labels, the notes, the table, the book and Wordsworth are all on a remote server in London.

Scenario 5: Planning Permission

It is a sunny day and, relaxing in her garden, Elizabeth puts on her SAR glasses to read an email. The glasses are set to show all features of the local shared reality and she is taken aback: there is a patch of wasteland nearby, but a computer generated building now stands on it!

Interested, she walks across the road to the 'new building'. It is clearly a SAR illusion, but the detail is high. It has been made with 1:1 scale and stands there in three-dimensions. A notice nearby informs her that this is a planning application and is here for public viewing.

A planning application? She must have missed seeing the physical notice. She can hardly miss it now though. Last year, the local authority decided that the architect's plans and drawings submitted with a planning application were inadequate for allowing people to come to an informed opinion. When planning applications are made for buildings now, it is often required that a computer generated, three-dimensional model of the building be placed in the shared space in the exact space that the proposed building will occupy.

Elizabeth does not think she will object: that land needed some sort of renewal for years.

Scenario 6: Viewing a Building

Mick's company has commissioned an architect to design a new building and, with other members of his team and the architect, Mick now stands on the place where it will be built.

'If you would all put your glasses on…' says the architect.

Half of them have them on already. When Mick puts his own glasses on and turns them on he sees a model of the building. It is full size and stands there in shared space, showing what the final building will look like.

It is certainly an interesting design! They walk around the edge and then wander into the hallway. This technology has been around a few years now and Mick knows that the building is simply computer graphics being placed in front of his eyes and superimposed on the future building site - the building exists only on a server - but he still finds it a bit disconcerting. His children are less worried by shared-augmented reality.

'Just look in any of the rooms to either side,' says the architect, 'The conference room is through there,' and she points to it.

'Is it all right to go upstairs?' someone asks.

There is a bit of laughter, but it is a poor joke. The stairs, like the building, are merely a ghost. Walking up them would be difficult.

'Well, we can in a way,' says the architect, 'Let me know when everyone has seen the ground floor and I will just slide the building down into the ground so that the next floor comes down to here.'

Scenario 7: Covent Garden

Susan stops in Covent Garden, London, for a cup of coffee at a pavement café. The place is famous for street performers and the usual collection of jugglers and mime artists are there.

Susan is surprised as a small dragon flies past her table. It is computer generated of course - she is wearing her glasses - and the artist who made it grins at her. The dragon flies through the shared space and lands at his feet.

It is quite good. Susan may even give the man a tip.

Scenario 8: The Protest

David is on his way to work His company has faced a lot of criticism lately: some people have been alleging that it not only profits from warfare, but actually encourages the instability that leads to it.

When he reaches his company's offices he is surprised. A protest is underway. Three-dimensional, virtual tanks stand in front of the building, pedestrians having to walk through them to get down the street or take their glasses off - which many would hate to do. It is not only tanks: anti-war slogans float in the air as well as huge money notes and dollar signs.

Some of the employees of David's company are standing on the pavement watching the scene. The director is there and he walks up to David.

'David, this is awful. I can't believe they would do this. What can we do?'

'Well, not a lot really,' says David.

'What do you mean? We need this junk removed right away!'

'That is going to be hard, right now,' says David, 'It is not really here! All of this is just on a server somewhere.'

'Well, who gave them the right to do this?'

'Nobody,' says David, 'They just put these things here and they exist on the server. There is nothing here really.'

'Well, if they put them here, surely we can delete them?' asks the director.

'No,' says David, 'We can make new objects, but we do not have rights over these. Only the creator has rights over them. The law is a bit behind on this. Hopefully, soon, there will be regulations about where you can leave things, or for how long they can stay there, but for now any space is up for grabs.

'That's awful! Can't we do anything?'

'We can hope for a change in the law soon, or we can hope that shared-augmented reality service providers will start to have some sense of social responsibility. In the meantime, well, two can play at this game - let's get our own people down here to put something in front of these objects.'

'That seems a bit feeble,' says the director.

'We could be in a much worse situation,' says David, 'Imagine if one of them had got into our lobby. He could pose as a legitimate visitor and then just download one of those huge dollar bills or a 3D tank and run out. We would be stuck with the stuff there.'

'That is ridiculous!' says the director, 'How could they have the right to do that to the inside of our building?'

'Because they wouldn't really be doing anything to the inside of our building,' says David, 'All they would do is stand in the building and make - or more probably take out of an inventory of previously made things - a 3D object. Their phone would send the details of the object and the GPS coordinates where it was dropped to a server where it would be stored. When other people came into the building and stood near those same coordinates their phones would get the details downloaded and they would 'see' the object as if it had been left there.'

'But for all practical purposes, if people make things inside our building they are altering the inside of it,' says the director.

David sighs. Only last week he has heard this man mention the 'irrelevance and fakeness' of shared-augmented reality.

'Well, that's how it stands at the moment,' David says, 'But they are discussing the idea of new laws to control this sort of thing.'

Scenario 9: The Store

Amanda is in her local park. She is wearing her glasses and has just walked through an area popular with amateur artists who leave three-dimensional sculptures in a constantly changing display and, on an open piece of land, she sees the outlines of a building ahead. This part of the park has been used - rather cheekily Amanda thinks - as a kind of unofficial marketplace for some months now and the building is a new store - visible only with her glasses - that has only appeared today. Amanda walks through the doorway - though nothing would have stopped her walking through the walls, of course - and into the store. In front of her are various pictures of clothes. Some are shown in three dimensions on simulated human figures. By each item of clothing a 'Buy' button floats in space, inviting her to send a credit transfer through her mobile phone.

Amanda has not made up her mind about this yet. Is it right for the shop to be here or not? The park is a public place and nobody owns the land, yet these people have appropriated it for their own use! At the same time though, she reflects, there isn't really a shop here.

Amanda knows that she could make the shop disappear in a second by simply setting her glasses to filter it out. In fact, because objects can be filtered, it would be possible for two or more shops made by different people to share the same space, with visitors deciding which shop to make 'real', but should it be her problem to filter things out in a public place?

At the same time, Amanda grudgingly admits, the experience of browsing in these places is different to the web. You see shops come and go in the same place and there is a certain unpredictability about it. You tend to meet other people while browsing and it can feel more sociable. It may be a bit healthier to actually walk while shopping too.

Shared-augmented reality is often used in real shops to complement displays, but how long will it be, Amanda wonders, before even some of those businesses start to launch shops here? Will this peaceful park become a crowded, virtual shopping mall, totally unseen to anyone without SAR glasses?

Scenario 10: Killer Robots!

In the same park, ten year old Peter is with his friends. They have guns connected via BlueTooth^™ technology to their mobile phones and they have used this to create a science fiction reality, overlaid onto the familiar reality of the park.

Peter has become popular since he learned to script robots and the children have had half an hour of chasing and shooting at the three-dimensional robots that hover just above the grass. Today he is showing off a new type of robot to his friends. A script can track the position of a mobile phone, provided that the mobile phone owner agrees, and this has enabled Peter to program his robots to chase people and shoot back. As well as robots, a few walls and other obstacles have been placed in the park to provide cover, but they also see the objects left by everyone else: a dozen metres away is a computer generated castle made by another group of children for their own game, though it is currently unused.

When the game is over they run to the gates of the park.

'Hey!' calls a man, pointing to one of the computer generated robots, 'Do you have to just leave this stuff littering the place up? Can't you tidy up after yourselves?'

'Oh, just filter it,' shouts back Peter, 'What's the big deal?'

Peter's mother wishes he would spend a bit less time on his computerised games. 'But at least you get some exercise,' she has admitted, 'When I was your age I just used to sit in my bedroom for hours with a games console.'

Scenario 11: Being There from Somewhere Else

Natasha is sitting at the personal computer in her home. She is writing a paper about social trends in shared-augmented reality. Some of her work involves examining the statistics of how the shared-augmented space changes over time.

Natasha could put her glasses on and wander around the world, but that could be a time consuming way to gather data. She finds it much easier to observe from a distance.

All of the data for the shared-augmented reality is stored on servers. Some of this data can only be accessed by actually going to the places in reality where it 'is' (or rather where its GPS coordinates, stored on the servers, say it is), but a lot of the data can be accessed remotely where people have had no reason to prevent this.

This is what Natasha is doing now. Her computer screen is showing the shared-augmented space for all of Britain. The images looks a little like a photograph taken from space at night: cities are brightly illuminated, as are roads. The ocean is almost totally black, while London appears to be almost melting in light. It looks like she is viewing the country's street lighting.

It is not lighting though: the computer is showing the distribution of objects in shared-augmented reality. These tend to cluster where people cluster, as there are more people to make objects in these places, so that the distribution of virtual objects seems to form an alternative virtual topography for the country.

Natasha zooms down to street level in London and individual objects start to resolve. Graffiti placed on streets passes before her eyes. Flying low over one region she sees a group of robots, probably created for a children's game, and a virtual shop. She knows, from visiting it in real life, that this region is a park, though, of course, she cannot see the ground, grass or trees: none of those exist in the shared-augmented reality so she would have to actually go there for that.

When exploring the shared-augmented reality in this way it can be confusing. It is designed to be overlaid onto reality itself, which provides a context, while to Natasha there is simply a black background, but the context often becomes apparent from the pattern of objects. In some places, however, it can be easier. Last week she 'visited' the shared space for a quarry, again while at her own home, and the rocks had been labelled and outlined to such an extent that there was quite a good schematic representation of it on her screen. This has also started to happen for some buildings. Some people, while bored in offices, have traced out the outlines of various parts of the structure, actually tracing crude copies of the buildings into the shared space in the same places as the originals, and when you fly over these areas you see a kind of crude schematic of the city streets.

Just because a building exists in the shared space this does not mean it exists in reality. The virtual building could have been built for a number of reasons: it could be as a historical recreation, to demonstrate a proposed building or simply for amusement. Natasha has already suggested a labelling system to allow objects to be designated as corresponding to real world objects or not.

Scenario 12: Paris

Simon is on holiday in Paris. He has not spoken French since school and hardly knows a word of it anymore - the signs above the shops in the city are meaningless to him. He puts on his glasses and activates them. He is pleased to see that translated versions of many of the signs hang in space near them.

How would it work?

All of this may seem very similar to 'holodeck' ideas. I would remind the reader that I am not proposing that we place holographic equipment or computers everywhere. It can work in pretty much the same way that Siemens^™'s idea would work.

The idea would use wearable computers. You would need a mobile phone, equipped with the relevant functionality, and you would need a headset. To be practical the headset would have to be quite unobtrusive and would really need to be very similar to a pair of glasses, containing screens to allow computer generated information to be superimposed on the user's field of view. More people are likely to be using equipment like this more often as wearable computing starts to get established.

The mobile phone and the glasses need to be in contact with each other. The mobile phone would contain a GPS receiver and the glasses would contain a device enabling them to determine their position relative to the mobile phone, although an alternative would be to build the GPS receiver into the glasses themselves: the main point is that the system must be able to determine where the glasses are. The glasses would also need to contain equipment to determine the orientation of the user's head - that is to say, in which direction he/she is looking. The connection between the glasses and the mobile phone may be via some sort of radio link similar to a Bluetooth^™ link, but it is also possible that the mobile phone could be built into the glasses to form a single unit. It is likely that that the mobile phone and glasses would perform some more conventional computing functions in addition to those discussed here, but this will not concern us in this article.

The glasses would be used to overlay virtual reality images onto a user's conventional view of reality. A computer would send different images to the left and right screens in the glasses to make the three-dimensional objects appear in front of the user: the user would see his/her normal view of reality and the virtual reality display at the same time. The GPS data and the orientation of the glasses could be used to adjust these images so that a three-dimensional object could be made to appear at a specific point in space, irrespective of the user's motion or head movements.

A way is needed of allowing the user to make objects or place them in the world. This could be done in a variety of ways. A small camera in the glasses could be used to watch the user's hand movements, allowing the user to use a hand as a kind of three-dimensional mouse to draw, manipulate objects and interact with menus. Some sort of stylus, designed to be easily observed by a camera or containing some sort of positioning system, could be moved around by the user. Voice control is a possibility for some purposes, as is selecting standard objects from a menu on the phone's screen.

All that is needed is some way for the user to specify the movement of a cursor and some editing functions, such as drawing lines or positioning objects, in the space in front of him/her. For the purposes of this description I will assume that a stylus is being used.

To make an object at a location the user takes out a stylus and uses it to draw lines in the air in front of him/her. The stylus is observed by a small camera in the glasses and is really nothing more than a simple optical target designed for easy detection by the camera. As well as drawing freehand lines the user can employ a number of editing functions; for example, straight lines can be made by selecting two points and then specifying a straight line between them and standard objects such as cubes, spheres and maybe even a three-dimensional equivalent of clip art can be selected from menus.

As the user moves the stylus to make the object the computer system in the glasses records his/her work. The glasses send appropriate images to his/her left and right eyes to produce a three-dimensional representation of his/her work. This means that if the user uses the stylus to draw a line in space then the glasses produce a stereoscopic image of the line in front of him/her and if the user moves then the graphics will be adjusted accordingly. All of this is intended to give the user the idea that his/her stylus is a device that can actually make physical changes and draw in the air.

As the user makes objects, or changes existing objects, the data is sent to the mobile phone and a description of each of the pieces of an object, together with its location in space (which is obtained by using the phone's own GPS coordinates and determining where the object is in space relative to the user's equipment) is transmitted over the mobile phone network to a centralised server. If part of an object is changed or deleted then these changes, together with their position in space, are also transmitted to the central server.

The central server stores details on all the editing that has been performed by users, together with the positions in space at which this editing was done. As an example, if you draw a line in space then details of this line, together with its coordinates, are stored on the server or, if you delete an object, then the server updates its records accordingly by deleting its record of that object. The same server can be used to store the data for a large number of users, irrespective of where they are, and it is this that will allow the overlaid reality to be a shared one.

When a user walks around the GPS receiver in his/her system constantly obtains its coordinates and these are transmitted to the central server. The server examines its database of all the editing that users have done - the data we have just been discussing - to try to find parts of objects which have coordinates that should be within visible range of the user. When details of these are found they are transmitted to the user's mobile phone and the user's virtual reality glasses can then make the stereoscopic images needed to make these objects appear real.

What this means is that if I go into a room, or a field, and make a cube, all the details of that cube, and its position, are sent to a server. When someone else comes onto the scene later his/her mobile phone will send his/her position to the server, which will determine that the user is now in visual range of the cube that I 'left' and will send details about this cube to the mobile phone.

Such a system can be extremely powerful and could create a complex, virtual reality which is overlaid on our existing reality, so that users can interact with both at the same time. Users need not be restricted to 'leaving' small objects: there is in principle no reason why entire buildings, or even cities, could not be constructed, just as is done by the inhabitants of SecondLife^™.

Importance of User Interaction

I have mentioned the importance of the idea that users are provided with a planet spanning reality (when enough countries have a system like this) in which they can build. It would be difficult to overstate the importance of this. It will allow humans to build communities which do all kinds of strange things and will increase the interaction between them. One of the things that most interests people is other people and a technology like this that uses virtual reality to augment reality benefits from all the richness of the relationships, connections and events within it, rather than trying to take these into an artificial world - not that I have any problem with the conventional sort of virtual reality: it has its place, as does shared-augmented reality.

The idea of users being able to build things is already established in SecondLife^™. An important benefit is that it causes the simulated reality to become complex and interesting on a large scale, very quickly, due to users doing all kinds of complicated things within it. This amount of content can be massive and can continuously change as users have different ideas and do different things. One big advantage is that the costs of running the system are reduced as users actually generate all the content: it would not be practical to do this on any large scale if a company had to generate it all.

Streamed Data

The simulated environment created by this sort of system would be dynamic: it would change continuously in real-time because users would constantly make changes to it by adding new objects and changing existing ones. This would prevent things from becoming stale, as you would constantly be seeing new things in the shared-augmented reality. This constant renewal of content is performed by users, so that users serve as a resource to keep other users interested.

This dynamic nature of the system suggests streaming as a way of delivering data. If the data for a large amount of the shared-augmented reality was stored in the user's phone it could become less valuable with time: by the time the user got round to visiting a lot of it, things could have been changed by users. Streaming would address this issue by having the data for a region sent from a server as it is needed.

SecondLife^™ uses such a streamed solution to deal with the dynamic nature of its environment and shows that it can work. Rather than all the data being stored on the user's computer, it is sent to it when needed as the user moves around the world.

Clearly, there is a role for some sort of caching here. SecondLife^™ clearly does some sort of caching. As an example, if you turned your back on a virtual building and then turned round so that it came back into view you would hardly want to have to wait while it loaded again. Similarly, if you walked away from an area and then returned immediately you would not expect all of its data to have to be reloaded again, but with a 'streaming' approach the emphasis is different to that of non-streamed approaches.

For the sort of streaming that we need for this sort of system the following ideas are important:

• Data can be cached, but cached data decreases in value over time as the reality-overlay changes to make it obsolete.
• The available storage capacity should be used to store data, despite the fact that this data's value is compromised by the dynamic nature of the reality. Some data which reduces the need for transmission from a server is better than nothing.
• The dynamic nature of the reality means that not all data can be usefully cached. The very large size of the virtual reality means that it would be impractical to continuously transmit details of all changes in the virtual reality to each user.
• As it is not feasible to cache a complete description of the world some sort of streaming is required in which data is transmitted to the user when it is required or when there is good reason for presuming that it is about to be required.
• When determining what data, out of all the available data, should be cached consideration should be given to the probability that the data will be required in the near future: the best data to cache is data that is likely to be needed soon.

The decision about what data should be cached can be made in a number of ways. The simplest way to do it would probably be to cache data on all objects within a certain distance, or range, of the user. As the user moved around objects that went out of range would be deleted from the cache and objects that came within this distance of the user would be stored in the cache. All of this updating of objects in the cache would be occurring on the edge of an imaginary sphere, at the centre of which is the user: objects are added as they pass through the surface of the sphere and enter it and they are deleted when they pass through the surface of the sphere and leave it. The sort of region used does not need to be a sphere: for example, data could be cached on all objects in a cube, at the centre of which is the user.

It may be that the available memory allows storage of more objects than are in the immediate vicinity of the user. In this case objects in regions that the user has recently passed through could be kept in the cache in case the user retraces his/her route. Another way of using more available memory would be to extend the size of the region around the user in which objects are cached to ensure that all the memory capacity is used.

If the user is standing in one place with all of the nearby objects cached then it could be desirable that, rather than doing nothing at all and simply waiting for the user to move before starting further downloading, the system attempts a sort of pre-emptive caching by downloading the objects that the user is most likely to encounter soon. A lot of different criteria are available for determining which objects are most likely to be encountered and these could be used to determine the value of caching the object, so that any storage space used for cached objects or bandwidth used for downloading objects to cache is allocated by preference to high value objects. Some of the criteria for assessing the value of an object could be as follows:

• How close is the object to the user? - downloading objects near to the user, followed by those further away, as has already been mentioned. This is based on the simple idea that the space near the user is most likely to be explored next.
• How often does the user visit the region in which the objects is located? - If statistical analysis of the user's behaviour shows that he/she often visits certain places then, if the system has nothing better to do for the user, it could be sending him/her object data for some of these places in case he/she visits them in the near future.
• Is the object in a popular place? - If the user is near a very popular place, even if he/she has never visited it before, it may make sense to download it if there are no more urgent demands on the system. This is based on the idea that if a lot of people visit a place then there is a good chance that the user is going to be one of these people.
• Is the object in a place which a statistical analysis of the user suggests he/she is likely to visit? - The user's behaviour could be analysed by software and his/her behaviour compared to that of other users. If some place, near to the user, is frequently visited by people who fit the user's sort of behavioural pattern then objects from that place could be downloaded and cached in anticipation of a possible visit by the user.
• Are any of the user's friends near the object? - People often go to places where their friends are. It would be possible to determine which people the user is near most often and regard these people as the user's 'friends' or to obtain this information from a contact list. It would be possible to determine how often the user tends to be in the presence of each friend. If one of the user's friends is nearby then it may make sense to start downloading data from the region around the friend, and from any space in between the user and the friend, in anticipation of the user going to meet his/her friend. The value of caching objects from near a user's friend could be viewed as greater for friends whom the user is frequently near and this could be used to determine whether or not to do the caching, or how much of it to do.
• Does the user's route suggest that he/she may encounter the object soon? - The user's route could be analysed and objects in regions which the user seems to be moving towards could be presumed to be more likely to be encountered soon by the user, increasing the value of downloading and caching them. This is compatible with other criteria; for example, a friend towards whom the user is moving could be viewed as more likely to be encountered by the user.
• How old is the object? - If an object was created a long time ago it is unlikely that it is going to be removed in the immediate future, and so caching it will be of higher value. Some readers may doubt this, but it is valid. A shared-augmented reality building that has stood there for five years, unchanged, for example, is more likely to be there tomorrow than a cube which was dropped there two minutes ago.
• Was the object placed by a user who makes changes infrequently? - If users who do not often make changes in shared-augmented reality place objects then there is reason to think that those objects are less likely to change quickly than objects belonging to people who make change continuously, so caching such objects will be of higher value.

Caching would occur a lot for short term storage of objects, but longer term caching may have some uses for objects that are particularly valuable; for example objects in places visited particularly often by the user or relatively long-lasting objects.

Transmitting data to be cached will consume bandwidth and take time, but an adaptation of Bluetooth^™ technology, or whatever equivalent fulfils the same purpose when a system like this is made, could help with this. Bluetooth^™ allows devices to communicate with each other by means of a short-range radio link and this does not place demands on the bandwidth of a mobile phone network. It can provide us with an alternative way of transmitting information. This is how it could work:

Each user's mobile phone will contain a cache of objects which he/she is likely to be near later. When two users are physically near each other, their mobile phones, each with its own cache, are also physically near. This means that the mobile phones can exchange data on any objects that appear to have a value to both users, over a Bluetooth^™ link without consuming bandwidth on the mobile phone network. This can be particularly useful in these situations:

• If the user meets someone who had recently visited a place that the user visits often then up to date information on objects in that place could be sent to the user's phone using a Bluetooth^™ link.
• If the user meets a friend then information about objects in places that the friend has visited recently could be sent to the user's phone using a Bluetooth^™ link on the basis that the user may soon visit them, possibly on the recommendation of his/her friend.
• If the user meets someone who has just come from a place that the user seems to be moving towards (from analysis of the route that he/she has been following), then information about the objects in that place could be transferred to the user's phone, from this other person's phone, using a Bluetooth^™ link. This would be particularly useful in crowded cities, where there is going to be a lot of demand for bandwidth anyway, and could help reduce it. While walking down a street you would often walk past someone walking the other way, who has just been at the other end of the street, where you are now going. The two mobile phones could communicate and, when your phone has determined this it could request a rapid download of the objects at the other end of the street from the other person's phone.

None of this would require users to cooperate consciously. It could be made automatic, so that phones assist each other by providing objects to cache. There could be a privacy issue if this were done - some people may not want their mobile phone to tell people where they have just been - but answers to this are as follows:

• The user of the other phone is not actually told where you have just been. His/her mobile phone simply has its cache updated with objects that you have been near. He/she may not even notice that it has happened.
• The process can be anonymous. When the caching occurs there is no need for other people's phones to store information about who sent objects to them.
• The process could be turned on or off by the user. The user could choose not to have his/her mobile phone work with other people's phones in this way.

Some changes may be required to the way that the Bluetooth^™ system works to implement these ideas.

Won't reality get cluttered up?

At this point, some readers may be thinking that the shared-augmented reality is going to get cluttered up with objects left all over the place. The current web shows how this can happen. Some things are created which are of dubious use in the first place, while others may have been useful at one time, but linger long after their purpose has been served. The idea of putting on a pair of virtual reality glasses in a street in June and seeing all the animated, virtual Christmas decorations that have been put up for the last fifty years may seem horrific, but there are ways of dealing with this.

For a start, remember that it is not the real planet that is being 'polluted' here, but an artificial virtual reality that is made to appear to be laid over the top of it. If you have started to think of this in terms of 'ruining the planet' then I should be quite gratified that you are viewing the shared-augmented reality as real enough to worry about. Many users would view the overlaid reality in this way once such a system was established. Secondly, if you think that implementation of a system like this would lead to massive information pollution then would it not be fair to say that you are admitting that the idea would be a runaway success? The problem with most ideas is getting people to use them at all. If we were faced with the problem of such massive use then that should be the least of our worries!

Despite this, the problem does need to be addressed if the system is not going to become a victim of its own success. A range of measures could be used and some of these could be as follows:

• Allowing users to delete objects created by other users, which means users can interfere with each other. Any user who dislikes the amount of clutter in an area can simply set about deleting other users' work. I do not like this idea too much for general use as it provides too much potential for a minority of users to make life miserable for everyone else, to interfere with any serious uses of the system and to harass individuals. It could have a place in certain 'zones'. A patch of land in a local park could be designated a 'free for all' area of this type where people are encouraged to go to extremes but are warned not to get too possessive about their creations. This does not mean that such creations need to be lost: users could take copies of their work. SecondLife^™ has a facility to do this, which allows a user to copy his/her 'in-world' objects into an inventory for later use.
• Having some areas in which objects have a limited lifetime, so that they expire after a certain amount of time and get deleted, though a user could always put the objects in storage so that he/she has a copy. This would provide a self-tidying world, but there will be places where we want more permanency: people would get attached to things that they make.
• Having areas which are regularly cleared of virtual objects. SecondLife^™ does this by providing some areas which are cleared of users' objects at the same time each day. Again, this would be useful in some places but would not be a total solution as users would demand more permanency.
• Having areas in which the degree of permanency of objects is controlled by some sort of voting system. As an example, an art area could be set up in a public park in which users are encouraged to create works of art and to vote for those that they think merit more permanency. This idea of using a democratic process to control the permanency of objects in public spaces was described by Arthur C. Clarke, although not in a virtual reality context, in The City and The Stars [4].
• Having some areas designated as belonging to an individual or group so that those individuals can control what goes on in them while nobody else is able to add objects or make changes. This happens in SecondLife^™ where users can own virtual land over which they have editing rights. This would be a little more complex in my sort of system as any land in it corresponds to land in the real world. How we allocate land is a serious issue: clearly we do not want someone able to requisition the local park for his/her exclusive use, but anyone who actually owns the real world land, or has control over it, should probably have control of the virtual reality laid on top of it. In SecondLife^™ users can buy land and it may be that virtual land could be sold in a system like this - subject to some condition that users have to be able to demonstrate that they own the real-world land. As an example, I may decide that I want exclusive control of the virtual reality overlay in my house, so I may make an application to buy it, similar to the way people 'stake a claim' now when they buy web domain names, and at the same time demonstrate proof that I have the real world land, or maybe simply that I am able to receive mail that is sent to its real-world address. It may be that I should only have to demonstrate that I own the real-world land if a dispute arises. Land which is operated in this way would certainly give more permanency to the objects on it: the person who controls the land should be able to decide for him/herself when things need to be deleted.
• Having some areas designated as 'no build areas' so that no objects can be placed in them by the general public at all. Clearly, I would not want too many of these as it would defeat the whole purpose of the idea.
• Limiting the number of objects that an individual user can have in the shared-augmented reality at any time. This may need to be done irrespective of the 'cluttering' issue. All of these objects need to be stored on a remote server somewhere and there will not be an infinite amount of storage capacity, nor will there be infinite bandwidth to deliver objects to users. One solution may be for each user to have a 'shared-augmented reality account' with a service provider who provides the storage space for the user's objects and limits the amount of memory that is available for this purpose. Users could be allowed to buy more storage capacity, though some may come with the basic account in the way that hosting of web sites is provided with by some internet service providers (ISPs) with internet access accounts. It should be noted that there is no reason why such an 'object hosting account' should have any sort of spatial restrictions on where objects can be placed: this is, in fact, undesirable. It is preferable that the hosting account simply provides the user with memory that he/she can use to place any objects anywhere throughout the shared-augmented reality, subject to any other restrictions that there may be on the placement of objects in the shared-augmented reality, of course.

The sort of people who send out spam emails would love to get their hands on physical reality, so all of this is a major issue.

The idea of 'filtering' would also almost certainly be used. Just because all these objects are out there in space, this does not mean that a user has to see all of them. Users could specify which objects they wish to see, or specify objects that they do not want to see. For example:

• Users could choose to see only their own objects.
• Users could choose to see only objects created by specific users or groups of users.
• Users could choose not to see objects created by specific users or groups of users; for example if they see that a place is saturated with useless objects created by a particular user then they could choose not to see them.
• Users could choose to see only objects of a certain type. This would require a system in which each object was given a classification of some kind.
• Users could subscribe to 'anti-spamming' services which cause them not to see objects which appear to be useless to them or about which other users have registered complaints.

This issue of 'cluttering up' would probably merit a long article in itself: what I have said about it here is only basic. These measures would need examining in greater depth and there will be many more solutions, many of which would probably end up being used somewhere in the world.

Safety

The pervasive nature of this idea introduces some safety issues. This is mainly because it could have an impact on almost any aspect of human life - and many aspects of human life allow people to harm themselves or others.

Some dangers which the system could pose are as follows:

• Distraction while performing dangerous tasks - The virtual reality graphics could distract people while they are doing other things. An obvious example of this is driving and it is clear that some restrictions, whether they control where content can be placed or when people can use an augmented reality system, would need to be in place. The system could have some limited role in driving, however: it could display extra road signs, navigational information, warnings, etc. Navigational information could appear in three dimensions in the driver's field of view; for example a floating arrow over a roundabout exit could tell a driver to take it. Alternatively, it could simply be made illegal to use this sort of system while driving. People already have lots of devices, such as mobile phones, personal CD players, personal DVD players, portable games consoles, etc which can distract the user if used inappropriately. This is no different.
• Hiding dangers in reality - Some dangerous feature of reality could be concealed, deliberately or accidentally, by means of overlaid objects. As a simple example, in principle, a virtual manhole cover could be placed over a real, open, manhole to make it look like there was a cover there. Forgetful users who tried to walk up virtual stairs or lean against virtual walls would also get a surprise. Education of users would be important to deal with this and some restrictions on content may be necessary. The problem would not be too serious in the early stages, because the graphics would not be good enough to pass as real. Early systems would probably use 'line' graphics like those seen in old computer games and it would be some time before a photo-realistic virtual manhole cover becomes a possibility. This does not mean that no consideration of the issue is required. A primitively rendered object - for example a simple, luminous circle, over a manhole cover - could still deceive a user enough to make him/her presume that a solid surface is underneath it. User education would be an issue in this area and extensive testing would be required to ensure that the system provides users with some capability to distinguish between real and virtual objects while still making the virtual reality of an acceptable standard.
• Manipulating users - Impressive objects which provide a lot of fun could be created to attract vulnerable users to places where they may be in danger. Education would be important here as well.

Legal Issues

In one of the scenarios that I gave we explored the idea of the system being used to cause harassment. The existing internet can be used to cause harassment and I do not intend to discuss, in this document, the sorts of legal problems that would be caused in entirely conventional ways. Some new legal issues do exist though.

As an example, there is the issue of using someone's own home/business to harass them: if somebody could temporarily get access to your home or business and placed objects inside it, without your permission, would this (ignoring the issue of how physical access was achieved) constitute harassment? I have already discussed the issue of giving users control over certain areas and it relates to this too. The whole issue of harassment when objects are not really being placed in someone's private space, but for all practical purposes may as well be there, could give lawyers something interesting to think about. The existing libel and slander laws, I presume, should still apply to situations like this, but the whole issue of legal control of space and whether or not ownership of a property should legally give someone control of the overlaid virtual reality inside it is more complex.

People who do not think this is an issue could say that the fact that nothing is really being placed in the property makes it a meaningless concern. They could point out that it is only a consensus about how to interpret the stored data that could make users see an object in your personal space: if I stored an object on a server with certain GPS coordinates, then the object would only seem to you to be in a certain place if you used a system which interpreted GPS coordinates in a certain way.

I have doubts about this. Regardless of what is really there, seeing an object which appears to be placed in your own company's building or home, put there by someone who managed to get access, for the purpose of causing a nuisance, would feel like harassment to many people. The 'consensus' argument does not impress me: language is based on consensus. If I slandered someone I could try to argue that my words had no import and that it is only if someone else happened to adopt the same conventions for the meaning of words that any slander would occur. Such an argument would not impress a court: the conventions for how the English language is interpreted are sufficiently well established for them to be assumed.

I think we can argue that sufficiently well established conventions for how stored data relates to the positions of various virtual objects in space would give us reason to assume those conventions as real relationships between stored data and objects in space in the same way.

Users could inconvenience each other without virtual reality 'trespassing'. As an example, if users could put objects in public spaces then one user could leave objects intended to cause upset in the street outside the home of another user or, as in the scenario I gave, a protest could be mounted against a business by placing objects in proximity to it on public property.

We can be sure that a shared-augmented reality would create lots of interesting situations for lawyers.

Would there be a separate shared-augmented reality for each service provider?

In short: no, but let us examine this in more detail.

Throughout this article I have described the data storage as being on a 'centralised server'. I have mainly used such terminology to make it clear that no local hardware is required to store the overlaid reality or deliver it to users - other than the hardware required to communicate with the server. In reality, there would probably not be a single centralised server, but a number of servers. Some of these could be owned by separate companies in the same way that internet servers are owned by different internet service providers (ISPs). Would this not mean that there may be many different shared-augmented realities, rather than just one?

This would be undesirable. The whole basis of the concept is that a shared-augmented reality is shared. Clearly, if different users are in different realities, then the degree to which sharing occurs is compromised.

Fortunately, this need not be a problem. There is no reason why all the servers, owned by all the service providers, cannot contribute objects to a single shared-augmented reality experienced by all users, irrespective of which service provider each uses to access the system.

The best analogue for this is the current way in which websites hosted by a large number of service providers are all accessible on the internet irrespective of which service provider you use to connect to the network. When you visit a website you do not even need to know which service provider happens to be hosting it. Similarly, for the shared-augmented reality concept, it could be possible to stand in a street and see a variety of objects which are hosted on a large number of different servers by a large number of different service providers and not know where each object is hosted or who is providing the hosting.

For this to work, standards would be needed, in the same way that we have standards for the existing internet, so that the content hosted by all these different service providers could be understood by the equipment used by people who are connecting using different service providers: there needs to be a consensus on how the data describing objects is represented.

It is only a matter of time before technology of the sort that I am proposing becomes a reality and I suggest that it is in the interests of telecommunications companies, and other companies, to start attempting to influence the eventual choice of standard that is adopted.

Positional Accuracy

This idea relies on it being possible to determine the location of the glasses that produce the three-dimensional images with extreme accuracy. The current commercially available GPS system is not quite good enough, though the military system is better.

There are methods for increasing accuracy [17], such as differential GPS, which uses an extra ground station, and relative kinematic positioning and it should be possible, with reasonable progress, to get the error down to a few centimeters. GPS systems which will be available later could do even better.

It may be the case that the GPS receiver needs to be assisted by other methods to give adequate accuracy. Movement can be monitored by accelerometers and these could be used to determine position. Unlike a GPS receiver, they would need calibrating occasionally, but they could detect very small movements.

There is a range of possibilities. I shall not consider all of them in great detail here, but I shall try to give an idea.

One approach would be to use GPS with accelerometers to detect small movements, but to occasionally calibrate the system - it would be easy to check this by seeing if virtual objects that are supposed to be positioned in particular ways relative to real objects are actually positioned correctly. Users could probably manage an occasional manual recalibration without much trouble.

Another approach would be to use GPS and accelerometers, but to use some kind of map of the environment to calibrate the system. By examining the route that a user has taken, together with knowledge of how quickly the position obtained from the accelerometers becomes inaccurate, it should be possible to rule out certain positions, determine that some are unlikely and decide that some are more likely. This is because any particular position of the user, together with the record of his/her route, gives an idea of where he/she has been or, rather, a kind of fuzzy

idea of where he/she has been as there will be gradual degradation of accuracy.

As an example, if the user has changed his/her height at some point then routes that pass over steps at that point may be more likely to have been the correct one and this means that assumptions about the final position which suggest that the user ascended steps at a certain point are more likely to be correct. If the GPS data shows a user to be somewhere inside a room at a certain point then it can be presumed that he/she is more likely to have come through the door than the wall and this could be used to classify positions as unlikely that would suggest he/she came through a wall. Some readers may be sceptical of this, thinking that there could still be a large number of possible final positions that would all indicate that the user came through the same door, but when the route is examined in large numbers of places so that lots of examples of it being constrained are available, then a considerable improvement in accuracy could be achieved.

As an analogy, imagine being blindfolded and led around an area that you know. From an approximate knowledge of the route that you seemed to have followed and some knowledge of the surrounding area you could get some idea of where you are. A computer should be able to do this, using statistical methods, with greater accuracy.

For this to work some 'map' information would be needed about what routes are viable; for example storing some data about the height of land, where doors may be, etc, and this map information could be stored as part of the shared-augmented reality. Some of this could actually be created by users who want to increase the local accuracy of the system.

Some readers may object to this, by asking how, if a user decides to be helpful by marking the outline of a likely entrance into a building in the shared-augmented reality, he/she would know its actual location with sufficient accuracy, given that he/she is actually doing this to improve accuracy. The idea may seem circular. This is not really a problem, however. The absolute accuracy of coordinates in the databases of objects is less important then ensuring that they are placed accurately each time they are shown to a user: consistency in deciding where users are is more important than knowing where users are with real accuracy.

A further way of getting extra help with positional accuracy could be by means of devices that observe local reality in an attempt to get an idea of the user's position, or at least to align the virtual reality accurately with reality. The use of cameras and computer processing to obtain a stereoscopic model could help to identify the user's location, subject to the constraint that it would only work with relatively fixed features of the environment.

Some way like this of augmenting a satellite based GPS system would probably be necessary for another reason: satellite GPS only works outdoors. This problem is not insurmountable. Accelerometers in the device could, for example, track its position while you are indoors, re-calibrating themselves by using the satellite based GPS system when you go outdoors. A similar sort of approach is used with some existing satellite navigation systems so that they can show your position while you are driving through a tunnel.

What happens when real-world objects get in front of virtual objects?

Virtual reality objects are shown to the user at the same time that he/she is looking at real objects and this could cause some complications. If the virtual reality system shows an object to the user then he/she will see it - irrespective of whether or not a real object is in the way. Fortunately, there are ways of minimising this problem.

Problems caused by this could be minimised by only displaying objects that are relatively nearby in some situations. In addition, there could be a facility for users to mark the outlines of permanent objects, the data being stored with the data for conventional shared-augmented reality objects, so that they can be taken into account and a virtual object not shown if a permanent object is in front of it. This could be time consuming and I think that it would only be done in real world places that users wished to use frequently or near virtual objects that users intended to be relatively permanent.

A better solution could be provided later by improved technology if the virtual reality glasses are given their own capability to examine the real-world scenery in front of them, as was mentioned previously, crudely determine the positions of real-world objects, and use this information to determine which virtual objects are obscured by real-world objects and should not be shown. This could be done by adapting various sorts of technology already used in auto-focus cameras or by using a computer program to analyse the images acquired by a pair of cameras to produce a stereoscopic model of what is in the user's field of view in a way similar to that in which we produce a stereoscopic model in our brains from the left and right eye images. At the time of writing, analysing images in this way to produce three-dimensional models is a difficult process for computers and requires substantial processing resources, but it should soon be practical to do it in real-time.

There will be situations in which it is not desired that real-world objects obscure virtual ones. An example would be a virtual reality overlay intended to give the illusion that the user is looking inside a real-world object.

Accessing the Data from a Distance

In one of the scenarios that I gave I described how someone may sit at home browsing the database of objects in three dimensions on a home computer. This is given merely as an example of what could be done. The whole point of the system is for people to be at the physical place with which the data is associated, interacting with reality and with the virtual reality produced from the data, but there is no reason why the data could not be accessed remotely, so that you could see the virtual reality overlay for a place without actually going to it.

This could be interesting. As more data is added to reality, a kind of virtual 'skeleton' may appear around much of reality and this could give an insight into what is actually at a place if the data is examined remotely. The data could be examined by showing it on a computer screen, as with my example, or by using a pair of virtual reality glasses, just as the people do who are actually at the place.

It may not be desirable for all data to be accessible in this way. Psychologically, it could weaken the idea of the data being a 'real' part of physical reality. Furthermore, some data will be in private spaces: people may not want virtual reality objects in their homes to be viewable by someone 'surfing' the shared-augmented reality from his/her own home.

It would be possible to make some regions accessible remotely while others are not and it would also be possible to make some types of objects accessible by remote access while others are only seen by people who actually visit the location.

It could be hard to enforce this, however, as there would seem to be the possibility of someone feeding false GPS data into a mobile phone to make his/her phone obtain an incorrect position. Because of weaknesses like this it may be that there should be an option to 'lock' any really private objects so that they are only visible to authorised people.

Another issue is that of when users should be able to make changes to the shared-augmented reality from a distance, rather than having to physically visit a place to leave or change objects. Editing from a distance may be convenient, but if it happened too often it could compromise the feeling that the virtual reality was part of reality.

This idea of remote access may seem to some readers to give a feeling close to that of the cyberspace concept in William Gibson's science fiction novel Neuromancer [18] and later cyberpunk science fiction. It is in one way: objects in a simulated space all over the world (ultimately) would be accessible to a user and they would exist in a kind of space. If the capability to link objects to other internet systems is provided in the scripting language (as is already done in SecondLife^™ - we will discuss its scripting language shortly) it would also mean that a three-dimensional reality would exist that had contact in places with the conventional internet. There is one important way, however, in which this concept does differ from that of cyberspace. In Gibson's novel cyberspace was an internal reality, separate from the outside world and, to me, there never seemed to be a good reason for any particular topography in it: it was never really clear what should dictate the positions of objects relative to each other in Gibson's novel. One way could be by using the real-world positions of the computers supporting the objects, but if we are in cyberspace and not in the outside reality why should this be so important to us? In the system I propose the preferred topology is very clear. The sort of 'cyberspace' we are constructing here is intended to be placed in reality, rather than being a separate, virtual reality and objects in this sort of cyberspace have obvious positions based on real-world location.

Other Ways of Interacting with the Data

In the scenarios that I discussed I concentrated on the use of virtual reality glasses to view the shared-augmented reality, while briefly mentioning the use of a conventional home computer to access the data, but it could be accessed in other means. For example:

• A video camera could be equipped with the facility to access shared augmented-reality and transfer this data onto the recording so that the shared-augmented reality could be video recorded as if it were real.
• A handheld device, equipped with a camera, display screen and access to the shared-augmented reality system, could be used to view it.
• A video camera could be placed in a room and used to show, on a large display screen, the scene in that room, but with the virtual reality overlay that happens to be occupying it, so that anyone looking at the screen would see what the room looked like to people using virtual reality glasses. This could be a feature of night clubs, for example, and it could also show people who are not yet in shared-augmented reality what they are missing.
• While, in this article, I have tried to avoid ideas relating to placing advanced machinery everywhere, when holographic technology becomes sufficiently cheap there could be a place for it in this system. Holographic equipment could be built into the walls of some buildings, for example, and used to show the virtual reality overlay in a room inside, so that users there could experience it without the need for glasses. This would not be necessary but could be useful in some situations. I would also note that holograms would be limited in ways not generally shown in science fiction. In particular, holograms are not 'projected' into space in the way that science fiction often portrays them and are instead the result of precisely controlling the directions in which light is emitted from a surface. A hologram could only appear when there is an unobstructed line of sight between the observer and the equipment.

Other Ways of Creating Objects

So far in this article I have discussed the idea of creating objects in the context of users using their mobile phone, virtual reality glasses and whatever other input devices they carry around with them (or none) to create objects. Another possibility, however, is for the user to create the objects at home. This could give the user access to less portable input devices and could allow the use of design software running on the user's desktop computer.

When creating objects at home the user could still design them within the shared-augmented reality system itself, merely working from home to benefit from better input devices or he/she could work offline, producing the object on a home computer which could allow the use of sophisticated design software chosen by the user.

When the user has made an object it could be stored in an inventory associated with the user's shared-augmented reality account and the user could then take it out of inventory and place it in the shared-augmented reality at some future time when he/she is away from home.

Scripting

It is not necessary for objects to be passive. In SecondLife^™ users are provided with a scripting language Linden Scripting Language [19] that allows them to write computer programs which can be placed in objects that they create. This allows objects to behave and interact with users in complex ways: examples are provided by the animated flying dragon in one of the scenarios that I gave and by the robots in the children's game in another scenario.

Not all users would be willing to write scripts, but SecondLife^™ demonstrates that many people would welcome the chance to learn and use a scripting language. SecondLife^™ is full of scripted doors and windows, lighting effects, animations, weapons, quiz machines, gambling machines and many other examples. SecondLife^™ even contains objects that wander around by themselves.

Users who did not wish to make their own scripts could always obtain standard scripts made for general purposes and put them into their own objects.

The provision of user scripting is important because it has the potential to make the shared-augmented reality a more complicated place - and one that seems more real.

There could be significant advantages for any company that manages to be first and establish the language standard for scripting in shared-augmented reality. Linden Scripting Language may not be too far from what is needed as it was written to be easy to learn and to control objects in a three-dimensional environment.

Would scripts run on the client or the server?

Scripts could run on either the server (the computer which is hosting the shared-augmented reality object containing the script) or on the client (the user's mobile phone).

Most scripts would be used to control the behaviour of objects. The behaviour of objects is seen by many people and should be consistent for all observers. As an example, if a script makes a robot go left then everyone should see the robot go left at the same time.

For this reason, scripts would tend to run on the server. There could be a place for scripts which run client-side; for example, scripts that control the experience of shared-augmented reality on an individual basis for each user or that handle one side of an encrypted session between a shared augmented-reality object and a user.

Some readers may find it strange that I am suggesting server-side scripting when the existing internet uses a lot of client-side scripting (for example, Java and JavaScript), but the reason for the large amount of client-side scripting on the current internet is that web pages are usually not as shared as the shared-augmented reality would be: a script can be used to affect an individual user's experience of a webpage and it does not matter if his/her experience is slightly different to that of other users. Server-side scripting is already established on the existing internet (for example, ASP and PERL) and its use for this purpose is appropriate.

Scripting and Security

Providing scripting to users raises security issues. The main issues would be:

• Could a script attack a server? If scripts run on servers then would it be possible for a user to write a script which hacks a server somehow; for example, by erasing the objects of other users or accessing other users' account details?
• Could a script attack the user's mobile phone? If a script runs on the user's mobile phone, that is to say client-side, could it attack the user's mobile phone in some way?
• Could something analogous to viruses be created in which objects in shared-augmented reality can be made to self-replicate uncontrollably?

Regarding the first issue, a script could attack a server if the scripting language allowed it. The solution is to ensure that the scripting language does not contain the sort of commands, or allow the sort of access privileges, needed to allow an attack on the server. As an example, writing to the contents of all the memory locations on the hosting computer is not going to be allowed: users do not need this much power. In fact, the language would be very high-level and it is doubtful that there would even be a command to write to any absolutely specified memory location. Scripts need only run in a 'closed-off' area of the computer where they can only access data relating to the user who owns them and may even only be directly able to access data relating to the object that contains them.

Providing users with scripting abilities is not new. PERL and ASP allow web designers to place scripts in their web sites. These scripts are often hosted on a server owned by an internet service provider and the issue of scripts being used to do damage is adequately controlled.

Regarding the threat posed by scripts to a user's mobile phone, most scripts are expected to be run server-side, but in the event of any client-side scripts being used, the same precautions that have just been stated are relevant; that is to say, limiting the scope of the scripting language to preclude hostile scripts from being made.

The issue of making objects that can replicate uncontrollably is an interesting threat. This is already relevant in SecondLife^™, in which the scripting language allows a script in an object to make other objects appear. This means that a script in an object could say, essentially 'Make copies of this object' and the new objects would contain the same script, with the same replication directive, so that uncontrolled replication of objects would occur. I have not checked on all the details of what happens in SecondLife^TM to prevent this, but one solution is clearly policing by the administrators of SecondLife^™. Other possible solutions to this problem that could be used in shared-augmented reality are:

• Not allowing scripts in objects to create objects - I would prefer not to resort to this as allowing automatic object creation provides a lot of capability for interesting things.
• Not allowing scripts to create objects in public spaces - like the above option, but allowing scripts to create objects on land that the user controls and only imposing the restriction in public places.
• Having an upper limit on the number of objects in the world at any one time created by one user - As I mentioned earlier, hosting by service providers is likely to enforce a limit like this anyway and it would cause a virus to 'run out of steam' when it came up against a user's object limit.
• Limit the rate at which scripts can create objects - allowing object creation by scripts, but making it a slow process, so that the population growth for a virus would be slow, buying time for detection and some sort of response.
• Limit the rate at which scripts can create objects in public areas- like the above restriction, but only imposing it in public places and not on land that a user controls.
• Limiting the density of objects by one user in a public space - allowing users to do what they want on land that they control, but limiting the number of objects they can have in a small space on public land.
• Limiting the 'generation' of objects - Build a kind of aging process into objects. Give each object a counter when it is made by a script, initially set to some value. If a script in that object makes an object then the new object inherits the counter of the object that made it, with its value reduced by one. When an object has been made by a script and has a counter of zero it script is not permitted to create new objects. This idea comes from biology: some scientists speculate that a system like this in human cells may help to prevent cancer while also causing mortality. Such a solution would not be total - it would not stop growth - but it would prevent it from being exponential indefinitely.

But would people want to wear virtual reality glasses all the time?

Some people might and some would not. It does not matter. Users need not spend all their time in shared-augmented reality. They are likely to carry the equipment around with them anyway as wearable computers would have other functions such as communication, contact management, etc and a mobile phone has clear uses. As wearable computing gets more useful, people will be spending more time wearing the required equipment. Users will enter the shared-augmented reality when they wish to do so. This is quite adequate for establishing the system in society. People carry mobile phones around them and frequently send text messages, yet do not send text messages every second of their waking lives: this does not prevent text messaging from being socially significant.

Could objects be made solid?

The short answer is 'no', but with some qualifications.

It may be possible to imagine exotic technologies that do allow objects to be solid in a system like this, but it is dubious that this would even fall under the definition of a shared-augmented reality as the objects would probably have to be real and any system that could support this would probably do it by allowing objects actually to be physically created very quickly. It may be possible to imagine this with extreme technologies - the molecular nanotechnology 'utility fog' concept seems to be close - but this article is not about such technologies: it is about what we could do, reasonably economically, with what we have now or not much more than what we have now. The objects in the shared-augmented reality only exist as stereoscopic images produced in special glasses. They cannot be solid.

If it is important that shared-augmented reality objects appear solid there is one way of dealing with this: a real, solid object can be put in the same place as the shared-augmented reality object. This may appear strange: if we have gone to all the trouble of putting a solid object there does that not make the presence of a virtual object redundant?

There could be a use, however, for combining virtual objects with real objects. A real object could be very plain, while a virtual object has a detailed texture. Simply by doing some editing, the texture of a virtual object could be changed, giving us the advantages of using shared augmented reality while the presence of the real, physical object gives us solidity. The physical object could also be more crudely shaped. As an example, we could make a detailed statue in virtual reality and overlay it on top of a very crudely made statue - maybe even just a simple block of wood. A user encountering the statue would see the detailed virtual statue and if he/she tapped it with his/her fingers he/she would feel a hard surface. There could be a reasonable illusion that the hard surface is part of the virtual statue, and not associated with a much simpler object in the same place.

Real, physical objects could be placed inside or near virtual objects to complement them in various ways. As an example, a virtual building could be made with a very basic real, physical structure which provides simple walls, floors and stairs. These may or may not be overlaid by more detailed objects, and virtual objects could be placed throughout the building to turn it into a more detailed structure.

One of the scenarios that I gave described a game, played with virtual robots, in which virtual walls had been made to provide cover. A game like this could be set up inside a building with a more solid environment. Simple wooden walls and ramps could be used to make the environment and overlaid shared-augmented reality objects added to make the walls, ramps, and floors look more details and provide other features such as killer robots or monsters.

Could Objects be Placed on Users?

In two of the scenarios that I gave - the one with the practical joke in the pub and the one with the robot game - I mentioned the possibility of objects sensing the position of users and behaving accordingly.

This is quite feasible. Each user's mobile phone is sending details of its location to the server continuously anyway, for any of this to work, so all that would be needed for an object to follow a person around is the facility for a script to access the information about the locations of nearby users.

Some users may not be too happy about this from privacy point of view and their concerns could be dealt with in these ways:

• Allowing a user to enable/disable the visibility of his/her coordinates to scripts in other users' objects.
• Only allowing scripts to determine the coordinates of users who are nearby, preventing objects from tracking users who are far away.
• Making it so that if a script wants the coordinates of a nearby user a request is made to the user by the system. This solution could be problematic if lots of requests get made all the time as it could cause annoyance to users.

If objects can be made to follow users then it raises the idea of objects that follow users so closely that they appear to be attached to users as if they are worn. A simple example would be a red sphere that follows you around and attempts to keep you in its centre: if the sphere's script could do this then it would appear that the sphere has been placed on you and that you are wearing it.

Some readers may wonder about taking this further. Could virtual costumes be made? For example, could I make a virtual spacesuit which follows me round so that I appear to be wearing it? The answer is - not quite yet. To produce something as complicated as clothing would require that objects could be positioned accurately on a user's body, but the shapes of people's bodies change as they move their limbs. The system would need to know where the user's arms and legs are to position virtual clothing on the user and this is not part of the proposal I am making for the sort of system that we could make now.

If shared-augmented reality would not be able to provide virtual clothing just yet it could provide some fashion accessories. Various objects could be made to accompany a user to complement whatever real clothing he/she wears. As an example, people in nightclubs could use objects that follow them and produce virtual lighting effects.

Shared-Augmented Reality and Business

The most obvious businesses related to this proposal are as follows:

• Producing the hardware and software to allow users to connect to shared-augmented reality.
• Providing the network services that allow people to experience shared-augmented reality.
• Hosting objects within shared-augmented reality.

There would, however, be many other business opportunities associated with producing objects and scripts for use in the shared-augmented reality. A business could, for example, make a virtual animal which follows the user around, a virtual building, a part for a virtual building, an object for a game or a virtual effect for use in a nightclub and sell copies of it to users for real money. Some objects could be sold to fulfil trivial purposes while others would be more practical.

If it seems wishful thinking that users would pay real money for virtual objects like this it should not: thousands of ring tones are currently bought and downloaded every week.

The Future

Technological developments would increase the sophistication of shared-augmented reality and the following could be expected to happen:

• Improvements in the realism of graphics, the amount of detail that can be stored, the memory available for storing objects, the ease with which objects can be built and the ease with which objects can be scripted.
• Improvements in the ability of objects to interact intelligently with humans. This would be caused by general progress in artificial intelligence.
• Reductions in the size and weight of the equipment needed to experience the shared-augmented reality; for example, smaller, lighter virtual reality glasses. Eventually, it may even be possible to have the display screens built into contact lenses, or for the imagery to be placed directly into the optical centres of the brain through some kind of machine/brain interface [20] so that users do not have to wear any glasses at all.
• Virtual clothing becomes possible in shared-augmented reality.

Conclusions

A proposal has been made for the use of mobile phone networks and virtual reality equipment to construct a shared virtual reality, modifiable by its users, which is overlaid on reality so that objects within the virtual reality appear to be part of reality.

Such a system could have a considerable impact on society and play a part in almost all aspects of human life.

In the author's opinion, construction of basic systems with this sort of capability is now feasible or very nearly feasible.

Note

The author has an account on SecondLife^™ (http://www.secondlife.com) under the name 'Diaspar Medici'.

References

[1] Interapple, Inc PA USA. (?). Holodeck. Retrieved on 14 December 2004 from http://www.webster-dictionary.org/definition/Holodeck
(Related to fiction).

[2] Bell, J. (?). Holodeck and Computers FAQ. Retrieved on 14 December 2004 from http://www.calormen.com/Star_Trek/FAQs/holodeck-faq.htm#how%20real
(Related to fiction).

[3] Clarke, A. C., (New Edition, 1993). The City and The Stars. London: Victor Gollancz. Chapter 7, pp62-63. (Fiction). (Originally published:1956. New York: Harcourt, Brace and World )

[4] Ibid. Chapter 3, p24. (Fiction).

[5] Linden Research Inc. (?-2004). SecondLife^™: Your World. Your Imagination. Retrieved on 13 December 2004 from http://www.secondlife.com.

[6] Linden Research Inc. (?-2004). Linden Lab: Creators of SecondLife^™. Retrieved on 13 December 2004 from http://www.lindenlab.com.

[7] Wagner James AU (writing as Hamlet Linden). (2004). New World Notes: The State Of The Game-Nation: An NWN-At-SXSW Presentation. Retrieved on 14 December 2004 from http://secondlife.blogs.com/nwn/2004/03/sxsw_talk.html.

[8] Gwyneth Llewelyn (SecondLife^TM name). (2004). Gwyneth's Home. Retrieved on 14 December 2004 from http://secondlife.game-host.org/article5visual1layout1.html.

[9] Phil Cassidy (SecondLife^TM name) (?). The Fever Nightclub: A place for funky 70's classics. Retrieved on 14 December 2004 from http://cromwellgames.co.uk/fever/fever.html.

[10] ?. (?). Trade and Services Directory: Locations - Malls. Retrieved on 14 December 2004 from http://trade.secondserver.net/show_catagory.php?catagory_index=8.

[11] Wagner James AU. (2003). New World Notes: The War of The Jessie Wall. Retrieved on 14 December 2004 from http://secondlife.com/notes/2003_07_14_archive.php.

[12] Jupitermedia Corporation. (?). Short Message Service. Retrieved on 14 December 2004 from http://www.webopedia.com/TERM/S/Short_Message_Service.html.

[13] MIT Media Lab. (?). Wearable computing at the MIT Media Lab. Retrieved on 14 December 2004 from http://www.media.mit.edu/wearables/index.html.

[14] Trimble Navigation Ltd, (2004?). All About GPS. Retrieved on 14 December 2004 from http://www.trimble.com/gps/.

[15] Dana, P, H. (1994). Global Positioning System Overview. The Geographer's Craft Project, Department of Geography, The University of Colorado at Boulder. Retrieved on 14 December 2004 from http://www.colorado.edu/geography/gcraft/notes/gps/gps_f.html.

[16] Gibson, W. (1993). Virtual Light. New York: Bantam Books. (Fiction).

[17] Wilson, D. (?). David Wilson's GPS Accuracy Page. Retrieved on 14 December 2004 from http://users.erols.com/dlwilson/gps.html.

[19] Brashears, A., Meadows, A., Ondrejka, C., Soo, D., (2003). Linden Scripting Language Guide. Retrieved on 14 December 2004 from http://www.sluniverse.com/lsl.html.

[20] Branwyn, G. (1993). Wired 1.4: The Desire to be Wired. Margulis, Z., (?). Going Mental: Let Your Neurons Do the Typing. Retrieved on 14 December 2004 from http://www.voidspace.org.uk/cyberpunkproject/desire_wired.shtml.