Faster Maintenance with Augmented Reality


Faster fix: A U.S. Marine technician wears an augmented-reality headset as he carries out a maintenance task inside an armored vehicle. Credit: Steven Henderson and Steven Feiner

This article in Technology Review is a very good example of the use of AR in a Reverse Logistics scenario.  It demonstrates a real world use with quantifiable benefits.

Columbia University researchers have developed an AR system that allows Marine mechanics to perform a maintenance routine in approximately half the time traditionally taken.

The Columbia researchers worked with mechanics from the U.S. Marine Corps to measure the benefits of using an AR headset when performing repairs to a light armored vehicle. Currently, Marine mechanics have to refer to a technical manual on a laptop while performing maintenance or repairs inside the vehicle, which has many electric, hydraulic, and mechanical components in a tight space.

A user wears a head-worn display, and the AR system provides assistance by showing 3-D arrows that point to a relevant component, text instructions, floating labels and warnings, and animated, 3-D models of the appropriate tools. An Android-powered G1 smartphone attached to the mechanic’s wrist provides touchscreen controls for cueing up the next sequence of instructions.

I’d love to see this being applied more widely in the technology repair industry.  Keeping abreast of all the designs and build methods of the equipment we repair is one of the biggest challenges for us at Service Plus.  AR systems could dramatically improve our performance and ensure consistent service levels.  Obviously, there are also significant benefits in developing countries.

A great video of the system here.


Layar + iPhone = Cool


Layar has a smart way of displaying tags without cluttering the screen (too much).

Probably the most progressive and most watched of the geospatial augmented reality developers out there is Layar. Their Android version provoked a lot of interest and they have now released their iPhone app. It’s a free program and appears to be available worldwide (although some of the content is limited).

The most interesting thing about Layar is that they have opened their platform to other developers to provide “Layars” of content. Each Layar can have a unique look and feel… for instance, the Wikipedia Layar shows tags as the Wikipedia “W” until they are highlighted at which point they become the Wikipedia “globe”. Whilst many of the Layars are fascinating, unfortunately there is very little data available in New Zealand so I wasn’t able to try them out.

One thing I do like about Layar is the free text Google search. Searching for “computer service” came up with about 20 hits within 1km of me and rotating to select different tags is the easiest of the video overlay type apps I have tried. Layar have cleverly avoided the necessity to hold the phone over your head to see further away hits (a la acrossair) by positioning the tags on a grid representing distance. Also, rather than trying to display all of the information in the tag on the overlay, Layar draws a line to the bottom of the screen and references the info there. This allows a far more comfortable user experience.

Battery usage and heat are big issues with video overlay type AR apps on the iPhone, and Layar is no exception. You can choose a map or list view but that displays 360 degree hits so you lose the pointing functionality. It also doesn’t bookmark the last place you were at in the app… so if you get map directions, then reopen Layar, you have to navigate back to the place you left off. GeoVector World Surfer is superior in both these aspects.

There are a couple of Twitter Layars and I must admit that it feels a little voyeristic to view tweets from people based on them being nearby… but I suspect that the social media aspect of AR will be a big development in the future.

I’d strongly recommend Layar to any iPhone 3GS users out there. As a party peice you’ll get plenty of “ooohs” and “ahhhs”. There’s still a lot of room for improvement in the AR sphere, but this a good start… and it definitely has the “Cool” factor.

Two new AR iPhone apps

Two new iPhone 3GS augmented reality applications were launched over the weekend.  I think it’s fair to say that the iPhone now has a broader range of AR systems available than the Android platform… a big change from about a month ago when the iPhone had none.

Wikitude has launched the iPhone version of their popular “magic lens” type Android app.  With this system users are able to view tags overlaid on the camera display.  The content is limited to Wikipedia, Qype, and Wikitude.Me (user generated data).  My first impressions are that it is more effective than acrossair’s NearestWiki (mainly becasue the tags are much smaller and therefore don’t crowd the screen), and there are some nice features like the little “radar screen” in the bottom left corner, and the ability to easily adjust the range.  However, there are some flaws like the fact that when multiple tags are in the same area they get in the way of each other and there is no way to get to the ones in the back.  Also, the range always defaults to the maximum every time you open the app… it would be easier to have it “remember” the range set at the last use.

Overall, though, a pretty nice app and one to watch in the future.  Also, Wikitude have released an API to allow third party developers to use their system as a platform… very interesting.

Cyclopedia is very similar to Wikitude in that it presents Wikipedia content in a magic lens display (click the link to see a video demo).

Unfortunately, I just couldn’t get it to work on my iPhone… so I can’t comment on its functionality except to say it seems to have a bug at boot up.  I’m not making too big a deal out of that because I’ve noticed the iPhone OS 3.1 seems to be causing a few issues in AR type apps (GeoVector have just released an update addressing a bug caused by 3.1).  Hopefully, they will sort this out soon and I will be able to use the app properly.

Why is Reverse Logistics so important now?

The concept of reverse logistics is complex and foreign to most people, but it is more important now that it has ever been. The economic downturn has resulted in organisations seeking to reduce costs and improve customer satisfaction – usually regarded as incompatible concepts.  Manufacturers cannot maintain their high cost bases, but at the same time must not risk losing even more sales through reduced customer satisfaction.  Retailers must focus on sales above all else, but can’t allow that to reduce the quality of their service.

Luckily, specialists in reverse logistics are there to achieve both outcomes.  Organisations can outsource their returns and repairs operations to specialists and achieve both reduce fixed costs, and the service improvements that specialists can offer.

Additionally, the environmental effect of constant consumption is massive.  Reverse logistics has a critical part to play in protecting the world we live in by extending the life of goods and recycling them after their use.

This article from Dr. Harold Krikke in the Reverse Logistics Magazine discusses the current economy and the influence reverse logistics has.

Reverse logistics not only saves money, it actually creates value by supplying vital materials, components and products to the ‘forward’ chain. A well-known European copier firm now sources almost 50% of its parts and materials via its own Asset Recovery department, often in collaboration with first tier suppliers and specialized repair firms.

We overlook the post-sales management of consumer goods at our peril.

acrossair for iPhone 3GS – my first impressions

acrossair has just launched their iPhone 3GS apps

acrossair has just launched their iPhone 3GS apps

The AR field is really hotting up now.  acrossair has launched a suite of new apps for the iPhone 3GS (only) that display tags for searched items on the video display of the real world.

Fancy a coffee and would like to know where the nearest coffee house is? Look no further than ‘Nearest Places’. Whether it’s to find out where the nearest hotel, supermarket or museums are, this will guide you to where you would like to be.

I’ve been using the Nearest Places app today and it is a nice example of simple video overlay AR.  Here are my impressions.

First the bad stuff:

– I had a couple of crashes when first using it (but none since).

– More distant targets don’t display unless you scroll the screen down or tilt the handset further and further over (you find yourself looking up at the phone over your head).

– The content is patchy (at least it is in NZ). Many of the targets that I can easily find on Geovector World Surfer are not displayed on Nearest Places.  Also, I found many double-ups (I’m guessing they’ve sourced content from a couple of providers).

– You can’t search in horizontal (map view) mode.

– The tags are too big (contain too much info) and cause too much stacking of the search results. I think the display will get very cluttered as more geo-located content becomes available.

It seems to me acrossair have rushed this app to market in order to be one of the first AR apps on the iPhone. However, this is otherwise a good example of an early AR app and all the issues above are easily resolved in updates and future releases.  It has some great features:

– Free text searching (something that is lacking in GeoVector’s app).

– Colour coded categories are displayed in the viewer as you tilt the screen up from map mode.  Nice and intuitive. You can swipe to scroll through the categories and the appear to rotate around you.

– The entire screen is used for displaying the results.  No tool bars or buttons are used.

– The display can be used in both landscape and portrait mode with seamless transition.

– Tags show target name, distance, address, phone number and star rating. Click on the tag to see more info and get directions.

– It’s fast.  The tags move smoothly as you pan, and search results come up quickly.

It’s curious that acrossair has chosen to release multiple apps rather than one “killer” app that incorporates all of them.  Certainly their Nearest Wiki app could simply be a category in Nearest Places.

Overall, I think acrossair is a true contender in AR, but they will need to continue to develop their usability and content to ensure users like me keep using their apps.  It is one thing to download an app and be excited by it’s novelty… quite another to find it so useful that you keep using it regularly.

I guessed wrong…

Layar is going 3D

Layar is going 3D

Layar and Wikitude have now made their statements and they haven’t released their iPhone apps at all… but their news is even more interesting.

Layar have announced that they are adding 3D to their system as of November.    3D is important to augmented reality as it allows a truly immersive experience.  The examples that they’ve demo’ed on their site aren’t that inspiring, but it will now be up to content providers, developers, and the public to create suitable 3D content.

Wikitude’s news is possibly more important.  They have proposed a new open markup language for augmented reality called ARML (Augmented Reality Markup Language). There’s been a lot of discussion about the need for a standard language in AR.

Establishing an open ARML specification will allow users to access any AR data in the physical world from any AR browser. It will also accelerate innovation and allow for more, better, and less expensive AR apps.

As the technology develops, AR players will need to collaborate in certain areas but will also need to protect the intellectual property that is particularly relevant to them.  Defining that boundary between open and proprietary is going to be one of the biggest challenges for the industry.

What will be the AR hardware standard?

GeoVector's World Surfer is a great example of an iPhone AR app

GeoVector's World Surfer is a great example of an iPhone AR app

As an iPhone user, I think it’s great that these augmented reality apps are finally coming out for my addiction, but is the iPhone really the best device currently available for AR?  After all, AR has been available on the Android G1 phone for nearly a year now (see Wikitude).

Certainly the iPhone provides a fantastic user interface (and that is essential for AR).  I also suspect that all the upcoming variants of Android phones are going to be a nightmare for AR developers to maintain compatibility; the last thing anyone wants is a flaky AR experience – and this is going to be a major issue for Google as AR becomes mainstream.

But probably an even bigger reason is that iPhone users are generally happy to pay for applications… and that is critical in the developing field of AR because, currently, no-one appears to be making money in any other way from AR apps (sweeping statement).  That will change as the technology matures, but by then the horse will have bolted.

Obviously, many other AR devices are being developed – and several of those promise to be much better suited to AR – but it will take a long time for them to gain the critical mass needed to attract developers.

So I’m going on record as saying the iPhone will be the springboard platform for augmented reality… feel free to prove me wrong.