Available online at www.sciencedirect.
com
ScienceDirect
Procedia Computer Science 25 (2013) 71 – 79
2013 International Conference on Virtual and Augmented Reality in Education
3D Outdoor Augmented Reality for Architecture and Urban
Planning
Arnis Cirulisa*, Kristaps Brigis Brigmanisb
a,b
Virtual and Augmented Reality Laboratory, Sociotechnical Systems Engineering Institute of the Vidzeme University of Applied Sciences,
Cesu str. 4, Valmiera, Latvia, LV-4200
Abstract
The aim of this paper is directed in field of architecture and urban planning process improvement, by allowing more precise
evaluation of new constructions and objects look and visual influence on environmental surrounding. Nowadays there are
several solutions for city visualization in virtual reality which provide wide functionality, including spatial visualization.
Despite all modern technologies, which provide photorealistic models on stereoscopic monitors and screens, it is still
noticeable barrier among virtual buildings and buildings in real environment. To enhance the immersion level of urban
planning solution, main task is to find a way how to use augmented reality to allow urban planning experts move around the
city streets and project virtual three dimensional buildings, allowing to see real city and virtual buildings at the same time.
© 2013
© 2013The
TheAuthors.
Authors.Published
Publishedby byElsevier
ElsevierB.V.
B.V.Open access under CC BY-NC-ND license.
Selectionand
Selection and/or peer-review
peer-review under
under responsibility
responsibility of theofprogramme
the programme committee
committee of theInternational
of the 2013 2013 International Conference
Conference on
on Virtual
Virtual
and and Augmented
Augmented Reality
Reality in in Education.
Education
Keywoards: Virtual reality (VR); augmented reality (AR); markerless tracking; urban planning.
1. Introduction
Due to rapid development of virtual and augmented reality technologies, due to improvements of
performance parameters and increase of technologies availability, it is important to discover new innovative
approaches of technologies usage in different branches. Architecture and urban planning has an essential role in
our lives and fundamental contribution can be delivered to this field of specialization by modern technologies
like augmented reality (AR). It is important to provide wider use of qualitative AR technologies solutions, to
come out of narrow scope of usages, mostly based on two dimensional data depiction or marker based systems.
* Corresponding author. Tel.: +371 28301333
E-mail address: arnis@va.lv
1877-0509 © 2013 The Authors. Published by Elsevier B.V. Open access under CC BY-NC-ND license.
Selection and peer-review under responsibility of the programme committee of the 2013 International Conference on Virtual and Augmented Reality
in Education
doi:10.1016/j.procs.2013.11.009
72 Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79
Depiction of three dimensional (3D) data is crucial for urban planning stages and using AR technologies it is
possible to achieve more precise evaluation of new constructions and objects look and visual influence on
environmental surrounding.
There are already available solutions for city visualization in virtual reality, but to achieve better results and
improve immersion level of planning process, AR technologies are a key, providing endless possibilities,
however not all ideas can be reached immediately, and still lot of work and research should be done for AR
logical and physical structure. Although Furht [1] bring forward four types of applications that are most often
used for AR research and these applications are advertising and commercial, entertainment and education,
medical, and mobile applications for smartphones. For all that Furht [1] also encourage new directions, because
there are many possibilities for using AR in an innovative way and also new challenges AR is facing to go from
the laboratories to the industry. In this paper the innovation is based on development of application City 3D-AR
for urban planning, allowing to merge real city with virtual three dimensional (3D) buildings (Fig. 1).
Fig. 1. Bus station augmented with new building for urban
planning and public discussions
To enhance the immersion level of urban planning solution, main task is to find a way how to use
augmented reality to allow urban planning experts move around the city streets and project virtual 3D
buildings, allowing to see real city and virtual buildings at the same time. Beneficiaries are architecture centres,
local authorities, companies of architects, art institutions, municipal archives, museums, and market parties. But
not only, as Allen et al. [2] proved, such interactive solutions are great way to increase public participation in
urban planning processes, only functionality of the system should be limited, so the users be able to view the
available 3D models and vote for the models according to personal preference. The idea is to keep the
functionality to a minimum in order to limit any confounding factors in the user experience of the system. As
such, the graphical user interface (GUI) had to accommodate these functions in the most direct, user friendly
way possible. Additionally, the physical structure and immersion level also plays a role, professionals could
have expensive head mounted displays (HMD) with AR support and wearable computer, common users can use
tablets for more reasonable prices (Fig. 1.), however latest achievements in HMD development from Vuzix [3]
offer light way and high performances HMD sets, which have tendency to decrease price year by year. And
Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79 73
good evolution direction, is that HMDs with AR functionality for mobile devices appears, meaning comfortable
replacement for wearable notebook solutions.
The main challenges of this paper and the project City 3D-AR are depiction of bulky three dimensional
objects in outdoor environment, buildings position detection that depends on participant's distance and viewing
angle. Traditional approach with fiducial markers cannot be used; it is suitable for indoor solutions in short
distances.
Project Work Executed at Sociotechnical Systems Engineering Institute of the Vidzeme University of
Applied Sciences, Virtual and Augmented Reality Laboratory. Two basic sets of hardware and software were
used. The first set consists of tablet PC (Asus Transformer Pad TF300TG) with Android operating system,
integrated position sensors, and visualization software libraries (more detailed in last section). The second set
contains notebook (Asus G71Vseries), head mounted display with functionality of augmented reality (Vuzix
Wrap 920AR), USB positions sensors (OceanServer evalution kit OS5000, Phidgets GPS and Spatial 3/3/3),
gyroscopic mouse (Logitech MX Air Mouse) and VR/AR authoring platform (3DVIA Virtools) with building
blocks for AR support.
2. Merging real and virtual environments on mobile devices
The history of necessity to merge real and virtual environment is longstanding, however definitions are
formulated just in last decades. If the first approaches of mixing real and virtual appeared in cinematography in
1950s, then definition appeared only in 1997 by Azuma [4]. History facts are summarized by Kent [5], and the
first entry devoted to cinematographer Morton Heiling, who created Sensorama simulator in 1957, this
simulator was patented and provided visuals, sound, vibrations and smell. The AR term appeared only in 1990
by Tom Caudell whose work was related to aircraft maintenance works. Azuma's definition [4] in 1997
prescribed AR as combination of real and virtual worlds in 3D with real-time interaction capabilities. Despite
this definition, most of nowadays AR applications still offer real worlds augmentation with two dimensional
data. Most of applications from Google Play and Apple's App Store for Android and iPhoneOS based
smartphones and tablets provide two dimensional information. For example, text or image based descriptions or
instructions, navigation arrows and instructions, or poorly interactive marker solutions with 3D object
integration. Rapid growth of mobile devices usage and also technological leap of supported features makes
professionals to think over new solutions in different economic sectors. Modern mobile and entertainment AR
systems use one or more of the following tracking technologies: digital cameras and/or optical sensors
(Nintendo Wii, Sony PlayStation Move, Microsoft Kinect), accelerometers, GPS, gyroscopes, solid state
compasses, RFID and wireless sensors. These technologies offer varying levels of accuracy and precision. By
Kent's literature analysis [5] most important is the position and orientation of the user's head. Tracking the
user's hand(s) or handheld input device can provide 6DoF interaction technique. In Furht's turn [1] performance
is relevant consideration. In spite of rapid advances in industry of mobile devices, their computing platform for
real time imaging applications is still rather limited if done using cell phone's platform. As a result, many
applications send data to a remote computer that does the computation and sends the result back to the mobile
device, but this approach is not well adapted to AR due to limited bandwidth. Nevertheless, considering the
rapid development in mobile devices computing power, it can be considered feasible to develop real-time AR
applications locally processed. This Furht's conclusion [1] confirmed a choice for City 3D-AR pilot product use
the tablet PC with camera and already integrated sensors. More immersive solution with Vuzix AR HMD will
be developed after development of theoretical model and working portable tablet solution. The usefulness of
mobile devices use is also affirmed by Lee [6]. Mobile computing devices have been widely deployed to
customers; the interest in mobile AR is increasing. As the need for mobility is growing, computing devices
shrink in size and gain acceptance beyond an audience of tech-savvy specialists. With the help of enhanced
74 Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79
computing power, small devices have sufficient computational capability to process complex visual
computations. For example, a cell phone with a camera and wide screen can serve as an inexpensive AR device
for the general public. Emerging wearable computers are also excellent platforms for enabling AR anywhere
users may go.
3. Technologies and approaches of augmented reality (AR)
Today, image processing and computer vision technologies have been progressed to a stage that allows us to
infer the 3D information of the world directly from the images. Because of the success of these technologies,
more and more vision-based AR applications are emerged. Augmented reality refers to the combination of
virtual objects and real-world environment, so that users can experience a realistic illusion when using the
interactive virtual object to explore the real-world environment [7]. Several technologies and approaches of
augmented reality implementations are available today. These technologies can be divided in two basic groups:
marker-based systems and marker-less systems. Kan et al. [7] explains marker-based systems by use of specific
marker for 3D tracking and positioning. This marker is employed to identify the corresponding virtual object
that is to be placed in the real-world environment. When the marker is used as a tracking target, it has to be
registered in the system in advance, as well as the virtual objects it associates with. However, since the
registered information is independent for different AR systems, markers used in one system may not be applied
in another system, unless an additional registration procedure is applied. For that reason in effective way RFID
technologies could be used. Ginters et al. [8] introduced AR-RFID low cost visualization solution for logistics.
To visualize the items in the storehouse which must be moved to assembly bay the AR-RFID solution is
used ensuring additional possibilities of checking. That allows reducing the amount of potential errors and
diminishing possible losses. RFID scanner reads the component code and 3D model of the item is visualized on
the screen. AR-RFID by Ginters et al. [8] eliminates the problem of generating an equally stable and
recognizable fiducial marker, because single marker is used only for tracking, not to recognize the 3D model.
This makes it possible to use single marker with the highest stability. To identify the 3D model, an
identification code, which is read from the RFID tag, is used. The 3D model is found using the identification
code and displayed on the screen of the tablet PC. As well considerable marker-based version is described by
Kan et al. [7] how traditional fiducial markers are replaced by QR codes, allowing to use AR in public domain
systems. The QR code can be easily generated by any user and the AR system can track it no matter what the
information it embedded. Thanks to FullHD video cameras and increase of computing performance, it is easier
to pull down traditional marker restrictions, Seac02 company offers in its authoring platform LinceoVR to use
self-taken photos as markers, providing limitless object-marker alignment options also solving the problem
with fiducial marker hiding, which sometimes influences immersion level, if it is visible.
The second AR technologies group is based on marker-less solutions, meaning that marker-less pose
trackers mostly rely on natural feature points which are visible in the users environment. Four requirements are
summarized by Herling and Broll [9] to allow accurate pose determination from natural points. These
requirements are as follows: fast computation time, robustness with respect to changing lighting conditions and
image blurring, robustness against observation from different viewing angles, scale invariance or changing
viewing distance. Numerous feature detectors have been proposed providing different properties concerning
detection robustness or detection speed. Herling and Broll [9] divides exiting detectors in two classes: corner
detectors spotting corner-like feature, blob detectors not spotting unique corner positions but image regions
covering blob-like structures with a certain size. Unfortunately these marker-less solutions cannot be used for
new building layout in vacant environments and urban environments. They are more suitable for existing
buildings and objects real-time overlay with additional data. In next section sensor marker-less tracking will be
introduced as most appropriate solution for outdoors environments (not separate objects) augmentation.
Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79 75
4. New usage possibilities and necessities of AR
Demands of real life scenarios and AR offered possibilities not always fall together. In this section GPS,
electronic compasses and inertial sensors will be introduced as the most versatile devices set to provide virtual
and real object tracking and placement capabilities in AR scenarios. As given by Reitmayr and Drummond [9]
these technologies have drawbacks as well. While GPS has satisfactory accuracy and performance in open
spaces, its quality deteriorates significantly in urban environments. Both the accuracy and the availability of
GPS position estimates are reduced by shadowing from buildings and signal reflections. Similarly, inertial
sensors are prone to drift and magnetic sensors are disturbed by local magnetic fields encountered in urban
environments. However worldwide practice indicates that most use occasions in public and professional
domains have GPS and other local sensor related technologies. Most commonplace in public obtains AR
applications developed for urban exploration, museums, travel and history, shopping, customer service, safety
and rescue operations, moving and decorating home. Already well known applications are Wikitude, Nokia City
Lens, Google Goggles and Metaio Junaio. The beginning of theoretically similar solutions, were introduced in
Azuma's research on virtual labels over outdoor landmarks in 1999 [11] and Thomas et al. ARQuake game in
2000 [12]. What still lack in available applications, is the depiction of marker less still or moving 3D objects.
Today's sensors capabilities in stability and precession have noticeably improved. For example, GPS accuracy
[13] is increased with differential GPS or DGPS which brings the accuracy of readings to within 1-3 meters of
the object, as compared to the 4-20 meters of normal GPS. DGPS works using a network of stationary GPS
receivers. The difference between their predefined position and the position as calculated by the signals from
satellites gives the error factor. This error component is then transmitted as a FM signal for the local GPS
receivers, enabling them to apply the necessary correction to their readings. Or Real Time Kinematic (RTK)
satellite navigation is a technique used in land survey and in hydrographical survey based on the use of carrier
phase measurements of the GPS, GLONASS and/or Galileo signals where a single reference station provides
the real-time corrections, providing up to centimetre-level accuracy.
Nonetheless most research work in AR community is devoted to video sequence analysis, model-based
tracking and hybrid solutions [6, 10, 14, 15, 16, 17, 18, 19].
Last section reveals algorithm and formulas used for City 3D-AR project, in which augmentation with 3D
models and sensors is accomplished, considering it as future widespread solution in different domains. The
importance of such solution for cities and urban planning is affirmed by Netherlands Architecture Institute and
their first implementation in UAR (Urban Augmented Reality) project.
5. Implementation of City 3D-AR for urban planning efficiency improvement
The main goal of a pilot product City 3D-AR is to provide 3D object placement in real space, using GPS
longitude and latitude coordinates. Physical structure is given in figure 2.
Also viewers or participants GPS coordinates are measured all the time, to support participant's movement
and automatic 3D objects transformations, rotations and scaling according to new angle and distance of a view.
Live video stream is provided from integrated tablet video camera or two cameras in HMD case, where
separate cameras are used for each eye to support stereoscopy. GPS sensor is used for participant's location
calculation in 3D space, because in current version is assumed that single 3D object (building) is still, but
participant can change location. Digital compass is used for sight direction detection and field of view
calculations, but gyroscopic sensor for tablet or viewers head orientations. Database of 3D objects is necessary
to provide an ability to change different architectural buildings in chosen spot. Selection of colour, material and
shader properties should be provided as well, to offer more interactive AR environment. For now simple 3D
76 Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79
object formats are supported, like OBJ and OFF.
Fig. 2. City 3D-AR physical structure
User uses his visual sense, to receive information from augmented environment, for interaction he can use
tablet's touch-screen, or other portable, wireless devices, like gyroscopic mouse or data glove for simple
selection operations.
Detailed achievements of City 3D-AR results are available in flow chart (see Fig. 3).
As mentioned in introduction, the first version is realized on AdroidOS tablet PC Asus Transformer Pad
TF300TG. AndroidOS includes support for high performance 2D and 3D graphics with the Open Graphics
Library (OpenGL), specifically, the OpenGL ES API. OpenGL is a cross-platform graphics API that specifies a
standard software interface for 3D graphics processing hardware [22]. For software development AndroidSDK
and Eclipse (Juno) with Android Development Tools (ADT) plugin is used. For 3D object depiction in block
nr. 2 OBJ and OFF file formats are used, for drawing both formats Android model viewing application's
ModelView code [20] is used. For video (block nr. 8) and 3D model (block nr. 14) data depiction two panels are
used which overlay each other in full screen mode. In bottom (SurfaceView class) there are data from video
camera and above it 3D model data (GLSurfaceView class). GLSurfaceView is an implementation of
SurfaceView that uses the dedicated surface for displaying OpenGL rendering [21] and solves transparency
problems. In 3D object's properties value for unit of measurement should be converted in meters, to gain
natural size for the building.
Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79 77
Fig. 3. City 3D-AR detailed flow chart
As the reference point and the centre of whole coordinate system tablet PC is used. In calculation process of
tablet PC's GPS coordinates and objects reference coordinates and direction angle in degrees, the local
orientation towards North of tablet PC is taken into account. For distance calculation d between two GPS points
(block nr. 9) haversine formula is used [23]:
a = sin²( /2) + cos( 1) * cos( 2) * sin²( /2)
c = 2 * atan2( a, (1 a))
(1)
d=R*c
where is latitude, is longitude, R is earth’s radius (6371km)
To gain higher precision between two GPS points Vincenty's formula [24] could be used which is used in
geodesy and is based on two related iterative methods to calculate the distance between two points on the
surface of a spheroid, developed by Thaddeus Vincenty. They are based on the assumption that the figure of the
Earth is an oblate spheroid, and hence are more accurate than methods such as great-circle distance which
assume a spherical Earth.
For direction calculation between current GPS position and object's reference GPS point (block nr. 10)
bearing formula is used [23]:
= atan2( sin( )*cos( 2), cos( 1)*sin( 2) sin( 1)*cos( 2)*cos( )) (2)
78 Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79
OpenGL functions [21, 22] for actual transformations, rotations and scaling are used:
glRotatef(float angle, float x, float y, float z)
glTranslatef(float x, float y, float z)
glScalef(float x,float y,float z)
where accordingly the current matrix is multiplied by a rotation matrix, current matrix is multiplied by a
translation matrix and current matrix is multiplied by a general scaling matrix. Also OpenGL [22] successfully
solves issues related to visible/invisible edges and sides.
6. Conclusions
Depiction of three dimensional (3D) data is crucial for urban planning stages. Importance of new
technologies usage is indisputable. By precise prediction of new architectural objects look and influence on
surrounding environment, lot of financial resources can be economized before constructions works are started
or completed. Also ease of public interests integration in discussion process can improve population
satisfaction and share decision making process.
Application City 3D-AR for urban planning, allowing to merge real city with virtual three dimensional (3D)
buildings and its logical structure can be implemented in different physical structure solutions. By using
geodesy, trigonometry computations, graphics libraries, are carried back AR technologies which physical
structure is based on GPS, gyroscopic, compass and inertial sensors.
The above-mentioned solution is still far from a real use for professional application and validation under
urban planning requirements is continuing, but these are perspectives for the nearest future. Also software
modules set developed and merged, serves as an experimental platform for future work, to ascertain actual and
effective use cases, considering viewing distance, movement speed, density of buildings, performance issues,
models complexity.
Acknowledgements
The current article was prepared in the framework of Valmiera Municipality Grants to Vidzeme University
of Applied Sciences in 2012 “Virtual buildings outdoor visualization for urban planning”.
References
[1] Furht B. Handbook of Augmented Reality. Florida Atlantic University: Springer; 2011
[2] Allen M, Regenbrecht H, Abbott M. Smart-phone augmented reality for public participation in urban planning, in: Proceedings of
the 23rd Australian Computer-Human Interaction Conference, Dunedin, New Zealand, ACM, 2011, pp. 11-20.
[3] The World Leader in Augmented Reality Hardware Systems, www.vuzix.com/augmented-reality/ (2012)
[4] Azuma R. A survey of augmented reality, Presence: Teleoperators and Virtual Environments 1997; 6(4): 355-385.
[5] Kent J. The Augmented reality Handbook - Everything you need to know about Augmented reality. 2011
[6] Lee T. Multithreaded Hybrid Feature Tracking for Markerless Augmented Reality. IEEE Transactions Visualization and Computer
Graphics 2009; 15(3):355-368.
[7] Kan T. QR code based augmented reality applications. In: Handbook of Virtual Reality. Springer New York, 2011. pp. 339-354.
[8] Ginters E, Martin-Gutierrez J. Low cost augmented reality and RFID application for logistics items visualisation, in: Annual
Proceedings of Vidzeme of Applied Sciences, ICTE in Regional Development 2011, pp. 23-33.
[9] Herling J, Beoll W. Markerless tracking for augmented reality. In: Handbook of Virtual Reality. Springer New York, 2011. pp. 255-
272.
[10] Reitmayr G, Drummond T. Going out: Robust Model-based Tracking for Outdoor Augmented Reality, in: Proceedings of the
International Symposium on Mixed and Augmented Reality, ISMAR 2006, pp. 109--118.
Arnis Cirulis and Kristaps Brigis Brigmanis / Procedia Computer Science 25 (2013) 71 – 79 79
[11] Azuma R, Hoff B. et al. A Motion-Stabilized Outdoor Augmented Reality System, in: Proceedings of IEEE Virtual Reality 1999,
pp. 252--259.
[12] Bruce T, Close B et al. ARQuake: An Outdoor/Indoor Augmented Reality First Person Application, in: Proceedings of the Fourth
International Symposium Wearable Computers 2000, pp. 139-146.
[13] Techniques to Improve GPS Accuracy, http://roseindia.net/technology/gps/techniques-to-improve-GPS-accuracy.shtml (2008)
[14] Guo J. Markerless tracking for augmented reality applied in reconstruction of Yuanmingyuan archaeological site, in: 11th IEEE
International Conference Computer-Aided Design and Computer Graphics, 2009. pp. 324-329.
[15] Karlekar J. Model-based localization and drift-free user tracking for outdoor augmented reality, in: Proceedings of the 2010 IEEE
International conference Multimedia and Expo ICME 2010, pp. 1178-1183.
[16] Ziaei Z, Hahto A. et al. Real-time markerless Augmented Reality for Remote Handling system in bad viewing conditions. Fusion
Engineering and Design 2011; 86(9–11):2033-2038.
[17] Dähne P, Karigiannis J. Archeoguide: System Architecture of a Mobile Outdoor Augmented Reality System, in: Proceedings of the
International Symposium on Mixed and Augmented Reality, ISMAR 2002, pp. 263--264.
[18] Gruber L, Gauglitz S. et al. The City of Sights: Design, Construction, and Measurement of an Augmented Reality Stage Set, in:
Proceedings of the International Symposium on Mixed and Augmented Reality, ISMAR 2010, pp. 157-163
[19] Comport A, Pressigout M. et al. Real-Time Markerless Tracking for Augmented Reality: The Virtual Visual Servoing Framework.
IEEE Transactions on visualization and computer graphics 2006; 12(4): 615-628.
[20] 3D Model Viewer for Android, http://code.google.com/p/modelview-android/
[21] OpenGL, http://developer.android.com/guide/topics/graphics/opengl.html
[22] McReynolds T, Blythe D. Advanced Graphics Programming Using OpenGL. Elsevier Inc.; 2005.
[23] Veness C. Calculate distance, bearing and more between Latitude/Longitude points. Movable Type Ltd., http://www.movable-
type.co.uk/scripts/latlong.html, 2002.
[24] Vincenty's formula, http://en.wikipedia.org/wiki/Vincenty's_formula
