Ritsumeikan Univ.
College of Information Science and Engineering
Physical Computing Course


You need a player that can play mpeg movies�D

[section]Themes of Research

Gr.1 Mobile Mixed Reality System

- Framework for mobile mixed reality


�R���Z�v�g�} ���s�҃i�r�Q�[�V�����V�X�e�� �o�ዾ�^HHD CattleMutilation
Researches on Mixed Reality (MR), a new human interface technology, have been watched with interest. Up to now, most MR systems were indoor stationary type. In the years ahead, mobile MR systems using portable equipments will become widely developed. They can be used for various usages. According to such a background, we have been developing framework which is applicable to various portable equipment for mobile MR systems. Our framework can absorb differences of performance and functional capability of equipments and provide independence from applications. We designed the framework as a server-client architecture and implement some mobile MR systems for validating and improving the framework.

Until now, we developed some mobile MR systems based on our framework using cell-phone, smart-phone, PDA, wearable PC and laptop PC. "Cabling system" enable us to see cables under floor boards of OA floor. "Mobile navigation system," a blue bird takes us to the destination. "Campus guide system" annotates buildings on campus. We implement mobile MR systems and try to make an improvement of our framework continuously. And with the aim to verify function of the framework, we made three systems which are described in above.

- history

2006
We tried the opposite approach that developed "LAN cable line support system" which helped easily and dynamically wiring LAN cable line on a patch panel. And we confirmed that the framework that had ability to solve social problems.

2007
As a new technique of information presentation, we developed "A view administrative function" by a binocular Hand Held Display (HHD), which kept up the hue of sequence annotation. And through the use of a view administrative function, we improved on this canvas guide system.

2008
We redesigned it based on the problems of the conventional framework.

2009
We developed "Cattle Mutilation" which an user shots down UFOs taking cows away, it confirmed to control the motion of contents and share the same MR space in real time at mobile devices.

2010
We have included a physics engine to the framework. In addition, it is able to control the sound.This expression of the framework has been improved.

2011
Furthermore, we developed a framework in two directions. One framework that can be distributed across multiple servers features. Another type is a framework of P2P. Respectively, expand the scale of application and fast response is possible.

In addition, we developed "Interior coordination system" which supports interior decoration of a room by using mobile devices, "MR Gymkhana" which an user controls a vehicle by physics engine and "Educational material applications" which support learning in study. As a result, the framework was effective for developing mobile MR systems with confirming its function.


"Underfloor Wiring Support System"


"Pedestrian Navigation System"


"Campus Guide System"


"Campus View Application with HHD"


"Cattle Mutilation"


"Interior Coordinate System"


"MR Gymkhana"



- SFINCS (Semi-Fiducial INvisibly Coded Symbols)


SFINCS-TT SFINCS-PM
Geometric registration between the real and virtual worlds is one of the most important problems for realizing the MR environments. A variety of registration methods have been proposed in worldwide. In general, traditional methods can be classified into three types: sensor-based methods, image-based methods, and hybrid methods that integrate sensor-based and image-based technologies. Among them, image-based registration methods, which use markers placed in a real scene, have more potential because they do not require any sensors except a camera. In particular, ARToolKit is globally known for its high quality and portability. However, there are many claims that such fiducial markers are not visually appealing. Accordingly, we propounded semi-fiducial invisibly coded symbols (SFINCS) that are able to achieve a good balance between elegance with regard to the environment and robust registration.
We attempted to realize SFINCS using the following two approaches: (1) using markers that are harmonious with the real scene, and (2) embedding information into objects that naturally exist in the real scene.

- Based on approach (1)
We proposed a registration method using two-tone colored markers. This method is named as two-tone colored marker method (SFINCS-TT). Our two-tone colored markers are similar in color to that of the background objects. These markers have the same hue value as the background color and differ slightly in brightness. T-shaped, L-shaped, I-shaped, and like markers are placed at the corners of flat colored plain surfaces of actual objects.

- Based on approach (2)
We developed a geometric registration system using visually unobtrusive flat posters arranged in the real world. This method is named as poster masquerading method (SFINCS-PM) which uses posters that have some common characteristics for geometric registration. These characteristics are used for finding the posters in real time.
In these two methods, we have developed an authoring tool to design markers efficiently, and we have implemented our methods using mobile devices like smartphones and tablet computers. We have studied further convenience for registration methods.


"Registration method using two-tone colored markers "SFINCS-TT""


"Registration method using posters "SFINCS-PM""



- Mobile Registration

Mobile regist MobileTracking_ExecutionScreen MobileTracking_SceneryOfOperation
In recent years, many applications that achieve Augmented Reality on the mobile devices are developed along with the spread of such as the smartphone. Those applications estimate the position and orientation of the mobile device by using sensors such as GPS for realizing registration of real world and virtual objects. However there is a problem that those applications can't draw virtual objects by the right position accurately, because the measurement error which caused by estimating the position and orientation of the mobile device with sensors such as GPS is large. High accurate registration is required so as to achieve Mixed Reality that merge of real and virtual worlds. Then, many methods of estimate the position and orientation of the mobile device that use the artificial marker or the natural feature points are proposed. However, those methods often take too much data processing load to the mobile devices. Then, we proposed the client-server model to overcome a limitation of treatment performance.

In this proposed mechanism, the first registration is processed by the server, because it takes too much time when it is executed on the mobile device. Furthermore, Client executes camera tracking for real-time after first registration. Information which can be used for registration differ in according to the environment. Our suggesting system is possible to realize registration in various environments by changing the best registration method. The purpose of this study is to solve the problems in installing each registration methods on mobile devices. Moreover, we aim to build our proposal methods to existing common ground for mobile devices.

In current research, we introduced a natural feature-basedregistration method using reconstruction techniques into our proposal mechanism. In the future, we will introduce various registration methods into the servers of proposal mechanism to achieve more general one. In addition, we will also construct registration techniques which track the objects besides natural feature points for the client side.