Teaching VR with Magnum

Teach­ing about vir­tu­al real­ity al­ways be­ne­fits from prac­tic­al ex­per­i­ence. For that, a big part of the lec­ture Vir­tu­al Real­ity, hos­ted at the Lud­wig-Max­imili­ans-Uni­versität München, open for stu­dents of both Mu­nich Uni­ver­sit­ies, is to get hands on in­to their own VR pro­ject. This pro­ject has to run on the fa­cil­it­ies of the Vir­tu­al Real­ity and Visu­al­isa­tion Centre (V2C) of the Leib­n­iz Su­per­com­put­ing Centre (Leib­n­iz-Rechen­zen­trum, LRZ) of the Bav­ari­an Academy of Sci­ence and Hu­man­it­ies.1 Here, stu­dents use the fol­low­ing five-sided pro­jec­tion in­stall­a­tion for their own pro­ject.

Hav­ing past ex­per­i­ence with OpenSG, last year we de­cided to use Mag­num as the graph­ics en­gine for the stu­dents to start with — and we will stick to it for this year as well. We were lucky to get Vladimír to vis­it us in Mu­nich and give an in­tro­duct­ory lec­ture on how to use Mag­num. In this lec­ture, the stu­dents got a gen­er­al over­view of how Mag­num is struc­tured and learned how to cre­ate their own ren­der­ing ap­plic­a­tion us­ing the scene graph, the mod­el load­ing fea­tures and the in­put event sys­tem.

Al­though this yiel­ded a great start­ing point, our five-sided pro­jec­tion in­stall­a­tion re­quires that you code for a setup that con­sists of 10 nodes that render and one node that works as a mas­ter node. For that, we provide our own small lib­rary that helps to keep one ap­plic­a­tion on those 10 nodes in sync.

The ap­plic­a­tion must be built as two ex­ecut­ables — one for ren­der­ing, which runs on the render nodes, and one that runs on the mas­ter node. We have for each a dis­tinct class with a dif­fer­ent set of fea­tures. The mas­ter node works as a syn­chron­iz­a­tion serv­er. That means data is dis­trib­uted from here to each of the render nodes. This data can be track­ing data, trans­form­a­tions from in­ter­ac­tions or phys­ic sim­u­la­tion as well as all oth­er kinds of in­puts or dy­nam­ics. For the render nodes, the mas­ter node also func­tions as a syn­chron­iz­a­tion point for buf­fer swaps. This means that after all ren­der­ing is fin­ished each node sends a re­quest to the mas­ter node and waits un­til it gets a re­sponse. The mas­ter node it­self does only re­spond if all pre-de­term­ined nodes have sent a re­quest. This way we can be cer­tain that each node is ren­der­ing the same scene with the same in­put data.

The syn­chron­iz­a­tion lib­rary it­self is based on Boost.Asio which provides an ab­strac­tion lay­er to UDP and TCP net­work­ing. Based on this we fur­ther im­ple­men­ted so called synchObjects that can hold a fixed size gen­er­ic data type. Those synchObjects al­low to set data on the mas­ter node and to syn­chron­ize them with all the pre-defined render nodes us­ing udp mul­tic­ast.

In hind­sight, the biggest chal­lenge for us was to provide the stu­dents with a work­ing start­ing pack­age, com­prised of a Mag­num ex­ample that works on our in­stall­a­tions but also gives the stu­dents the pos­sib­il­ity to de­vel­op on their own com­puters, which might use Win­dows, a Linux de­riv­ate or ma­cOS. For this, it really helped that Mag­num is in Homebrew, al­lows you to use Vcp­kg or to build your own Ubuntu pack­age.

Our stu­dents could eas­ily start cre­at­ing their own pro­jects us­ing Mag­num. The sheer amount of pos­sib­il­it­ies and the size of doc­u­ment­a­tion was both an ad­vant­age and a chal­lenge. A small ex­ample for this might be get­ting the ro­ta­tion mat­rix of a trans­form­a­tion — while one might think us­ing the func­tion ro­ta­tion() as in

Matrix4 transformationMatrix;
Matrix3x3 rotMatrix = transformationMatrix.rotation();

is the ob­vi­ous choice, con­sult­ing the doc­u­ment­a­tion gives you the note that this func­tion as­serts that the trans­form­a­tion is not skewed in any way and so you might rather con­sider us­ing the func­tion ro­ta­tion­Scal­ing() as in

Matrix4 transformationMatrix;
Matrix3x3 rotMatrix = transformationMatrix.rotationScaling();

How­ever, this also teaches a valu­able les­son — read the doc­u­ment­a­tion!

What our stu­dents really liked were all the de­bug­ging func­tion­al­it­ies, es­pe­cially the De­bug class that sup­ports us­ing Debug{} << myData; for al­most any type. Fur­ther­more, hav­ing the Prim­it­ives lib­rary, the built-in Shaders as well as the GL ab­strac­tion lay­er that does most of the te­di­ous work for you al­lowed our stu­dents to quickly de­vel­op their own ap­plic­a­tions and try out dif­fer­ent ap­proaches.

Over­all, we are very ex­cited about all the dif­fer­ent pro­jects our stu­dents de­veloped with Mag­num. The pho­tos presen­ted above are from our Open Lab Day in Decem­ber 2018 where our stu­dents presen­ted their work to fam­ily, friends and the pub­lic.

1.

^ The V2C at LRZ of­fers mod­ern tech­no­lo­gies for visu­al­ising sci­entif­ic data. This al­lows for a more rap­id ad­vance­ment and sig­ni­fic­ant en­rich­ment of sci­entif­ic know­ledge. The abil­ity of sci­ent­ists to un­der­stand their data and dis­cov­er new in­ter­re­la­tions in them is vastly im­proved by the three di­men­sion­al, high-res­ol­u­tion data pro­jec­tion on the Power­wall, by the use of the five-sided pro­jec­tion in­stall­a­tion, and by the in­ter­act­ive nav­ig­a­tion provided in the V2C lead­ing to break­throughs in un­der­stand­ing and in­ter­pret­ing res­ults. In ad­di­tion to be­ing used in the nat­ur­al sci­ences and in tech­no­logy, com­plex data­sets in hu­man­it­ies and so­cial sci­ence re­search are also visu­al­ised us­ing the V2C, for in­stance in the fields of arts and mul­ti­me­dia, ar­chae­ology, and ar­chi­tec­ture. A new LED-based Power­wall ex­tends the V2C cap­ab­il­it­ies with the latest tech­no­logy.