~ 4 Minute Read.

This is the third post in a se­ries about the vir­tu­al re­al­i­ty es­cape room project I did to­geth­er with Daniel Bo­gen­rieder and Rox­anne Low at the Uni­ver­si­ty of Kon­stanz.

Aug­ment­ed Vir­tu­al Re­al­i­ty

I’ll be frank: cur­rent gen­er­a­tion aug­ment­ed re­al­i­ty head­sets are far from our dream of what aug­ment­ed re­al­i­ty should look like. Es­pe­cial­ly sta­bil­i­ty of the aug­ment­ed im­age is a huge prob­lem, ev­ery­thing is swim­ming around, not ac­tu­al­ly root­ed where it should be.

Hence, it is way less fun than it could be. Since our es­cape room was in vir­tu­al re­al­i­ty with the HTC Vive Pro and we matched the re­al room 1:1, ba­si­cal­ly try­ing to repli­cate re­al­i­ty, we were able to cre­ate per­fect aug­ment­ed re­al­i­ty, though.

Arrow hologram animation

The above gif shows one of our holo­grams that would get ac­ti­vat­ed once you put on a vir­tu­al aug­ment­ed re­al­i­ty head­set to en­ter aug­ment­ed vir­tu­al re­al­i­ty 1. The an­i­ma­tion played there is just a uv tex­ture off­set, no shad­er mag­ic in the an­i­ma­tion. To be able to have dif­fer­ent col­ors with sim­i­lar pat­terns on them, the in­put of the shad­er is a mask, though, so that you can spec­i­fy the two col­ors that should be ren­dered in the white/black pix­els of the mask. We didn’t ac­tu­al­ly use this fea­ture of the shad­er in the end though.

Crispy VR Text in Uni­ty

For the long­est time peo­ple have been com­plain­ing about how un­read­i­ble text is in vir­tu­al re­al­i­ty. Even with many counter ex­am­ples to prove it (We even made a Speedread­ing Game at Vhite Rab­bit!), peo­ple still ac­cept this as a fact im­ply­ing the rule of “avoid text in VR”.

In my op­pin­ion it is a mat­ter of how the text is ren­dered. Of course tiny or un­der­sam­pled text is un­read­i­ble. How about mak­ing it big­ger or over­sam­pling it in­stead then? Oth­er op­tions are mesh fonts, as they are an­ti-aliased by MSAA—they are not wide­ly avail­able, though.

In Uni­ty it is a mat­ter of turn­ing up the font size and scal­ing down the char­ac­ters again. This will cre­ate a high­er res­o­lu­tion font tex­ture, hence crispy crispy text!

Comparison of font texture resolutions.

To en­able this you need to ad­just a set­tings in two pan­els: the font set­tings and the text mesh set­tings.

Out­side Ren­der­ing

As the room we were repli­cat­ing in VR had win­dows, we had two op­tions: ren­der the out­sides of the win­dow or put down the blinds.

Ob­vi­ous­ly the lat­er is cheap and we were go­ing to this prop­er­ly… anec­dote on the side: in our fi­nal pre­sen­ta­tion the blinds were ac­tu­al­ly down in the re­al room, so it would have matched the re­al room more ac­cu­rate­ly to not have done so.

As noone was will­ing to mod­el the for­est and build­ings that were to be seen out­side, we in­stead took a set of two rough 180° im­ages with a smart­phone. Then one of these would each sur­round a set of win­dows as seen in the fol­low­ing im­age:

Hemisphere's for the outsides of the room.

One prob­lem emerges from this: the hemi­spheres need to be big enough for you to per­ceive them at in­fi­nite dis­tance (where stereo vi­sion turns in­to monoc­u­lar), but then they over­lap!

A cus­tom shad­er solved this for us: a ray cast from the shad­ed frag­ments world po­si­tion through the plane of the win­dowed wall in di­rec­tion of the view lo­ca­tion would give us the in­ster­sec­tion. If the in­ter­sec­tion point’s X co­or­di­nate is left to the cen­ter of the wall, the point would be shown for the left hemi­sphere and dis­card­ed oth­er­wise. Anal­o­gous for the right hem­ishere.

In prac­tice this looks like this:

Dual hemisphere shader for rendering the outsides of the virtual reality escape room.

Up Next

To­mor­row I will go in­to the dif­fer­ent in­ter­ac­tion tech­niques and hard­ware we used for this project.

Wow, that was fun to write. :P

Writ­ten in 30 min­utes, gifs in 120 min­utes, ed­it­ed in 15 min­utes.