I've been affected by the VR flu that's been going around lately. No, I think it's the future. It's going to be my future. And it's going to be big. But for now I want to publish an experiment related to that. I've been extending
that raycasting demo to work with cardboard (a cheap way of doing VR using your smartphone). It's been interesting to experiment with that.
For the candy just go to
canvasvr.qfox.nl.
Besides the base raycaster demo there's some toys you can play with. On a global scale I don't think I can make this render feasibly within the expected parameters of VR. That is to say, I don't think this will be playing at 90 or even 60 fps any time soon, let alone the fact that the (mobile) browsers won't let me get past 30. But in reality, including distortions this demo runs at about 20 fps on currently high end phones (Galaxy Note 4) which really is fine to test stuff with but only to a certain degree. It's certainly not going to do you any good for serious stuff, even if serious just means casual gaming.
That said, the demo is packed with tweakable values and is a nice and lightweight way of getting closer with the involved math.
As usual from me, this demo doesn't use any libraries. All the math, all the stuff is raw JS. I've had some opportunity to play around with the barrel distortion (
blog post), the raycasting in general (
blog post), the gamepad api (no blogpost), and some VR parameters.
Dual rendering
Every frame is rendered twice with slightly different viewing positions and angle. These parameters are adjustable on the fly. Frankly I this demo doesn't appear to be suited to properly test certain parameters. At least I can't really say anything is good beyond simply looking straight and applying a few pixels distance between the cameras. Looking into the same direction seems to give the best result.
The cameras have their own canvas. It will first render the scene on them for both eyes. Then it generates the static stuff (minimap, menu, debug, messages) once and paint that in the same position on each canvas. This seems to work pretty well, or at least the effect works. Stuff is static and crisp.
Gamepad API
The demo has tentative joypad support. Be warned that I've only tested this with the samsung gearvr joypad on the note 4. For one it only has one LR button. And I'm sure some mapping will get wonked on other gamepads. Sorry about that. Another thing is that it uses polling in general and I have no idea how that'll work out for iOS, for example.
It was the first time for me working with the gamepad api but it's almost a must. Looking around is nice but to get any interaction going you really need some kind of additional input device. Look-and-wait approaches only get you thus far.
In this demo I've mapped the input from gamepad and keyboard. I like how I handled that, though as I said the gamepad needs some more testing. In the end the
Input singleton should tie the various input singletons together. The demo only contains a good start of what that should look like.
Gamepad and menu
As far as controls go, right now, the arrow pad move you forward backwards and side ways. The analogue stick, well both of them actually, will change your heading left-right. That's where you're looking and it will display "nose". One analogue stick up-down will adjust the nose distance, the other one up-down will change the angle of the camera's (inverted adjustment to each), this will display "focal point".
The select key will reload the page (super handy while developing!). The start button toggles the menu at it's current point. The R and L buttons (one of them, anyways), will cycle through menu items.
The four buttons are either "ABXY" or "1234". "A", the bottom one, will toggle the gamepad debug window, which shows which joypad buttons are detected. If that panel is not green, no gamepad is currently detected at all.
The "B" button (right one) disables the orientation API from updating the heading. Especially in the browser this API can be a bit erratic and disabling helps while debugging certain specific angles.
The "X" button (the left one) toggles source textures. There are three textures, each with or without the word "cat" written on them for orientation purposes. There's a wolfenstein (4 wall) texture, a random color texture, and a wireframe texture. Pressing X cycles through them.
The "Y" button toggles the minimap. Simple as that.
There are five menu items. Movement is disabled while the menu is open. Input does something specific to the item that is selected.
FOV simply changes the field of view up or down with the left-right arrows. You can also toggle barrel distortion with up or down, provided your browser supports webgl at all.
Cover displays a green cover over the canvas. The up and down arrows toggle whether the circle is shown or its hole. The left and right arrows increase or decrease the size. You can use this to get a feeling of exactly how much you are actually seeing of your screen. When looking at the cut-out, increase the size of the cover until you don't see any green anymore. Then take the phone out of your holder and see how much of the screen you can actually see. It's an interesting experiment for sure.
Grid allows you to paint a grid in four different modes. Up-down toggle the mode between rectangle-distorted, rectangle-straight, rectangle-mixed, circle-distorted, circle-straight, and circle-mixed. Left and right will increase or decrease the grid. Basically you can see a rectangle or circle grid. And you can choose whether the barrel distortion should be applied to it. As matter of experiment you can also turn off the barrel distortion completely for one camera to compare the difference. You can also toggle painting the rays as a trapezoid with up or down. Trapezoid (sideways) is proper but much much slower as it will paint each ray one vertical pixel strip at a time, rather than the entire ray at once. There's no other viable way of doing this with 2d canvas currently.
HMD has (currently) two presets: full and wood. I've got a wooden "cardboard" and these are the camera size parameters for that one. I've made it default if >1000px because on large screens the fps just tanks. Doesn't really matter. Press up or down to cycle through the list of two items.
View has the most interaction and allows you to play around with the camera in various ways. Left-right changes the camera width, up-down the camera height.
(I'm aware of the flipping of the left canvas while you press left-right, but couldn't be bothered to figure out what was going on there.) The (on my pad) right analogue stick allows adjusting the barrel distortion strength and the distance between the camera's. Left-right changes the strength (
0.035 seems to work best for me, but feels quite low) and up-down change the distance between the camera's evenly. Use that to adjust the position if your holder needs it.
The other analogue stick allows positioning of the canvas, it basically changes the offsets of both cams at the same time.
Info just displays some parameters you could note down. Maybe useful to create configurations for specific holders (cardboard vs gearvr, etc).
These settings have an associated keyboard button as well, check the input.js file for details.
Barrel distortion
Since you're looking through a lens you kind of need some barrel distortion effect to counter what the lenses are doing to your screen. This is called the barrel distortion. I found
this youtube video explained it very well so go watch that for details.
As for my implementation, there are some caveats. One it requires webgl. Only more recent devices will support that. For two it adds a considerable latency since it has to paint the canvas after the scene has been been rendered, distort it, and render it back. Quite inefficient and slow, but it'd be even slower to do the distortion in 2d canvas.
It will automatically be disabled if it can't get a webgl context. You can toggle it in the FOV menu item by pressing up or down.
I don't really like my implementation either. Sure it works, but I think it's too strong in the middle and too weak on the edges and I wish I knew how to fix that. I will, eventually, but for now I'll have to live with it.
It's hard to say what parameter works best. For now it seems to be best at
1.035, but that seems quite low for me. Any stronger and the center gets over-distorted, so that sucks. What's worse is that disabling the effect doesn't really seem to be a problem for the experience. I'll need to dig deeper into the why for that.
Raycasting
One of the bigger problems with the whole demo is the imperfectness of the raycasting. I've been a bit sloppy which has led to floating point errors. Most notable are the fact that rays on the right side don't properly clip, which leads to artifacts and blinking. This causes movement to be very "busy". Rounding errors and anti-aliasing are causing skewed walls to be very trippy while moving. I'm certain this reduces the whole immersive part of it. Then again, it's not like a realistic environment anyways. Note that the wireframe texture mode exposes the ray rounding problem quite clearly; the right line of each wall will flicker on and off because it's not properly handled. Could be done, don't want to be bothered with that right now since it's just an experiment and won't lead to an actual product.
Fin
What else. I don't know. You can find the demo at
canvasvr.qfox.nl. I'll add it to my project page later. Noticed recently that this page is lagging about 2 years behind anyways :p
It's been fun making this demo. I got a better feeling of the math involved, on some do's and don'ts of VR, and opened up a Pandora's box of new questions regarding it. For now I just want to put this demo behind me and move on to something more serious. It's either webgl or maybe even non-web.
At this point I'm starting to realize that, if I'm to do anything serious in this space, I may not be able to stick to the browser. The vendors are simply not putting enough effort into it. Afaik, at the time of writing, there's ONE guy for google and ONE guy for mozilla working on WebVR. Their efforts are great, but they just don't have the resources to pull this by themselves, obviously. Besides that, webvr is more about hardware support like the Oculus, and less about cardboard.
I think the cardboard approach (a phone holder) is super strong. Pop the phone in your pocket into a 10$ carry case for VR. It's not going to be as super as the Oculus or GearVR, but it'll suffice for the impression.
I think I'm going to try writing a small WebGL demo next and watch perf. See if I can get it to a satisfying level.