Mobile Performance & Quirks

Brace yourselves, this is going to be quite a ride.

Thursday I added a bit of code that lets the character move gradually towards a point that you click on the screen.
So far, I was moving the character with the arrow keys, which meant I couldn’t test much in mobile. I have been loading the page and watching it render, just to make sure it basically worked, and eyeing the FPS meter, but I couldn’t make it move.

Now that we can finally move on mobiles, I started experimenting with them a bit more…
And I found a good number of surprises:

TL;DR: Cell phones are awesome. And… they SUCK ASS. Read on, the details will be useful if you’re doing something like this


1) iPhone does not bubble up the click event on non-link elements.

Basically, tapping the canvas in iPhone didn’t do anything (I’m listening for clicks, on all other platforms, with window.addEventListener(‘click’))
I should’ve known about this, I read it in PPK’s blog, but that was a long time ago, I obviously didn’t remember.

PPK’s workaround works, adding an onclick to the canvas. I could also listen to clicks on the canvas itself instead of on the window.
The problem with that is that when I tap, the whole canvas goes a bit dark and then ligther again, “acknowledging” my click. That pretty much sucks ass.
On android, it’s worse, the whole thing is painted green. Green! Yuck. That’s easy though, I can resort to UA-sniffing and only do it for iPhone, but still not cool.

Fortunately one of the commenters in PPK’s blog figured out that it’s not that events don’t bubble; they just don’t bubble all the way up to body; but if you wrap the canvas in a div, and listen for clicks in it, it works, and nothing flickers. Works on Android and the desktop browsers too. Solved! Thank you PPK!


2) GetImageData performance is horrendous

Remember our cool lighting? It uses GetImageData and loops through all the pixels to make the sprites darker.

function DarkenCanvas(baseImage, ratio, toColor) {
	var tmpCanvas = document.createElement("canvas");
	tmpCanvas.width = baseImage.width;
	tmpCanvas.height = baseImage.height;
	var ctx = tmpCanvas.getContext("2d");
	ctx.drawImage(baseImage, 0, 0);
 
	var pixelData = ctx.getImageData(0, 0, tmpCanvas.width, tmpCanvas.height);
	for (var i = 0; i < pixelData.data.length; i+= 4) {
		pixelData.data[i] = toColor[0] + (pixelData.data[i] - toColor[0]) * ratio;
		pixelData.data[i + 1] = toColor[1] + (pixelData.data[i + 1] - toColor[1]) * ratio;
		pixelData.data[i + 2] = toColor[2] + (pixelData.data[i + 2] - toColor[2]) * ratio;
	}
 
	ctx.putImageData(pixelData, 0, 0);
	return tmpCanvas
}

And in my Android HTC Desire S, it took 3 minutes to load the page. 3 fucking minutes!!!

Turns out, GetImageData doesn’t work too well on mobile. It’s not really GetImageData itself though, it’s the looping through the pixels that takes forever.
For comparison, my desktop browsers take 6ms, iPhone takes about 500ms, and Android takes about 4 seconds for each image.
5 sprites layers, 8 light levels…. You do the math.

Of course I didn’t expect this to be as fast as my PC, but considering how relatively fast everything else runs, I did not expect a 500x slowdown here…

One obvious solution here is to trade CPU for bandwidth. I can simply have the files pre-darkened in the server and load them already processed.
However, I’d rather not waste that bandwidth, and more than anything, processing on the client gives me HUGE flexibility. I can change the number of “lighting levels” by simply changing a constant, and the image files generate themselves.

On this one, I just gave up and asked.
It turns out that there *was* a simpler way, as I suspected, I just completely misunderstood how canvas compositing works.

Simply drawing a black semi-transparent rect with compositing mode “source-atop” does the whole trick.

I don’t know how fast or slow that is. The page went back to loading instantly, so I don’t care anymore (I know, I’d make a really shitty scientist)
Thank you Stack Overflow!


3) Viewports…

I saved the best for last… “mobile Webkit” is awesome for displaying webpages that are meant for bigger screens…
But to try and have a full-screen game, it’s a steaming pile of shit. Particularly on iPhone, where I spent 99% of my time trying to find a decent solution until I gave up and hard-coded the hell out of it. Piece of crap phone.

Let’s start from the beginning…
From the very first version of the engine, I added these meta tags:

<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no, target-densitydpi=device-dpi" />
<meta name="apple-mobile-web-app-capable" content="yes" />
<meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />

These are pretty much all I could find regarding making a cell phone viewport behave “the proper way” (ie. without any JS weird hacking).

I’m also setting, onLoad and onResize, the main canvas’s width and height to window.innerWidth/Height. That’s basically all you need to do for desktop browsers (and for the crappy experience i’m about to describe on cell phones which, technically, does work):

With those settings, this is what I’m seeing on my phones:

  • iPhone 4: portrait: reports 320×356. landscape: reports 480×208.
    In both cases: Wastes about 120px vertically with HUGE blue bars at the top and bottom. Sprites look very low-res (very shitty).
  • Android: portrait: reports 480×720. landscape: reports 800×400.
    In both cases: Wastes about 80px with a black bar at the top. Sprites look fucking awesome.
  • Android switching from portrait to landscape: reports 533×267. Wastes about 80px with a black bar at the top. Sprites look very low-res. It’s basically zooming in to about 150% for some reason. Also, it’s blitting about half the number of times per frame as in portrait, but frame rate is only 30% higher. Something’s wrong, it’s going too slow. I can pinch-zoom-out, and everything’s awesome again. Why the hell is it doing this?

So, basically, both phones are wasting half my screen, the iPhone is throwing away its beautiful Retina display, and Android is auto-zooming in when changing the orientation, plus it lets me zoom in even though I told it not to, and it zooms in on double click.
The default behavior is worse than shit, frankly.

So, let’s take these one by one, starting on the easier one…


Android’s zooming

With a bit of googling, it turns out this is an HTC Android issue, not Android in general. Apparently, non-HTC Androids do respect the “no zooming” directive. I didn’t really find a workaround, so what I’m doing is letting the user know that he’s zoomed in and he should zoom out. I still have the problem of zoom on double-tap, and I don’t know, yet, how much of a problem that’ll be. I’ll revisit this later if it becomes an issue. Bottom line, games will have to manage being zoomed in, most likely, and adapt correspondingly. If you’re doing a game where the user can zoom in, you just got that for free. You do need to adjust the number of pixels on the canvas, though, to not lose resolution (more on this below). It’s not pretty, but it sounds doable.

What I found, that is very useful, is how to detect this, and also how to figure out the ratios to correct for it. This is the code I’m using for this:

if (Window.deviceType == "android" && window.innerWidth != window.outerWidth) {
	// just show the "please zoom out sign"
	// but ratio between outerWidth and innerWidth, plus scrolling properties can let you correct.
}

Android’s top bar wastes screen

For the top bar that wastes screen, what we need to do is just scroll down. This implies making our page a bit taller than the window. “A bit” being exactly that bar’s height, or it’ll either stay partially visible, or some of our content will be off-screen.

I found that in android, this top bar’s height is the difference between window.outerHeight and window.innerHeight. This is simple, then:

Window.chromeVerticalSpace = 0; // All desktop browsers
if (Window.deviceType == "android") { Window.chromeVerticalSpace = window.outerHeight - window.innerHeight; }
elCanvas.width = window.innerWidth;
elCanvas.height = (window.innerHeight + Window.chromeVerticalSpace);

I’m also scrolling to the bottom of the page onLoad, to hide the top bar.

window.scrollTo(0, Window.chromeVerticalSpace);

Not exactly pretty, but gets the job done. I really want to see what happens with non-HTC androids, though, this may not work there…


iPhone’s wasted screen space

This is where the fun begins…

First I tried to find some other magical way of finding out how big the top bar is. I couldn’t.

So, I decided to hard-code it to 60px (trial and error gave me this number), and use the same solution as in Android.

This looked OK at first, but on closer inspection, particularly when changing orientation, it started to break in weird ways that I found no way to compensate for… Basically, if the page loaded on portrait, or on landscape, it’d show perfectly. However, as soon as I rotated the phone, it’d either be too tall in landspace, or too short in portrait (just *exactly* short in portrait, in other words, it took the exact height it needed to show the top bar at all times, with no scrolling possible).

This is *very* puzzling to me. Basically, the phone will give you some screen sizes for portrait/landscape, but when changing the orientation it seemed to give different ones. I tried setting the canvas to 1px by 1px to see if the canvas being there was affecting the window size, but that didn’t solve it. It does have something to do with it, if my canvas is 250×250 (and thus it fits always), I consistently get always the same screen and window sizes. But when I’m actually trying to maximize my canvas, it goes batshit crazy.

I don’t have the exact details at this point because I spent literally 4 hours trying crazy shit and rotating my phone, and by the time I was done I was ready to kill the moron responsible for this, so the details are a bit fuzzy now…

In the end, it left me with no choice. I just hard-coded the canvas size. This is the absolute shittiest solution in the planet if you ever want to do anything correctly, but I just lost it. Although there is a lot of very good information out there on the topic of mobile viewports (most of it thanks to PPK), I couldn’t find anything on having an element take up the whole screen consistently. I’m probably missing some really obvious option, but I just had had enough, and hard-coding an iPhone is “safe enough”, since there’s only one iPhone screen resolution (well, 2 really), and that’s not likely to change.

Fucking piece of shit phone.

var size = [320, 416]; // Default portrait
if (window.innerWidth > window.innerHeight) { size = [480, 270]; } // Default landspace

iPhone Homepage

Of course, if they bookmark your page in the homepage, then both the top and bottom bar disappear (due to the meta viewport), and you need to contemplate that as part of the hard-coding. Fortunately, there is a pretty property, if you know you’re on an iPhone:

if (window.navigator.standalone) {
	if (window.innerWidth < window.innerHeight) { size = [320, 480]; } // Fullscreen portrait
	else { size = [480, 320]; } // Fullscreen landscape
}

There’s also a bunch of nice things you can do to make the “homepage” experience better (better icon, splash screen, etc, etc)

The big problem with the homepage is that it runs ridiculously slow, because of the whole “no Nitro if you’re outside the sandbox” issue.
There’s pretty much fuck-all you can do about that, especially since even using things like PhoneGap you’ll have the same problem. This is very, very, very bad. Nothing to add there, though.


Retina display

Finally, the iPhone 4 lies to you about the screen size (backward compatibility, blah blah), and tells you it’s NOT a retina display. And there’s no way to coerce it into having the right dimensions/proportion. HOWEVER, you can exploit the fact that a canvas “size” and its element’s size don’t need to be the same. In other words, you can have more pixels in the canvas than the DOM element takes, and the contents get basically stretched. Fortunately, the iPhone reacts beautifully when you do this, by using all its nice, tiny pixels. Also fortunately, we at least have window.devicePixelRatio to detect Retina…

elCanvas.width = size[0];
elCanvas.height = size[1];
elCanvas.style.width = size[0] + 'px';
elCanvas.style.height = size[1] + 'px';
 
if (window.devicePixelRatio > 1) {
	elCanvas.width = size[0] * 2;
	elCanvas.height = size[1] * 2;
	Viewport.screenScalingFactor = 2; // Notify the viewport that screen coordinates are wrong
}

That final line is to let my “screen coords to map coords” method know that all the coordinates it’ll get are wrong and it has to correct accordingly.


So that’s it for today on mobile…
I can’t wait to get my hands on an iPad and see how bad it is…
Oh, and I need a non-HTC android to see how many of the things I call “Android” are just HTC specific.

The other big thing on phones is, obviously, performance. It’s horrendously slow right now (although faster than I would’ve expected before starting this whole project), but I have plans for that too.

One comment

Leave a Reply

Your email address will not be published. Required fields are marked *

*

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre lang="" line="" escaped="" highlight="">