Tuesday, June 10, 2008

iPhone Background Processing: Not Fixed But Halfway There

Yesterday, unless you are on another planet, you know that at Apple’s annual World Wide Developers Conference (WWDC), Apple announced a new iPhone, and showed off a bunch of cool new applications. They also made a few new announcements about their software development kit (SDK). For me the most important of these was around a new feature for app developers relating to background processing.

We asked Apple for background processes

Several months ago I wrote a fairly popular article on why the iPhone needed background processes. The gist of my argument in that piece is that background processes are needed for three things. To allow applications:
  1. to be notified about things happening in the outside world, such as notifications of inbound instant messages or email.
  2. to notify the user about things happening locally. An example might be an alarm clock.
  3. to broadcast information about the users current situation to the outside world such as his/her location or presence.
In the article, I asked developers to speak out. “if you agree, I strongly suggest you say something. Link to this piece or restate the arguments elsewhere.” and I ended with the somewhat tongue in cheek “Please, Mr. Jobs, tear down that wall.”

They heard us… kinda

And so, it appears that, at least to some extent, the rallying cry worked. Apple announced a new mechanism for receiving background notifications. We (meaning all of us, not the “royal we”) did make a difference.

Specifically, since the solution will not be available until September, I must assume that this feature is a late addition in response to the noise over this issue since March when Apple initially announced the SDK. To see Apple responding directly and quickly to market requests is indeed refreshing.

Unfortunately, Apple’s response does not go far enough, though this first stab at the problem is compelling. Apple has fully solved the first problem in the above list, which is being notified about external events, including things like inbound messages.

The way it works is that application developers will be able to send messages from their web servers to Apple servers. In turn, Apple’s web servers can directly push notifications to iPhones. Apple’s stated design goal here was to skirt the need for developer accessible background processes by implementing their own single background notification process that all developers can effectively share.

This is indeed a great solution for notification. Unfortunately it does not eliminate the need for additional developer accessible background services.

We still need other background services

Apple’s new solution resolves the first of the three above points. You can now build an instant messaging client for the iPhone. Unfortunately, you still can’t do things like:
  1. build an alarm clock.
  2. write an app to broadcast your mobile location, unless you stay exclusively in that application. So no taking phone calls and no letting your phone go to sleep.
  3. write code that *responds* to notifications. All the new notifications do is send a message to the user. They can’t trigger code execution.

As I explained in the last article, these are important functionalities for really delivering on the full potential of the phone as a developer platform. I am particularly excited about the idea of my mobile phone being able to detect information about the world around me, and notifying me of relevant things in my environment. With the new GPS functionality I’d also like to be able to track my whereabouts. Where did I go today? How long did I stay? I believe this kind of almost “agent-like” functionality is a hugely compelling application class that is not being enabled by the current application model, but should be.

Mobile Me reminds us of a fourth reason background functionality is key

Interestingly, beyond the application categories I had considered in my last article on this subject, Apple’s introduction of Mobile Me is a vivid additional demonstration of the need for background processing. Mobile Me is a cloud based system that among other things, allows contacts, email, and calendar information to be synced to your iPhone in the background.

Mobile Me looks great, but I wonder why I as an application developer am not being given the tools to write such an application. Of course I have not seen how it actually works, but presuming that it really does sync in the background, I am left wanting to ask Mr. Jobs why I can’t do that too.

Apple’s “background apps take too much power” argument

In Apple’s defense, their argument is that background applications take too much power and add too much complexity. Yesterday they held up Windows Mobile as the straw man for why you don’t want background processing.

But of course we all know that because something has already been done a bad way doesn’t mean there isn’t a good way to do it. So I accept the argument that in considering these options, complexity and power consumption are critical considerations. But I reject the notion that they cannot be addressed.

The solution

As I see it, the solution to this problem is a special *very* slow task manager specifically for background tasks. The idea is that applications can register to run background tasks on some periodic but very infrequent basis. They would be allowed to execute a very small number of instructions, and they would be set to run at most every few minutes. In this way, it would be possible to do some very quick processing at very long intervals. By making the background processing very lightweight, no Windows style task management would be needed, and worries about battery power would be essentially eliminated.

Additional functionality could be tied into the GPS and the new notification system to allow background code to be triggered to do specific things when, for example, the user’s location changes, or some external event happens. This would allow for applications to do things like respond in the background to user movement.

The bottom line is that I do like the idea of specialized services to handle background needs in ways that are battery friendly. Hopefully the new notification system is the first in a series of such services that can truly take mobile platforms to the next level.

Related:
The Awkward Marriage Of Google And Apple in Mobile
We Have The iPhone Because Steve Jobs Has Big Brass Balls
Apple Says: We Weren't Lying, AIM *will* Be A Turd
The Anti-iPhone: Google's Android
Apple's iPhone SDK Inhibits Real Mobile Innovation
The iPhone Doesn't Suck - duh

24 comments:

Anonymous said...

What you are describing is cool, but I think the push notification service fulfils the needs of the vast majority.

And I know it was just an example, but there is already an alarm clock on the iPhone, lol

Adam MacBeth said...

The reason you can't write a background sync service is that Apple wants to push people towards MobileMe. If there's no alternative, then won't everyone just cough up the money for it?

http://adamac.blogspot.com/2008/04/iphone-sdk-is-brew-part-deux.html

StCredZero said...

1) Why do we need an alarm clock? Answer: this doesn't seem like a very pressing need.

2) Why shouldn't location broadcast be rolled into the iPhone's location APIs? Answer: The location API is actually the logical place for this. Everyone rolling their own would be a waste!

3) I agree that devs should get a way to register callbacks against notifications. But this need not be in the form of a full "background process." What do you have in mind that wouldn't best be put on a server or happen when the user returns to the app?

Anonymous said...

No, Hank, you didn't "ask" Apple for background processing. You violated your NDA and cheerfully leaked confidential information in an attempt to force Apple to cave in. There's a difference.

You also proclaimed that "Apple's SDK prohibits mobile innovation" and was only good for games.

Well, guess what? Lots of professional programmers (most of whom did not violate their NDAs) have successfully written innovative non-game mobile applications.

Carry on whining, Hank. :-)

Hank Williams said...

Anonymous,

No I do not have an NDA with Apple since I am not currently writing for the platform and have not downloaded the SDK. My analysis comes from public statements made by Apple, which even if I did have an NDA would have been fair game to comment on.

Additionally, you seem to be having a hard time reading complete sentences. I did not say it would only be useful for games. What I said was "Apple’s new platform will make it possible to create lots of great games, and really pretty more functional user interfaces."

I stand by that statement. In the next sentence I suggest that the platform does not shift the paradigm of mobile applications. And I also believe that is the case. As an example, most of the serious apps are either ports from other phones, or could be done on other platforms. That would, by definition, mean they are not shifting the application paradigm, though they are making it radically easier to make cool apps.

In any case since you seem to be seeing what you want to see I suggest you carry on hallucinating :-)

parveenkaler.com said...

The alarm clock would need to run on the server and trigger a notification on the iPhone.

There is a very slow task manager that can execute code: Me.

Your service that is sitting on your server would push a notification. A dialog pops up on the iPhone. The user touches a button on the dialog box and task switches to your application.

Having shipped games on the Sony PSP, what you are proposing is a very, very bad idea. Background processes are a bad idea for memory-constrained devices that need 100% uptime.

The iPhone does not page out to disk and it has a limited amount of RAM. These resources should not be handed to background processes.

Note that the iPhone notification system works just like the notification system for the XBox 360.

Anonymous said...

Actually, they did say that a push notification could invoke a dialog and that pressing a button in that dialog could start up your app with fresh information, so in fact they *can* trigger code execution.

patpi said...

"Of course I have not seen how it actually works, but presuming that it really does sync in the background, I am left wanting to ask Mr. Jobs why I can’t do that too."

Because AT&T don't wont to give you possibility of building Skype-like application :)

Anonymous said...

No, Hank, I don't have any trouble reading complete sentences. If you have to resort to childish taunts, I guess you don't have anything resembling a valid argument.

You wrote, "Apple’s iPhone SDK Prohibits Real Mobile Innovation." What part of that sentence don't you understand?

"In the next sentence I suggest that the platform does not shift the paradigm of mobile applications. And I also believe that is the case. As an example, most of the serious apps are either ports from other phones, or could be done on other platforms."

Yes, you obviously believe that. The title of your blog says it all -- "everything sucks."

Programmers who are actually working on the iPhone say otherwise.

Whether you recognize it or not, you're insulting thousands of programmers who are developing what they consider to be innovative applications even though you think they're crap.

You admit you haven't even downloaded the iPhone SDK, but we're supposed to dismiss the opinion of everyone who's actually working on the platform because you disagree with them?

I don't think so.

Hank Williams said...

"Whether you recognize it or not, you're insulting thousands of programmers who are developing what they consider to be innovative applications even though you think they're crap."

uhh... wow. I really do think apple fanboyism stunts reading comprehension.

No I do not think all iPhone apps are crap. They are generally quite cool. They simply, in my view, are not paradigm shifting. Sorry if that concept is confusing to you. But of course *anything* critical of apple or the apple universe is, to you, the equivalent of treason. Unfortunately if you read all the related links you will see that Apple gets at least as much (probably more) praise than criticism. So the whole whiny "your insulting all of us" drivel just doesn't fly.

Anonymous said...

"They simply, in my view, are not paradigm shifting."

Obviously. I'd say that medical imaging program shown the other day was far more "paradigm shifting" than Yet Another Instant Messaging client, but of course you disagree.

In your view it seems that "everything sucks" and nothing is paradigm shifting except the program you want to write. Except you're not even an iPhone programmer.

So, everyone developing for the iPhone is a "fanboy" and the only person whose opinions count is the guy who hasn't even looked at the SDK?

What a sucky attitude.

Now on iTunes: Hank William sings "Apple won't give me background processes and my dog got run over by a train." :-)

Hank Williams said...

"more "paradigm shifting" than Yet Another Instant Messaging client, but of course you disagree."

Who said anything about an instant messaging client being paradigm shifting?

"In your view it seems that "everything sucks" and nothing is paradigm shifting except the program you want to write."

I guess you haven't read much here, including my coverage of the iPhone.

"Except you're not even an iPhone programmer."

And that is relevant to whether the iPhone needs background processing how?

"So, everyone developing for the iPhone is a "fanboy" and the only person whose opinions count is the guy who hasn't even looked at the SDK?"

Nope. I havent said anything about "everyone developing for the iPhone". I have just said *you* are a fanboy, and not a very effective one at that. And I am not sure where in this blog it talks about whose opinion does or does not count. And even if I did, I am not sure why you would care. It sounds like you have some serious persecution complex going on here dude. I think you should seek professional help.

Anonymous said...

>"Except you're not even an iPhone programmer."

"And that is relevant to whether the iPhone needs background processing how?"

You haven't even downloaded the SDK, but you're demanding that Apple rearchitect the SDK to make it easier for you to write apps that you aren't even working on, and you can't understand how that is relevant?

You throw a shitfit about how big bad Apple is "prohibiting" you from creating innovative mobile applications, but in reality it isn't Apple that's stopping you from writing iPhone apps. It's Hank Williams.

If you looked at the SDK and read the documentation instead of throwing feces, you would have some idea how the software works. You could offer informed opinions instead of calling anyone who disagrees with you names.

Your "fairly popular article" proposed architectures that wouldn't work even if you had background processes. You would know that if you took the time to read the docs instead of throwing feces.

The title of your blog says it all. Not "everything sucks," Hank. What sucks is your attitude.

Anonymous said...

I just got out of the WWDC08 and there appears to be no one realizing the value of background process. Lots of fear that something will go wrong.

At any rate, Apple must get this background registration fixed so we can give end-users what they need for a connected social network.

mrb

silpol said...

talking about background application, I am afraid Apple doesn't tell what they should tell, i.e. the root of problems is not in power efficiency - it is somewhere else... I can speculate a lot, whether it is evil Steve's mind (to push everything on their servers and milk stupid Apple funs as dog tears newspaper) or just not so great as their were painted Apple engineers...

The fact is that during course of development in Nokia tablets, it has been learned that on ARM-based platform (and yes, Steve runs almost same HW) CPU and processes does not constitute main power hog - main evil is in screen (not even screen but amount of light necessary to make it minimally visible) and radio (bluetooth, wifi, phone engine in case of iphone), and Linux as platform is full of bg processes, so... Yes, Apple just uses background processes *prohibition* as lame excuse for something else, what they don't want to say. No wander, Apple fun will eat whatever is fed from Steve's hands ;) Don't be morons, don't take Steve's words for granted.

Jon said...

Interesting points, though I don't know if background processes are the answer. We need to find a way to enable these types of things without enabling viruses as well.

BTW, it is possible to build an alarm clock using the current system. When the user sets an alarm, you send a message off to your server... which then pushes a notification at the appropriate time.

You are correct that it is not possible to have it broadcast my current location at all times... though I don't know if that is something I would want. Matter of personal preference... though I would want to be notified if an app was doing that so I could remove it.

M said...

The very first application I was excited to develop when the iPhone SDK was announced is one which I believe would need to run in the background. A similar application was alluded to by Hank.

I want my iPhone to monitor my location and alert me when I am close to a point of interest. There are MANY reasons this would be useful and I don't see a way this can effectively be done without background processing. I would be fine to have my application sleep for 30 sec and then run a tiny bit of code to compare my location with points of interest and go back to sleep if none were near. This would limit the battery consumption, and I have no issue with strict limits on the energy used when in background mode. I would need very little.

One of my big issues with my WM 5 and 6 phones is that the only way to actually turn off applications is with 3rd party apps. When I’m done with a program and close it I want it to turn off not just go to the background. I understand that poorly done background applications can cause issues but there has to be a happy medium.

I believe the only reason for the missing functionality is that Apple wishes to have an edge available only to their, and their partners’, products.

Dave said...

It's ridiculous that people fail to grasp that, while possible to write an alarm clock by sending messages through a server, an alarm clock shouldn't have to be tied to any server at all!

Or an RSS reader that updates its feeds automatically every half hour (and don't say, "but the application developer's servers can send a notification to every user every 30 minutes to update their feeds")

Or a calendaring app.

Or the aforementioned location updater that could turn off your ringer automatically when you get to work or whatever.

Luckily, jailbreaking will continue until Apple pulls their head out (on this issue... the iPhone does rock, overall).

Anonymous said...

My main problem with this is the fact that this has pretty big security implications. If you want (for instance) an IM client you'll have to give out your usernames/passwords to the developers of that app because it's their servers that need to maintain connections to a Jabber server, AIM, MSNM, etc. I really don't want to do that. I also really don't want all my instant messages to go through the server of some third party developer. It's already bad enough that those messages get routed through Microsoft/AOL.

Anonymous said...

Better hope that your Hotel room has good network connectivity because you're alarm won't ring if there's no network coverage to send that push message from Apple's servers. What otherwise is a simple use-case that is completely local to a device turns into something that will also require a server infrastructure to support. I have no problems with having the user prompted before allowing an app to run in the background, but it's naive to believe that it's OK for Apple to create background applications and services and selling the importance of those (such as Enterprise Push E-mail) without providing the mechanisms to allow 3rd parties to do the same. Mr. Williams is dead-on accurate. Apple's in the process of allowing 1/3 of the use-cases for background services. Hopefully a year from now, Apple will be on the road to supporting all three use-cases.

Sam Johnston said...

You might well be interested to know that I have just discovered that Apple have used an instant messaging protocol (Jabber/XMPP) for the MobileMe notifications, and will likely do the same thing for developers later in the year. See: http://samj.net/2008/07/apple-iphone-20-real-story-behind-push.html

Sam Johnston said...

You might well be interested to know that I have just discovered that Apple have used an instant messaging protocol (Jabber/XMPP) for the MobileMe notifications, and will likely do the same thing for developers later in the year. See: http://samj.net/2008/07/apple-iphone-20-real-story-behind-push.html

Anonymous said...

Having not read all the comments - let me say that it would be ridiculous to rely on the push service for alarms or notifications that are not based on external events (such as being notified of a todo or calendar item). Say I am in a plane or have no cell coverage, and a time critical todo item is coming due - because of my bad memory, I may forget (which is the reason I bought a PDA in the first place - to solve this issue of forgetting - an organizer never worked, because I would forget to look at it every day!). I really need the ability to have a todo app where I can say, "remind me on the 10th to get this done before the 12th". I refuse to purchase an iPhone, because the only way to accomplish that now is to use the calendar, which sucks, big time.
Defend Apple all you want, but their controlling behavior is just plain stupid. Yeah, you can come up with slick arguments on why they are allowed to do background processing, calendar alerts, etc - and all the 3rd party developers are not. But, in the end, it just plain cripples 3rd party developers and forces their apps to suck. I don't care about your apologist arguments, Apple fanboys, all I care about is a product that does what I want, and the iPhone is not it. The only reason I get passionate is because I WISH the iPhone was that product. It is slick, and many things are well thought out... but because of issues like this, I am left with products that do what I want, but with much less polish. Frustrating, to say the least.

Aaron said...

well i have read all the comments, and i am a developer working on the sdk and i think that background processing is the only brick wall on this platform.

its forcing the development of short term, singular task based apps, and leaving large scale innovation out in the cold.

this is an amazing device, with incredible potential. honestly speaking, i don't know one developer who doesn't always have performance in mind. it would, after all, be in the devs best interest to create an app that is light weight *and* effective. i say leave it up to the developers. we're not going to knowingly create an app that becomes a resource hog. not to mention that apple can easily test the apps coming through their appstore. seriously, when you got the last say of what can and cant be installed on the device, what do you have to lose...

Post a Comment