O'Reilly Radar - Insight, analysis, and research about emerging technologies.

Innovation from the Edges: PayPal Taps the Developer Community to Build Next-Gen Payment Apps

2009-12-14T15:00:00Z

Two enduring tenets of Web 2.0 are "A platform beats an application every time" and "All the smart people don't work for you." Companies that take those bits of wisdom to heart find ways to engage developer communities to extend their products--and the result can be creative, surprising new applications that would never have been developed from within. Online payment giant PayPal recently announced the PayPal X APIs, a new group of developer APIs designed to enable new applications that can more tightly integrate with PayPal services. To encourage developers to create some awesome applications with the APIs, PayPal is offering prizes $100,000 and $50,000 (in cash plus waived transaction fees) for the best new applications. We caught up with PayPal's director for their Developer Network, Naveed Anwar, as he prepared to deliver a talk in Beijing, and he filled us in on what the new PayPal APIs bring to the table for application designers, and laid out the details of the challenge.

James Turner: In the last week or so, you've released new API. Can you talk a little bit about what's different with them, in comparison to how people have interacted with PayPal in the past as developers?

Naveed Anwar: We haven't actually released new APIs; what we have done is that we announced a set of APIs on November 3rd, which was our adaptive suite. Adaptive suite APIs include adaptive payments and adaptive accounts. Then we also announced our authentications and permissions API. What we had done at Innovate was to make those APIs exclusive -- adaptive account, authentications and permissions, exclusive to the attendees of the conference. We did a full-on release for those APIs on November 3rd when we announced the platform. But we've had an overwhelming response from the community where people where saying, "You're limiting us from actually using these APIs. Is there any ability for us to get these early on rather than to wait until 2010?" Based on the feedback, based on what people were looking for, we decided to open it up to everyone now, instead of sometime in 2010.

Very briefly, adaptive payments has a lot of core features. Some of them are the ability to send money, receive money, do parallel payments, do chain payments, do PIN authorizations. Adaptive account is the ability to create single or batch accounts in an in-flow, versus the idea of actually going and creating an account off an experience of the PayPal account. And authentication APIs is the way of actually interacting and accepting PayPal as an authentication mechanism on your site. Those are just like very high-level things that we have announced with our launch. But PayPal X per se is not just the new stuff that we announce, but also a slew of everything that is our merchant services, our web check-out flow, all of those products which have been out there in the community and people have been utilizing for the last seven years. Those all come under the same umbrella.

James Turner: So most people are familiar probably with PayPal as a third-party payment mechanism; that you go to a site, you buy something and then you get redirected to the PayPal site. What new functionality does this bring into that picture?

Naveed Anwar: Well, what you describe over there is a very typical merchant flow where someone wants to create PayPal as a payment mechanism. And once they are about to complete their transaction of the shopping cart experience, they click on a button which they come to a PayPal site and basically helps them complete a transaction. With the new set of APIs, what we've decided to open up is the ability for people to enhance that experience. Rather than to jump from their site over to the PayPal site and lose context, now people can actually incorporate that experience straight into their application. You don't have to jump onto the PayPal site to complete a transaction or to create an account. That is one major functionality that we're adding. And really, it's removing those barriers that developers have had in the past of actually trying to think of coming up with a payment solution, and giving them the set of tools which lets them concentrate on the core experience of the application versus trying to figure out how are they actually going to get payments integrated; how can they come up with a payment mechanism; what are the rules and regulations needed to set up a payment system. All of those headaches are taken away from the developer and the entrepreneur or the merchant. And that is where the solution from PayPal comes.

James Turner: So most developers have probably at some time in their life had to integrate in with a credit card processing system. What do you see the real ease-of-use or value-add on a developer perspective for using PayPal instead?

Naveed Anwar: The biggest advantage that I see is that as you look where the industry is moving towards, it's really focusing around digital goods as well as mobile. There isn't any platform out there which supports cross-border transaction and cross-border ability to pay anyone and receive goods. I see a lot of movement of goods which are actually created outside the US and then shipped over to the US as well as digital goods such as video games where a lot of the development community is out in Southeast Asia. In particular, you have developers setting out in India, China, as well as Singapore where someone wants to buy a quick experience on a digital good, be that a game on Facebook, be that a game in some other social networking environment. And that ability of PayPal, where we support 190 countries and 24 currencies, is something that no one else has been able to offer. We've done a lot of partnerships with a lot of banks and extended the reach. And that is the real value proposition of someone who integrates with PayPal X. From the very first day, you become global in nature. And with the scale of things which move these days across the globe, PayPal fits very beautifully into this as a solution for the community.

James Turner: One of the big hopes in the community has been to see micropayment become more feasible. Traditionally, the sub-dollar transaction level just hasn't been economically feasible because you get killed by the per transaction cost. Is this going to help people out?

Naveed Anwar: Absolutely. I think as the platform opens up more APIs -- I mean we did do a very brief announcement around new pricing, which will be available next year. And we'll make an announcement on what date it's available, but going down to a flat transaction model for goods moved across in certain areas. But the real thing which I look over here is that as they're looking and working with our community, we're trying to mold our roadmap based on feedback from the community. So a lot of times, you'll have companies out there talk about like, "We can offer you a certain amount of feature sets and we'll work with you." But really, no one actually reserves the road map to work with the community.

In the case of micropayments, in the case of other APIs, we've been hearing a lot of feedback from the community. What we announced on November 3rd was that we're reserving 30 percent of the roadmap based on direct feedback from the community where they get to vote on certain features that the community has been asking. And based on the ranking of the community, we will put them into our roadmap and release them. And that is one of the things that we heard loud and clear, that they wanted a micropayment solution. They wanted to see new pricing. And we touched a little bit on that on November 3rd. And early Q1, we will be announcing when that pricing will be available to the rest of the community.

James Turner: One of the things we're seeing as PayPal matures and expands is a lot more of PayPal as almost a financial institution. As you become more of a generic payment methodology for sites and other institutions, what kind of assurances are you going to be able to bring to the end users that things like fraud protection and challenges to charges are going to be easy for a consumer to do?

Naveed Anwar: So I'll take a step back for this one. Security comes to the core of what we do. That is one of the main reasons why we took a while. I mean the idea of opening up a platform is not something that came overnight. We've been thinking about this long and hard. And anytime we do deal with financial institutions and the complexity of moving money, the fact of risk management, the fact of fraud management comes in majorly. We've spent the last three years looking at all of the possible avenues in making sure that we are building out the safest and secure platform which we will open up to the developer community by making sure that we're building the appropriate risk models in place, building out the appropriate fraud deterrences in place. And that's something that we take very seriously.

That said, PayPal has been doing this for quite some time. For the last seven years, if you look at the PayPal history of being able to be the number one P2P, person-to-person payment solution, and then three years ago when we launched our merchant services program, the fastest growing merchant services program accepting PayPal, the fact that we moved almost $2,200 a second through our pipeline into 190 countries; over 24 currencies gives me quite a bit of assurance that what we are trying to build out over here is not just safe and secure, but it's also keeping in all of the models of risk that we know of to date as well as building a mechanism to anticipate things which might be hitting this industry and building a good set of resources behind it. So I feel that what we're opening up from the platform side is not just something we came up overnight but spent quite a bit of time looking into all of the details and something that we will keep monitoring very closely.

James Turner: You have, right at the moment, a challenge going on for your developers which I believe is going to end this week. Can you describe a little bit about what that’s about and if there's still time for people to get involved?

Naveed Anwar: Absolutely. The last day for people to actually register, which is basically giving your idea of what you want to submit for participating in the competition, is December 16th. Now that said, there's always that possibility of extending that deadline because a lot of people have been asking for some time. But December 16th is the current deadline for which people can submit an idea. There are two prizes. The first prize is $100,000 in which $50,000 is a cash price and $50,000 in waived PayPal fees. The second prize is $50,000: $25,000 in cash and $25,000 in waived PayPal fees. The ideas are simply to utilize and come up with an innovative application using our set of APIs, and let the community basically vote on it. So the first set of ideas, once they are submitted, are going to be opened up to the community to vote on. And from that, a set of finalists will be selected which will be judged by a panel of judges from PayPal and across the industry which are listed on our rules and regulations site. And from there, the winners will be announced. We're shooting for them to be announced at the Demo conference coming up in March.

James Turner: Obviously you don't want to give people, "Here's what to do to win." But in your mind, if you had to come up with one interesting application that people could use as an example, what would you think something you've thought of would be?

Naveed Anwar: Again, like you said, I don't want to push people in a particular area, but really looking in ways on how we can solve the problem of transactions on mobile devices is something that excites and interests me a lot, if people can think around those areas. A couple of examples that I can share that people have already built applications on is a company called Fundraiser. Fundraiser, which is a company which launched at TechCon, utilized adaptive payments as a solution in which, very quickly, they came up with an idea of setting up fundraisers for any particular cause in which people could contribute online. And that would come through parallel payments and chain payments on features out of our adaptive set of suites. So those are the kind of ideas that we're looking for. Something very creative. Something out there which people want to solve for and can be innovative around it. So that would be something that I would suggest people to look in, particularly the mobile area.

James Turner: If I'm a developer and I'm considering doing this, I haven't actually had a chance to look at the APIs and the platforms, what would my choices be as far as languages, as far as platforms?

Naveed Anwar: We have sample code available in PHP, C, C++, Ruby. I mean it doesn't limit anything. Again, it's a true platform, whatever you feel comfortable in writing code in. I think you'll find a lot of examples available in those out over here. But, again, not to limit anyone. We've got sample code available on X.com. That is our developer portal where you find sample applications, our documentation APIs, SDKs, and all of the things that you need to get started on one single destination. So I encourage developers to spend a little bit of time thinking about their idea and then looking across some of the high-level documentation before they actually sit down coding.

James Turner: So just to finish off, obviously you're talking about a fair amount of money even for a company as big as PayPal. What is it that makes you want to kind of spark this kind of interest? PayPal, in a way, is a big enough brand that you don't really need to introduce people to the brand. Why do you need to run this big promotion?

Naveed Anwar: It goes back to the basics which I started off with, in that we can probably come up with solutions and think of use cases for one or two or three or four ideas. If you look at the whole global scale of ecommerce available, it is a business. It is an opportunity of almost $30 trillion. PayPal is doing, from this opportunity that is available out there, a very small percentage. And if we could actually facilitate the community out there who are building applications -- and an example I would use is like look, when Facebook opened up their platform, it allowed people to work in that particular environment, in the Facebook environment. When the iPhone opened up their platform, they allowed people to work in their environment which was build the applications on the iPhone. When PayPal was looking at opening up its platform, we are not limited by one particular area. We go into the enterprises. We go into social networking. We go into all the places where payment as a solution is needed. And if we can actually reduce that barrier of entry -- because at the end of the day, when anyone is building out a business and anyone is building out an application, they're looking at ways of monetizing it. Traditionally, the ways to monetize is to take click-through or inserting ads, but really not a mechanism in which people can actually get real money coming to their account. That's where PayPal comes in.

From a brand perspective, I think if you look at what PayPal started off with, it was kind of like four failed business models and the fifth business model was the ability to send money to each other, and through a demo which was going from one Palm Pilot to another Palm Pilot. So doing and working from a demo point-of-view where the developer community goes back to its roots and we really wanted to go back to our roots and work directly with the community. Hence, the reason -- the push around X.com, bringing back that domain, using that for the community, as well as looking at ways in which how the community can give us feedback on the 20th or the 30th use case for which I cannot think internally, but someone out there already has a problem and they want a solution. We want to work with them to have a solution for them. What do I get out of all of this? It's that when someone thinks about writing an application, be that in an offline mode or an online mode, and they want to think about integrating payment transactions as a solution, I want their day to start at X.com and finish at X.com. That's my ultimate goal.

Apps Per Seller Across the US iTunes Categories

2009-12-14T12:00:00Z

Measured in terms of number of unique apps^†, the Top 5 categories in the U.S. app store have been Games, Books, Entertainment, Travel and Utilities. But comparing categories in terms of number of apps doesn't capture the challenge of developing applications in different categories. As I noted in an earlier post, it's much easier to develop a Book app than an interactive game.

One crude measure for the relative complexity of developing apps across categories is to compare the number of apps per seller. The Top 5 categories in Nov/2009, were Books (17 apps per seller), Travel (6 apps per seller), Education (4 per seller), Reference and Sports (3 per seller). There were also 3 apps per seller in the Games and Entertainment categories in Nov/2009:

(†) Data for this post was for through 12/10/2009, and covers the U.S. iTunes App store.

Four short links: 14 December 2009

2009-12-14T11:00:00Z

Videos from the vault of the National Archives -- the public domain US government videos that public.resource.org have been scanning. Check out China's Great Leap Forward (the Beijing landscape has changed!), John James Audubon's Birds of America, and Nature's Half-Acre.
Browser Market Share -- fantastic circular visualisation of browser share over time. (via Mike Loukides)
NYT Year in Ideas -- some fantastic stuff, but I think my favourite is Zombie Attack Science.
dygraphs -- Javascript library that creates gorgeous zoomable graphs of timeseries data.

Four short links: 11 December 2009

2009-12-11T11:00:00Z

Real Time Text Taskforce -- standardising live typing ala EtherPad and Google Wave, for accessibility reasons.
NoSQL Required Reading -- papers and presentations to get up to speed in the theory and practice of scalable key-value data stores. (via Hacker News)
It's Official, data.gov 2.0 is Coming -- pointer to the design and philosophy document for the next iteration of data.gov. Interesting to see so much activity on US open government happening now: open government directive and progress report were released, along with a request for ideas on open access to publicly-funded science research.
Breakdancing Robot -- we live in the future, and it is good. (via @hollowaynz)

Visualizing and Categorizing the 911 Wikileaks Data Set

2009-12-10T20:42:41Z

On November 25th, Wikileaks released 500,000 text pager intercepts from the 24 hours surrounding the horrific 9/11 attacks. The personal, corporate and governmental come from the Washington D.C. and New York City areas. These can be found on their own subdomain at http://911.wikileaks.org/ and are released under the CC-BY-SA license.

As with the AOL search logs and the Enron email archives this data set will be examined and visualized. I am sure that the hope will be to gain an understanding of the thoughts and feelings of the people on the ground. Two applications have already been created.

911pagers is a site devoted to searching and community annotations of the corpus. You can see the theoretical communications with Guiliani (New York's Mayor at the time), random messages or the recommended . Built on Google Appengine, 911pagers will add ratings, timelines and keywords. You can track their progress via @911pagers.

The second project is an analysis of the frequency of 100 phrases such "flights cancelled" and "call home". Jeff Clark selected only the content from 8AM to 8PM on 9/11. He created a set of timeline graphs for each phrase (above). After the jump I've embedded a timeseries video of these phrases.

Here's the video:

Pager Data from 9/11 - Phrase Cloud Visualization from Jeff Clark on Vimeo.

Jeff says this about the video:

This is a visualization of text phrases taken from pager data during September 11th, 2001 from 8am until 8pm. The larger the text the more frequently it was used during the 12 hour period. Text appears bright during the times of high usage and fades away otherwise. Color hues are cosmetic.

Four short links: 10 December 2009

2009-12-10T11:00:00Z

Scriblio -- open source CMS and catalogue built on WordPress, with faceted search and browse. (via titine on Delicious)
Useful Temporal Functions and Queries -- SQL tricksies for those working with timeseries data. (via mbiddulph on Delicious)
Optimal Starting Prices for Negotiations and Auctions --Mind Hacks discussion of a research paper on whether high or low initial prices lead to higher price outcomes in negotiations and online auctions. Many negotiation books recommend waiting for the other side to offer first. However, existing empirical research contradicts this conventional wisdom: The final outcome in single and multi-issue negotiations, both in the United States and Thailand, often depends on whether the buyer or the seller makes the first offer. Indeed, the final price tends to be higher when a seller (who wants a higher price and thus sets a high first offer) makes the first offer than when the buyer (who offers a low first offer to achieve a low final price) goes first.
WiFi Science History -- Australian scientist studies black holes in the 70s, has to develop a way of piecing together signals that have been distorted as they travel through space. Realizes, when he starts playing with networked computers in the late 80s, that this same technique would let you "cut the wires". A decade later it emerged as a critical part of wireless networking. As Aaron Small says, it shows the value of basic research, where you don't have immediate applications in mind and can't show short-term deliverables or an application to a current high-value problem.

Four short links: 9 December 2009

2009-12-09T11:00:00Z

The Mythology of Bioinformatics -- worth reading this (reprinted from 2002!) separate of hype from history.
Policy and Internet -- new journal, with articles such as The Case Against Mass E-mails: Perverse Incentives and Low Quality Public Participation in U.S. Federal Rulemaking: This paper situates a close examination of the 1000 longest modified MoveOn.org-generated e-mails sent to the Environmental Protection Agency (EPA) about its 2004 mercury rulemaking, in the broader context of online grassroots lobbying. The findings indicate that only a tiny portion of these public comments constitute potentially relevant new information for the EPA to consider. The vast majority of MoveOn comments are either exact duplicates of a two-sentence form letter, or they are variants of a small number of broad claims about the inadequacy of the proposed rule. This paper argues that norms, rules, and tools will emerge to deal with the burden imposed by these communications. More broadly, it raises doubts about the notion that online public participation is a harbinger of a more deliberative and democratic era. (via Jordan at InternetNZ)
Xena -- GPL-licensed Java software from National Archives of Australia, to detect the file formats of "digital objects" and then converting them into open formats for preservation.
Nebul.us -- startup that aggregates and visualises your online activity. In private beta, but there's a screenshot and brief discussion on Flowing Data.

GWT Now With SpeedTracer

2009-12-09T03:00:00Z

Google is releasing v2 of GWT (pronounced "Gwit") tonight at a Campfire One in Mountain View. The open-source Google Web Toolkit enables developers to code Ajax web apps in Java. This latest release is focused on speed (just like the latest iPhone) and improved dev-designer collaboration.

I was on a call with Bruce Johnson and Andy Bowers to learn more about the release. There are three new major features being released tonight.

Of the three SpeedTracer (screenshot above) seems to have the greatest implications. It is a separate application that allows developers to watch "where time goes" as a page is being loaded. The example Johnson gave was a recent re-write of the AdWords Campaign Manager. There's a 100-row table that is a part of the application. In the first iteration the table took several seconds to load. Using SpeedTracer they realized that the order of the UI statements was delaying DOM translation. They changed a few lines and the table loadtime dropped to half a second.

SpeedTracer only works with Chrome. Google is making a play for the developer. And by focusing on developer tools they are likely to succeed. The speed improvements should translate to other browsers and in reality they have to. Focusing on speed improvements for a currently-minority browser would not win over many devs.

Code Splitting, the second feature, is a way of segmenting out non-essential portions of an Ajax app for future, as-needed download. The example given was that for a mail app you don't always need to be able to write or the settings tab. You could download the code for those functions on demand. Wave apparently uses this functionality to keep their loadtimes down.

Finally, UIBinder is aimed at improving developer-designer workflow. It allows designers to hand over HTML elements in an XML Template. They can also use GWT Widget library for certain elements. Developers can easily plug their application into this XML presentation layer. This is a far cry from being able to use Photoshop mockups to generate code ala Flash Catalyst, but if the design department knows HTML it should be very handy.

GWT is increasingly being used on Google's own applications. It was quite famously used to make Wave. It is also used on the revamped Orkut (I hadn't been there in over a year, it has had quite a makeover), AdWords, Google Squared (labs), Google Profiles, and Moderator (20% project). DoubleClick is being converted to GWT and a lot of Google's internal infrastructure is built using it. Quite notably some of Google's mobile app have been built with it: Mobile Maps and Latitude for the iPhone. It's great to see Google putting it's money where it's mouth is (quite literally with AdWords), but I really wonder if we'll ever see GMail powered by GWT.

In the future I'd be surprised if there wasn't increased support for HTML5 elements such as Canvas. I also expect that they'll add explicit support for mobile app. Of course, using GWT doesn't block a developer from using non-GWT functionality, but having explicit support always helps.

Four short links: 8 December 2009

2009-12-08T11:00:00Z

Python's Moratorium -- Python language designers have declared a moratorium on enhancement proposals (feature requests) while the world's Python programmers get used to the last batch of New And Shiny they shipped. I'm reasonably sure that the ALGOL designers went through exactly the same discussions, and I know Perl did too. So, don’t be afraid of it - don’t think that Python is evolutionarily dead - it’s not. We’re taking a stability and adoption break, a breather. We’re doing this to help users and developers, not to just be able to say “no” to every random idea sent to python-ideas, and not because we’re done. Reminds me of Perl god Jarkko Hietaniemi's signature file: "There is this special biologist word we use for 'stable'. It is 'dead'. -- Jack Cohen.
This Week's Finds in Mathematical Physics -- I can't meaningfully contribute to the math, but golly them pictures are purty! (via Hacker News)
x86 Assembly Encounter -- To use a construction industry metaphor, an average x86 assembler has the complexity and usefulness of a hammer, while the DSP world is using high-speed mag-rail blast-o-matic nail guns with automatic feeders and superconducting magnets. [...] I find it ridiculous that the most popular computing platform in the world does not have a decent assembler. What’s even worse, from the discussions I’ve seen on the net, people are mostly interested in how fast the assembler is (?!) rather than how much time it saves the programmer. (via Hacker News)
Finding Tennis Courts in Aerial Photos -- more hacking with computer vision techniques and publicly-available data. This is going to lead to good things (and some unpleasant surprises, as that which was formerly "too hard to find" ceases to be so). (via Simon Willison)

Twitter Approval Matrix - November 2009

2009-12-07T18:34:02Z

This is the sixth post for the Twitter Approval Matrix with data that spanned the month of November and different sources such as klout.com, tweetsentiment.com, twopular.com, scraping archives, and observations. This month I received help from Joe Fernandez the CEO of Klout.com. Joe continues to provide some great 'hard' data that allowed me to better place more items on the grid this month. This month I have included Twitter Trends which is simply the raw trend found on Twitter. When I mention "influence scale" this comes from Klout and is basically a weighted measure that shows how much amplification a message/topic gets from Tweeters who have lots of followers, get lots of RTs, and get a message widely disseminated.

A quick refresher, the matrix shows four quadrants used to describe trends found on Twitter. The Y-axis is partly analytical and shows popularity (mostly through scraped numbers) or perceived popularity (in the future nominated by you). The other part of the grid is more curated and subjective. The X-axis has been plotted based on my personal opinion. You may agree or disagree with my placements and that's all good to me. After all, this is partially about taste and numbers. The matrix and plots do not represent a thorough analytical treatment, but rather a view of the trends that could be found in data sources allowing me to plot with some sense of relevance.

For this post, I've limited the data and activity to the month of November. Again, I'll continue with this project as long as I get enough feedback/help. So, if you are interested in contributing, you can comment here, or read the original post to figure out the best way for you to submit your plots.

I hope you enjoy this and see it as a potentially useful tool to monitor trends that your fellow readers are both contributing to and tracking.

Four short links: 7 December 2009

2009-12-07T11:00:00Z

3D Touchscreens -- Japan Science & Technology Agency and researchers at the University of Electro-communications have made a "photoelastic" touch screen. The LCD emits polarized light, picked up by a camera over the screen. Transparent rubber on the screen deforms when pressed, and the camera can pick this up. Interesting hack, though it's not yet a consumer-grade product.
Eureqa -- open source tool for detecting equations and hidden mathematical relationships in your data. Its primary goal is to identify the simplest mathematical formulas which could describe the underlying mechanisms that produced the data. (via pigor on delicious)
Science in the Open, It Wasn't Supposed To Be This Way -- Cameron Neylon on the leaked climate email messages as a trigger for open data. One of the very few credible objections to open research that I have come across is that by making material available you open your inbox to a vast community of people who will just waste your time. The people who can’t be bothered to read the background literature or learn to use the tools; the ones who just want the right answer. [...] my concern is that in a kneejerk response to suddenly make things available no-one will think to put in place the social and technical infrastructure that we need to support positive engagement, and to protect active researchers, both professional and amateur from time-wasters. Sounds like an open science call for social software, though I'm not convinced it's that easy. Humans can't distinguish revolutionaries from terrorists, it's unclear why we think computers should be able to.
EtherPad Back Online Until Open Sourced -- Google bought collaborative real-time EtherPad and the team will work on Google Wave, but the transition plan was "you can't create more documents, and it'll all go away in March". Grumpiness ensued. Everyone makes mistakes online, but the secret is to listen, acknowledge the mistake, and correct your course.

Four short links: 4 December 2009

2009-12-04T21:30:00Z

Readability -- bookmarklet that takes the crap out of a web page, resizes, and reformats so it's easier to read. Doesn't work for all sites, but it's a hellishly interesting idea.
An In-Depth Look at Pivot, Microsoft's Newest Data Visualization Tool (TechCrunch) -- When turned on, Pivot can also make sense of your own browsing history (if you are using Internet Explorer). Pivot hosts a portion of IE within the app, and will get a sense of your browsing history over time and then will slice and dice your history based upon various verticals.
Eben Moglen, Lazarus Phenomenon, and The Girl Scouts of America (Brian Aker) -- Once software has been published under an open source license, it continues to be available, whether its current owners wish it to be or not. I've been in the library world long enough that while I respect Brian's argument, I know that preservation doesn't happen by magic. Perhaps we need an Internet Archive for software.
Three Lessons I Learned From Porting Diablo (Pete Warden) -- The second lesson I learnt was that I wanted to work with people like Gary, willing to help the whole team, rather than hunting for individual glory. I've since worked with a lot of 'rock star' programmers, and while they always look good to management, they hate sharing information or credit and end up hampering projects no matter how smart they are as individuals. Gary used his massive brain to help make us all more effective instead, and I've always tried to live up to his example.

The Lessons We Don't Learn

2009-12-04T01:37:45Z

In my Twitter stream today, Sylvia Martinez (@smartinez) retweeted a link to Seymour Papert's 1980 paper written for a Presidential commission that proposed that we provide a computer for every child in America. Long before One Laptop Per Child, Papert saw that computers should not be an "auxiliary" aid to learning but "fundamental" to changing how we learn. He understood that the computer by changing education could change our culture for the better.

I believe that the computer could be used as a powerful weapon to break down barriers related to gender, ethnic culture, class origin, and even genetic differences.

By no means do I believe that computers are the only thing needed to improve education but Papert was so right nearly thirty years ago in recognizing the potential for computers (and networks) to break down all kinds of barriers, and to open doors to opportunity.

On this same day, I heard from a physics teacher in California that he can't access the Makezine.com site. He was trying to download a project plan for the Wooden Mini Yacht in Volume 20 of Make to use in his class. His school district uses software to block access to any sites that have a "blog." The teacher said he calls up regularly to request access but even when he gets it, the change only lasts a few days and then the site is blocked again. It's a second such comment made by a teacher in recent weeks so I don't believe it's unique to this school. This is a high school teacher seeking free resources on the Web to use with students in the classroom. It's too bad that it's so hard for him to do what he wants. It is just one example of how our educational system fails to grasp the fundamental uses of technology.

After thirty years, Papert's call for action is still fresh today:

I believe that one of the most urgent national needs for the 1980s is to find ways to increase the technological sophistication of the education community, to create contexts in which educators can probe the potential effects of fundamental uses of computers.

It doesn't feel like we've seen much progress in education. Pappert made these strong recommendations but there was little urgency to act on them. Is that the pattern -- gather up the good ideas and then decide not to do anything with them? I was reminded of Taylor Marsh's recent article on a day-long Washington DC conference on the Innovation Economy. It was another talkfest about "what we should do" to improve the future of American competitiveness. One of the participants, Senator Mark Warner, said that we need "radical rethinking of high school and college. Does high school need to be 4 years, does college?"

After listening to a laundry list of such recommendations, Taylor concluded:

An innovation economy may be able to save our nation, but not with the current crop of political leaders, regardless of party, who don't seem to be able to take any good idea and move it forward. Small thinkers, vested interests, no political will to move forward together, with the upper crust stifling so many Americans who just never get a chance. [Can the Innovation Economy Save the U.S.? (HuffPost)]

Papert's paper ended with a warning: "Unless we do this, tomorrow will continue to be the prisoner of the primitivity of yesterday." Tomorrow is here and educators are still being held prisoners of the past. I just wonder if it's truly possible to move forward.

Google Android: on Inevitability, the Dawn of Mobile, and the Missing Leg

2009-12-03T21:00:54Z

If for no other reason than the "Anyone but Apple" crowd NEEDS an alternative, there is an "inevitability" meme associated with Google's Android initiative.

After all, Google is formidable, has a strong brand, and their (relative) openness is the "zig" to Apple's proprietary "zag." And of course, mobile is strategic to Google's future, so they can be expected to compete vigorously for market and mind share (via Android) over the long haul.

But, do those ingredients combine into a recipe that makes their success in the market inevitable? Over a year after Android's launch, I have to say that the jury is still out.

Why do I say this? Most basically because reconciling the fragmentation challenge of supporting a heterogeneous software platform running on top of divergent hardware form factors with the proprietary aspirations of handset makers, software developers and carriers is far harder to balance than most recognize.

But, you say, can't we look to the PC as a historical guide? Isn't this merely Windows vs. the Mac for the Mobile Broadband Era? Not necessarily. Unlike the PC, which was fundamentally a homogenization play (i.e., hardware and software was fairly consistent from vendor to vendor), mobile is heterogeneous in terms of its support for hardware, software, and service layer diversity, a trend that Android, if anything, seeks to accelerate.

Moreover, unlike the PC, where "good enough" was the bar required to seize the market, the mobile consumer expects seamless interaction and robust performance across multiple modalities, including voice, search, geo-navigation, communications, gaming, and social networking.

Anchoring all of this is a fundamental truth that for most consumers, their mobile device of choice is a lifestyle decision, a personal, ever-present extension of themselves that is resident in a way that never existed before with the PC--a value proposition that Apple has completely run with on iPhone (and iPod before that).

Fundamentally, though, mobile is a platform play, a game that is largely won by securing the hearts and minds of developers, and for them, the expectation bar is now set pretty high, owing to the success of iPhone across so many domains, including installed base (and guaranteed reach into that base), operating margins, developer ecosystem, application uptake, functioning marketplace, and the ability to target both carrier-based and tariff-free market segments (via iPhone & iPod Touch).

Simply put, developers will default to developing on platforms that:

1. Gain them a readily addressable audience
2. Allow them to make money
3 They enjoy personally using, as the best solutions often result from an "unscratched itch"

This is the bar that Android must satisfy to become a durable player in the market, and as I will lay out, they have a long way to go.

Inevitability: Just around the Corner?

When Verizon announced Motorola's Droid (with behind the scenes high-touch support by Google), the conventional wisdom was that this device was the closest proxy to being a viable competitor to iPhone, without actually being a serious threat.

In other words, a good, reasonably inspired, and largely caveat-free handset, but still a generation or two away from being Insanely Great.

Well, the first wave of market feedback is in, and the data, while encouraging, is clearly mixed. On the one hand, it looks likely that Droid will hit 1M units sold by year's end, which is great for a first-generation handset.

In terms of the larger market prospects for Android, one can also safely assume that this is the tip of the iceberg, and that another 5-6 increasingly rock-solid handsets are germinating in the ground, ready to raise the bar still further.

At the same time, this very premise--device diversity--presents a bit of a conundrum. Samsung couldn't care one whit about Motorola's success. Quite the opposite. So when they come out with their own rockin' Android handset, they should create their own distinct hardware form factor, and their own social service layer (it's not going to be Motorola's MotoBlur).

Who knows, they may offer up their own proprietary middleware to enable Samsung-brewed apps to play particularly well together, so as to incent developers to take advantage of the native features of Samsung devices. This is all good, right? One of the benefits of an open platform, right?

Now, as a developer, do you develop different versions of your software to take advantage of the cool features of each of these different devices (and the lifecycle of supporting same)? Do you focus on just the device that pushes the highest volume (and release more apps specific to that device)? Or, do you pursue a lowest common denominator that strives for uniformity across all form factors?

It's the consummate highest common divisor v. lowest common denominator forking decision, and the complexity of making such a strategic decision is muddied further by rumors that Google is going to come out with an official Google Phone that features the "Real Android," a move that, if true, could really upset the apple cart (no pun intended) with handset makers and carriers.

Post-Droid Launch: The Android Buzz Kill

It was inevitable, but with hype comes disappointment, and now that the first reports are in (from industry types on Droid - HERE, HERE, HERE), the crowd is railing on everything from an under-baked software platform to hardware problems (such as a useless physical keyboard, disappointing camera performance, and battery doors perpetually falling off; the latter of which must be serious Schadenfreude for iPhone owners, who've endured rants about the idiocy of Apple not allowing consumers to replace their own batteries -- owing to the absence of a battery door on the iPhone and iPod Touch).

Oh, and developers don't seem too happy about the way Android's version of the App Store model--the Android Market--works for them. Why? For starters, unlike the global reach into 50M+ iPhone/iPod Touch devices that iPhone developers can plan their world around, the Android Ecosystem works differently, with some handsets and some carriers having varying levels of app catalog completeness, not to mention, workflows that make it difficult to discover new apps and a payment process that is, well, Byzantine.

Plus, the different form factors are already exacting a tweak-and-debug "tax" to gain the leverage of multi-handset support, a lifecycle that only figures to get more complex going forward.

Perhaps all of this wouldn't matter if developers were seeing the kind of uptake (in downloads and dollars) that has been seen on the iPhone Platform, but so far, that is not happening, prompting mobile gaming app developer Gameloft to cut back its investment in Android, noting (according to Gameloft finance director Alexandre de Rochefort) weaknesses in Android's application store design.

"It is not as neatly done as on the iPhone. Google has not been very good to entice customers to actually buy products. On Android nobody is making significant revenue," Rochefort said.

In a word, "Ouch." All of this just goes to underscore how much Apple has gotten right, and that fact that it is non-trivial for others to emulate.

Is the Android Table Missing a Leg?

Imagine a table with four legs. In the mobile universe, the table's legs are:

1. Great handset functionality, horsepower and design
2. Thriving software developer ecosystem
3. Carrier coverage, reliability and performance
4. Media player and marketplace

So how does Android sit in each of these four categories today? Relative to the actual handset itself, if Droid represents the best bones to date upon which a second and third generation device will be built, it should be clear that Apple's design prowess remains a serious competitive advantage. That said, a score of device designs are just around the corner, so this is an area where evolution should be fairly rapid, although capturing market share from Apple is hardly a given.

As to the developer ecosystem's health, the simple truth is that Android + Android Market remains 12-18 months behind App Store and the iPhone Platform. Whether developers jump on the bandwagon early or wait until the platform matures is very much up in the air. In fact, early warning signs suggest that certain developers, including the aforementioned Gameloft, are measuring their investments in Android for the time being. The inevitability of the ecosystem is decidedly a "not yet" score.

It is the area of Carrier coverage that Android has its strongest leg. Personally, the ONLY reason that I remain a Blackberry Tour owner is that I am a loyal Verizon customer, and AT&T's reputation is tarnished, to say the least. Carrier diversity across the Android Ecosystem is one area that gives consumers the ability to pick and choose the combination of price, performance, and features that matters most to them.

However, it is in the area of Media Player and Marketplace that the Android table starts looking like it is missing a leg. Sure, there are a number of iTunes alternatives that work with the Android (doubleTwist is probably the best. Google should buy them).

But, I would argue that media is so endemic to what people "hire" their mobile devices to do for them that the end-to-end media sandbox shouldn't be loosely integrated into the device. It should be core to the platform.

This implies an iTunes-like client application plus an integrated media player that ties in seamlessly with the media marketplace functions.

I know that what I am about to say should be obvious, but many forget that nested Russian doll-style into the iPhone is an iPod; you know, the media player that should have been outflanked by "someone" long ago, but continues its dominance unabated.

Plus, think how the iPod Media Player + iTunes Client & Media Marketplace acts as foundation/feeder to the overall user engagement and monetization workflow that consumers happily default into with iPhone (and iPod Touch).  It's pretty damn potent, and just works, a bookend to the complaints about the clunkiness of Android Market. Android needs a better strategy here.

Netting it out: So much of the Google DNA is about loose coupling and 'good enough,' which doesn't pass the sniff test when you have experienced best of breed, tight integration and of course, a deep library of media and apps with iPhone.  As such, Android has a longer way to go to realize its "inevitability" premise than it might appear at first blush.

Also, while all of these problems are solvable, Google has neither shown that they have the wherewithal to succeed where others have failed (think: Microsoft, Palm, Symbian), nor is it like Apple is sitting on their hands. The folks in Cupertino, after all, have proven repeatedly that they can execute with laser-like focus--a truth which shows in the product, the user experience, market share, momentum, and the sexy margins that go with having hit the ball out of the park.

Related Posts:
1. Android vs. iPhone: Why Openness May Not Be Best
3. iPhone, the 'Personal' Computer: The Future of the Mobile Web
3. iPhones, App Stores and Ecosystems

Four short links: 3 December 2009

2009-12-03T11:00:00Z

How Robber Barons Hijacked the Victorian Internet (ArsTechnica) -- cautionary tale of the exploitation of a monopoly. Once installed as the dominant proprietor of the nation's telegraph system, public trust in the confidentiality of Western Union transmissions evaporated. Gould "scanned the telegraph, or manipulated it, as an open book to the secrets of all the marts," Josephson wrote.
350 Years of Royal Society Correspondence Online -- the concept is great, the content is great, the interface lacking. (via auchmill on Twitter)
Harvard Study: Computers Don't Save Hospitals Money -- The recently released study evaluated data on 4,000 hospitals in the U.S over a four-year period and found that the immense cost of installing and running hospital IT systems is greater than any expected cost savings. And much of the software being written for use in clinics is aimed at administrators, not doctors, nurses and lab workers. [...] He pointed to Brigham and Women's Hospital in Boston, Latter Day Saints Hospital in Salt Lake City and Regenstrief Institute in Indianapolis as facilities with some success in deploying efficient e-health systems. That's because they were intuitive and aimed at clinicians, not administrators. I'm not sure anyone, not even the study's author, knows what success looks like. Lower costs, yes. Data to improve quality of care, yes. Data to contribute to population statistics, yes. Greater throughput, yes. Fewer lost patients, yes.
Forecasting Private Consumption: Survey-based Indicators vs. Google Trends -- turns out that Google search terms over time beat some of the traditional consumer sentiment indicators. (via 130)