Talk Resources – Riding the Geolocation Wave

At both the REMIX10 conference in Melbourne, Australia and more recently TechEd New Zealand I presented on geolocation for developers.

This was the abstract:

It’s pretty obvious by now that geolocation is a heavy player in the next wave of applications and APIs. Now is the time to learn how to take advantage of this information and add context to your own applications. In this session we’ll look at geolocation at every layer of the stack – from open protocols to operating system APIs, from the browser to Windows Phone 7. Building a compelling geo-enabled experience takes more than simple coordinates. In this session Tatham will introduce the basics of determining a user’s location and then delve into some of the opportunities and restrictions that are specific to mobile devices and their interfaces.

The talk was filmed at TechEd New Zealand, and is available for download here: http://www.msteched.com/2010/NewZealand/WEB205

(Note: this version has a Windows Phone 7 demo in it too.)

GeoNZScreenshot

The first version of the talk was also filmed at REMIX10, and is available for download here: http://www.microsoft.com/australia/remix/videos/default.aspx

GeolocationScreenshot

Here are some links to the code and resources (but you really want to watch the talk first):

(Post last updated 7th Sep 2010 with new links and videos)

World Wide Telescope Backchannel Chat

Thanks to an invite from Nick Hodge, I spent an hour this morning in a chat with the creators of WWT. I was broadcasting notes of my chat to the Readify internal tech list, and this is a post-chat dump of that discussion.

10:00am

For those interested, I’ve been sitting on a backchannel chat around WWT.

There are some good questions being raised.

I asked about the similarities between it and terraserver (which was worked on by the same guys). Similar concept of a a tile-based system to pull down the images, but it uses a projection called Toast instead of Mercator. Mercator doesn’t work properly at the poles, and as you pan around the north/south axis actually change direction. (Try looking at the poles in VE and you’ll see this pretty clearly).

Toast is basically … Imagine the world as an octohedron. Take a single square sheet of paper and wrap it around that octohedron. Make the middle of the piece of paper the equator. Fold down the corners to the north and south pole. Then tile it for multi-resolution… They use a similar quadtree system for adressing these tiles.

I asked if Toast was something MSR developed. It’s been around for a while now to the point that they forgot who first came up with it. 🙂 NASA + CalTech have been developing it further and using it for their full sky projections. There are 50 full sky surveys in WWT.

Social networking on top of WWT. One guy has suggested a social network on top of it for amateur astronomers to upload and share their own data. There seem to be some plans around this coming which sounds cool.

Indigenous constellation data – american indian, aboriginal, etc. You can already create your own star maps with constellation figures – first step for the above idea. They’re also currently researching and working with indigenous groups to develop new tours. So like the current “Visit the centre of the Milky Way” tour and that kinda stuff, you’ll soon be able to do a tour of the Aboriginal Dreaming stories and stuff like that.

Infrastructure. Tile data is served off the VE servers (there’re already 1,000 servers there – might as well use them). The app servers run separately and are funded by MS External Technology Group (whoever they are).

Still going … will reply with mroe details in a min.

10:09am

First release of a dev kit in about a month. WTML. Communities (add menu items and data links to websites and stuff), data model, how to present your images, etc.

v2 they might release an object model for driving it yourself, embedding it and writing plugins that do more than just inject web links. “But that’s down the track for now”

Oh cool – you can right click on any point of the sky surface and copy a link to it. Send it to a friend and when they click it, WWT will open and sleuth to that point. You can integrate your own links there to your own sites too … with the communities API.

10:21am

Original prototype of the engine developed in 2001. Some parts of the model as far back as 1993. Client built over the last 18 months by essentially a 1 man team. Was done in Managed DirectX for development productivity. He thinks if he’d done it in C++ and OpenGL it would have taken him a team of 8 people and 2 – 2.5 years.

The main digital sky surface is one terapixel – one million pixels by one million pixels. They’re about to load in an IR based surface which is four terapixels!

They have more than a dozen TB of data directly available or connected to their databases. Summer release to would add an interface to the Hubble archive (another 400TB). That will be stored by the Hubble people still, but WWT as the client. They’re plugging in more and more sources, so answering the question of “how much data?” is becoming harder and if they can answer it accurately they consider they have failed.

They have two clusters they use for processing data. They had a third one which has now been transitioned into their app tiers. Serving millions of hits a day, so third cluster exclusively used to serve that.

Almost exclusively C# based applications.

Some partner called DigiPede (?) provided their grid solution.

Somebody asked about Easter Eggs. Apparently that’s a fireable offence at Microsoft now, so unlikely apparently. 🙂 The list of contributors is very publically listed instead.

The only secret code is a big ugly Close button on the about dialog which only appears if your username is “dinos”. Apparently a tester wanted a close button and kept reopening the bug so he put it there just for her. 🙂

Some things like the moon are dynamic – if you speed up time you can see it move across the sky. If you speed up more you can see it rotate and move through phases.

Man-made objects like ISS are hard to track because they move too fast across the sky so users would keep losing them. It’s more appropriate to do point-in-time type solutions. I asked about them providing a feed of man made objects so that when new shuttles launch we can track them – no answer as yet.

10:20am

Andrew Matthews sent me:

Hi Tatham,

I’d like to see the following on WWT. All of which is based on the idea that WWT is not just a platform for armchair astronomers, but a souped up interactive star chart that can be used by amateur observers.

  • Night sight mode with everything in red. Allows night observers to use it without getting after images
  • Telescope control software
  • CCD image registration software
  • Catalog identification of stars
  • Move to stars by catalog id
  • Ephemeris
  • Observing logs
  • RSS feeds with notable happenings coming up (ie. Transits, meteor shows, newly discovered comets, supernovae etc)

10:25am

Ooo – answer to my man made object feed question. Everytime you click into communities it’s reloaded, so if somebody exposed a community for the ISS for example, it’d be trackable and up to date. In the 2.0 timeframe communities would be able to describe orbital elements too (so they get animated in real time). A few months until that functionality.

Asking about night sight mode right now…

Telescope control software already in there.

CCD image registration? explain quickly for me

Catelog identification – already there. Right click on any point in the visual and it tells you what it is.

Move to stars by catalog ID – does that too I think.

Ephemeris? explain quickly for me

Observing logs … hmpf – will ask

RSS feeds for notable events – this is covered by the communities system. Communities API uses WTML to describe space stuff and integrate that data into the UI, add menu options to them and that kinda thing.

10:27am

They’re talking about an ISV who builds telescope control software who’s developed somethign that lets you stream imagery of live events into it! Cool …

Night sight:

“good suggesstion. the way I personally use this – i have many different astronomy apps and all their different night sight modes are unusable. in wwt you can adjust the colours in wwt and windows to pick ones with decent contrast and stuff. essentially, thats the way i use it. i used it a year ago at a star party to do all my imaging and control my telescope and putting a filter over it, it works fell.”

(Filters are a UI feature in WWT which allows you to adjust the hue of the whole interface – under the view tab at the top)

10:29am

Andrew Matthews sent me:

Image Registration: Take multiple pictures through CCD (or webcam) and then do image analysis to remove images blurred by atmospheric turbulence

10:32am

> Ephemeris? explain quickly for me

I googled it to find a definition. 🙂

A lot of objects in there already do that. The whole UI is point-in-time based, and everything moves accordingly. It plays out in real time, or you can accelarate it.

Some objects, like the moon, even rotate and move through their phases.

The communities support will be upgraded in v2.0 to include orbital object support. So if you publish a community, you’ll be able to markup your own orbital objects and it’ll position them and animate them accordingly.

10:33am

Missed out on the CCD image registration question … most likely a plug in you would develop yourself.

WWt isn’t meant to replace software like the $500 planetarium in a box solutions.

[11:45am Updated the username for the easter egg. ;)]

Location, Location, Location: My plan for location awareness, and the GeographicLocationProvider object

I know where I am. My phone knows where it is. Why doesn’t the webpage know where I am?

Think about these scenarios which will become more and more prominent as “the mobile web” starts to prevail:

  1. I visit a cinema website on my mobile. Rather than having to choose my cinema first, the website already knows which suburb I’m in so it defaults to the nearest cinema first.
  2. I need a taxi. I don’t know where I am, but my phone does. I want to be able to book a taxi and have the website discover my location automatically.

The key idea that I’m exploring here is the ability for a webpage to access this information through the standard browser interface.

I have a plan for making this a reality.

Windows Mobile has already taken a step towards baking location awareness into the OS with their GPS Intermediate Driver. The idea is that the operating system handles communication with the GPS unit, including all the various protocols. Applications then have a unified API for accessing GPS data. This proxy effect also facilitates non-exclusive access to the GPS.

But this doesn’t go far enough. Even with this unified API, very few applications are actually location aware. More importantly, I don’t want to have to download and install a piece of software on my device just to be able to see movie times. It’s just not going to happen.

We’ve also been making the assumption that location data comes from a GPS. Enter GeoPriv.

With the continuing rollout of VOIP, there are obvious challenges about the loss of location awareness. The current analog network makes call tracing relatively easy. It’s a fixed line of copper and the phone company knows where it terminates. This is a legal requirement for emergency call routing, as well as being immensely useful for scenarios such as a national number auto-routing to your nearest local store. Both of these scenarios become immensely difficult when you can’t even rely on there being a physical phone anymore – a piece of software with a network connection is now a fully fledged communication device that needs to support these scenarios somehow.

There’s an IETF working group tasked to solve this exact problem. The privacy impacts of sharing location data are so important that it’s in the name. They are the “Geographic Location/Privacy working group”, or “GeoPriv“. The best part is, they are living in a reality and delivering useful technology – and fast.

There are a number of key concepts they have identified:

  • We can’t go jamming a GPS chip in every single device we manufacture. We need to be able to capitalize on the ecosystem surrounding our existing devices to surface the information we already have.
  • Privacy is a critical element of any solution expecting wide spread adoption and trust.
  • There are two possible types of location you could need:
    • civic location (level, street number, street, suburb, etc)
    • geographic location (latitude, longitude, elevation)

Lets step away from mobile devices briefly and consider the laptop I’m writing this post on. My laptop doesn’t know where it is. Neither does my WiFi router, or my DSL modem. My ISP does though.

At some stage in the future, my modem will start receiving an extra DHCP option. In the same  way that my ISP supplies me with network settings like DNS when I connect, they will also start pushing out the address of their Location Information Server. My DSL modem will then push this setting out across my network. Finally, my laptop will be able to query this service to find out my current civic and/or geographic location. The privacy controls around this are beyond the scope of this post.

By asking the service provider for the information, these same techniques also works for mobile devices, 3G data connections, and all those other wonderful wireless technologies. Cell-based triangulation is already in use by phone companies around the world, including our national carrier here in Australia, however the interfaces are in no way standardized. The Location Information Server (LIS) and the HTTP Enabled Location Delivery protocol (HELD) solve this problem.

Now that our device is capitalising on the network ecosystem, getting it into the browser is the easy part. All that’s left is a thin veneer of JavaScript.

Location awareness is only becoming an increasing demand. I want to start the process of rolling in the JS layer now, so that as the supporting technologies come to fruition, we have the access layer to make them useful.

Inline with the XMLHttpRequest object that we’ve all come to know and love, I’ve started writing a spec for a GeographicLocationProvider object.

With XMLHttpRequest, we can write code like this:

var client = new XMLHttpRequest();
client.onreadystatechange = function()
{
    if(this.readyState == 4 && this.status == 200)
        alert(this.responseXML);
}
client.open("GET", "http://myurl.com/path");
client.send();

I want to be able to write code like this:

var provider = new GeographicLocationProvider();
provider.onreadystatechange = function()
{
    if(this.readyState == 2)
        alert(this.geographic.latitude);
}
provider.start();

Again, usage is conceptually similar to the XMLHttpRequest object:

  1. Initialize an instance of the object
  2. Subscribe to the state change event
  3. Set it free

The potential states are:

  • STOPPED. This is the state the the object is initialized in, and the state that it returns to if stop() is called.
  • RESOLVING. The object has been started, but not location information is available yet. In this state the browser could be:
    • prompting the user for permission,
    • searching for location sources (like GPS hardware, or an LIS endpoint), or
    • waiting for the location source to initialize (like connecting to satellites, or talking to the LIS)
  • TRACKING. A location source has been found and location data is ready to be queried from the provider.
  • UNAVAILABLE. No location data is available, and none is likely to become available. The user may have denied a privacy prompt, their security settings may have automatically denied the request, or there may be no location sources available. It is possible for the provider to return to the RESOLVING state if a location source become available later.

In more complex scenarios, the provider can be primed with a specific request to aid in evaluation of privacy policies and selection of location sources. For example, browsers may choose to hand over state-level civic location data without a privacy prompt. This data could also be obtained from an LIS, without needing to boot up a GPS unit. If the webpage requested highly accurate geographic location data, the browser would generally trigger a privacy prompt and boot up the most accurate location source available.

While we’ve now simplified the developer experience, the complexity of the browser implementation has mushroom clouded. How do we reign this in so that it’s attractive and feasible enough for browser implementers? How do we demonstrate value today?

You might have noticed that in my discussion of the JS layer I drifted away from the GeoPriv set of technologies. While any implementation should be harmonious with the concepts developed by the GeoPriv working group, we aren’t dependent upon their technology to start delivering browser-integrated location awareness today.

There are numerous location sources which can be used:

  • Statically configured location – with the network fingerprinting technology already in Vista, it would be relatively easy to prompt users for their civic location the first time location data is needed on a particular network.
  • GPS Intermediate Driver – already rolled into the Windows Mobile platform.
  • Location Information Servers – can be added to the mix later as LIS deployments become prevalent. This is the only one that is GeoPriv dependant.

The civic and geographic schemas have already been delivered by the working group as RFC 4119. There has been an incredible amount of discussion involved in developing a unified schema that can represent civic addresses for anywhere in the world, and this schema should be adopted for consistency. (Do you know the difference between states, regions, provinces, prefectures, counties, parishes, guns, districs, cities, townships, shis, divisions, boroughs, wards, chous and neighbourhoods? They do.)

Who is responsible for delivering this unified location layer?

I keep talking about the browser being responsible for managing all these location sources. Other than the JS layer, all of this infrastructure is client independent, so why don’t we just make the browser a dumb proxy to a unified location service. This service should be a component of the operating system, accessible by software clients (like Skype) and webpages via the browser proxy.

Windows Mobile has already started in the right direction with their GPS Intermediate Driver, however this is only one element of a wider solution.

What do I want?

  1. I want to see a “Location” icon in the settings page of my mobile device, the Control Panel of my Vista laptop and the System Preferences panel of my Mac.
  2. I want the browsers to expose the GeopgraphicLocationProvider object for JS clients. (The start of this specification is online now already.)
  3. I want the browsers to proxy location requests to the operating system store, along with hints like which zone the website is in.
  4. I want the operating system store to be extensible, implementing its own provider model which allows 3rd party developers to supply their own location data from software or hardware services as new discovery mechanisms are developed. We shouldn’t have to wait for widespread adoption before sources are surfaced into the store, and this should be open to software developers as well as driver developers.
  5. I want the operating system store to be accessible by any application, including 3rd party browsers.

How am I going to make this happen?

Dunno.

Right now, I’m documenting the start of what will hopefully be a fruitful conversation. Participate in the conversation.

Still more upcoming presentations

Update 4-Dec-07: The Canberra talks on Dec 20th have been bumped to make way for the Canberra IT Pro + Dev Christmas party instead. The party will be on at King O’Malley’s Irish Pub from 1630 on Tue 11th Dec 2007. (131 City Walk, Canberra City, ACT 2601, Australia). My next talk in the nation’s capital will now be on 20th March 2008.

Through December and January I will be delivering updated versions of my “Utilising Windows Live Web services Today” presentation. It is a hybrid of my Tech.Ed presentation and the content from my Enterprise Mashups talk at Web Directions.

The presentation covers a reasonably high level overview of the technologies that fall under Windows Live, and what APIs you can use to access them. For a welcome change, I actually have really licensing numbers and actually dollar figures to talk about too (none of this “You’ll have to call a sales rep” type talk). Most importantly, it’s all about technologies that are available today and many of which are free too. I’ll also do a number of code demos covering Windows Live Data, Virtual Earth, MapPoint and some ASP.NET AJAX.

Canberra Developer Users Group (Lunch): Thursday 20th Dec, 1230 at King O’Malley’s Irish Pub, 131 City Walk, Canberra City, ACT 2601, Australia

Canberra Developer Users Group (Evening): Thursday 20th Dec, 1630 at Microsoft Canberra, Walter Turnbull Building, Level 2, 44 Sydney Ave, Barton, ACT 2600, Australia

Queensland MSDN User Group: Tuesday 19th February, 1730 for 1800 at Microsoft Brisbane, Level 9, Waterfront Place, 1 Eagle St, Brisbane, QLD 7000, Australia

Hope to see you all there!