Presentations galore (well, 4)

Well, it’s time for me to hit the road again touting the benefits of Virtual Earth and Windows Live technology.

As a bit of a last minute swap, I’ll now be delivering Dave Lemphers’ session at Web Directions this week.

Enterprise Mashups

Enterprise Mashups are a great way to spice up existing investments in SOA and Web Services with new technologies such as maps and Web APIs. In this session, Tatham Oddie, will demonstrate how you can leverage technologies such Virtual Earth and MapPoint Web Services to build a simple mapping solution for visualising customers… and all in 15 minutes!

Thursday 27th Sep, 10:15am in the Microsoft Silverlight Lounge

I’ve also lined up some usergroup sessions to deliver a slightly tweaked version of my Tech.Ed presentation for those who either didn’t make it to Tech.Ed or were still sleeping off the party.

The presentation covers a reasonably high level overview of the technologies that fall under Windows Live, and what APIs you can use to access them. For a welcome change, I actually have really licensing numbers and actually dollar figures to talk about too (none of this "You’ll have to call a sales rep" type talk). Most importantly, it’s all about technologies that are available today and many of which are free too.

Utilising Windows Live Web Services Today

Windows Live represents a collection of opportunities to integrate with a new generation of online services. This session provides an overview of the Windows Live family – outlining the business and technical advantages, the range of integration options that exist for each service, real world examples and live demos.

Newcastle Coders Group: Wednesday 3rd Oct, 1800 at 9 Denison St, Newcastle West, NSW 2302, Australia

Canberra Developer Users Group (Lunch): Thursday 20th Dec, 1230 at King O’Malley’s Irish Pub, 131 City Walk, Canberra City, ACT 2601, Australia

Canberra Developer Users Group (Evening): Thursday 20th Dec, 1630 at Microsoft Canberra, Walter Turnbull Building, Level 2, 44 Sydney Ave, Barton, ACT 2600, Australia

I hope to see many of you around, with as many Virtual Earth, Windows Live, ASP.NET and CSS questions as you can think of.

Thanks again to Coatesy, Finula and the whole DPE team at Microsoft Australia for inviting me along to events like Web Directions, and their continued support for local user groups.

Geographic Proximity Searches in SQL 2005

Disclaimer: I don’t actually know anything about SQL performance. The techniques described below have been gleaned from other sources, seem to work for me, and it makes sense why they would. If you know more about SQL, please correct me. 🙂

Talking to Darren around Tech.Ed today, he expressed a need for a way to do proximity searches in SQL. These are queries along the lines of “give me all the records that talk about locations within 50km of X”. Now, in SQL 2008 this is baked into the product, but for SQL 2005 and below we need to do the maths ourselves.

The general query is relatively simple: calculate the distance-from-X for each row, then filter by the distance. Performing 6 trigonometric operations for every row, on every query is a pretty sure way to kill your average database though.

Instead, we add some persisted calculated columns to our table like so:

CREATE TABLE [dbo].[Locations]
(
    [LocationID] [uniqueidentifier] ROWGUIDCOL NOT NULL CONSTRAINT [DF_Locations_LocationID] DEFAULT (newid()),
    [Title] [varchar](100) NOT NULL,
    [Latitude] [decimal](9, 6) NOT NULL,
    [Longitude] [decimal](9, 6) NOT NULL,
    [ProximityX]  AS (cos(radians([Latitude]))*cos(radians([Longitude]))) PERSISTED,
    [ProximityY]  AS (cos(radians([Latitude]))*sin(radians([Longitude]))) PERSISTED,
    [ProximityZ]  AS (sin(radians([Latitude]))) PERSISTED,

    CONSTRAINT [PK_Locations] PRIMARY KEY CLUSTERED
    (
        [LocationID] ASC
    ) WITH (PAD_INDEX  = OFF, IGNORE_DUP_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]
GO

Basically, we take the reusable parts of our calculation and store them between queries. Storing these precalculated components means that our comparisons can be done using relatively simple maths and only one trig function per row.

Because they are persisted calculated columns, they are only calculated when the row is created or updated. SQL manages this for us.

Finally, here’s a stored procedure to query that table:

CREATE PROCEDURE [dbo].[FindLocationsNearby]
   @CenterLat float,
   @CenterLon float,
   @SearchRadius float
AS
BEGIN
    –Store the radius of the earth (in km so that we can search by km)
    declare @EarthRadius float
    set @EarthRadius = 6378.14

    –Calculate the X, Y and Z axis values for the center point
    declare @CenterX float
    declare @CenterY float
    declare @CenterZ float
    set @CenterX = cos(radians(@CenterLat)) * cos(radians(@CenterLon))
    set @CenterY = cos(radians(@CenterLat)) * sin(radians(@CenterLon))
    set @CenterZ = sin(radians(@CenterLat))

    –Perform the database search
    SELECT    Title,
            Distance = @EarthRadius * acos(ProximityX * @CenterX + ProximityY * @CenterY + ProximityZ * @CenterZ)
    FROM    Locations
    WHERE    @EarthRadius * acos(ProximityX * @CenterX + ProximityY * @CenterY + ProximityZ * @CenterZ) <= @SearchRadius
    ORDER BY Distance ASC
END
GO

This technique was extracted from an Access/SQL 2000 article I found on MSDN.

Tech.Ed 2007 Resources

Here’s a dump of links from my Tech.Ed deck for Friday. I’m posting them now so that I can give everyone just one link to note down… I don’t know what I’m allowed to do with my slides, so you may or may not see them on here later in the week.

Resources

Windows Live Development
http://dev.live.com

Live-in-a-Box
http://codeplex.com/liveinabox

MIX07 Recorded Sessions
http://sessions.visitmix.com

My Blog
(Notes and sample code)
http://blog.tatham.oddie.com.au

Community Resources

Virtual Earth Community Site
http://viavirtualearth.com

Windows Live Community Site
http://viawindowslive.com

Windows Live Developer Forums
http://forums.microsoft.com/MSDN

Managed Wrappers for Windows Live Data

Windows Live Data is the API you use for delegated access to your user’s personal data like contacts. It’s a pretty simple API, however that hasn’t stopped me writing some components for it! Today, I’m releasing them publicly.

Not only do these components make it easier to work with the API, but they also provide an abstraction layer so that as the API develops your application doesn’t necessarily have to.

(Note: This post assumes an understanding for the Windows Live Data API. If you’ve never touched it before, read this first.)

First up is the PermissionRequestHyperLink control. Placing this on your page gives you a nice designer experience for building those yucky URLs and setting all the right flags.

A basic request looks something like this:

<live:PermissionRequestHyperLink id=”PermissionRequestHyperLink1″ runat=”server” Permission=”LiveContacts_ReadOnly” PrivacyUrl=”~/WindowsLive/PrivacyPolicy.aspx” ReturnUrl=”~/WindowsLive/ResponseHandler.ashx”>Permission Request</live:permissionrequesthyperlink>

And gives you designer experience like this:

image

image

Next up is the response handler base class. Start by adding a new ‘generic handler’ to your project:

image

Change the generated class to inherit from PermissionResponseHandler instead of IHttpHandler, then implement the ProcessResponse and ProcessFailure methods like so:

public class ResponseHandler : PermissionResponseHandler
{
    protected override void ProcessResponse(HttpContext context, PermissionResponse response)
    {
        //Do something here like storing the token for future use
        //response.DomainAuthenticationToken
        //response.OwnerHandle
    }

    protected override void ProcessFailure(HttpContext context, PermissionResponseCode responseCode)
    {
        //Perform some nice handling here
        //responseCode
    }
}

How easy is that!

You can grab the code from http://svn.fueladvance2.com/FuelAdvance.Components/trunk/ (username: anonymous). You’ll find the components in the FuelAdvance.Components.Web.WindowsLive namespace.

If you’re using Subversion yourself, remember that you can configure this as an svn:external and then you’ll always be running the latest version.

Next up, I’ll probably be releasing some managed wrappers for the Windows Live Contacts API.

Update 9/8/07: Change SVN link to point at the solution instead of the project.

CSS Tip: Hover effects done properly

(This post is also available in Spanish, translated by Maria Ramos from Webhostinghub.com)

I was checking out Readify’s cool new TFS Now! offering this evening. Under the hood, they’ve solved quite a number of technical challenges which you can hear about during DEV316 at Tech.Ed next week.

This post was triggered by something that jumped out at me when looking at their consumer facing site though… In reality, you’ll find what I talk about here on almost every website that uses hover effects. I’ve chosen to use the TFS Now! site as a the example here because it’s a fairly simple real world page that clearly demonstrates the issue.

On their homepage, you’ll notice these nice big bananas:

image

When you hover over one of them, you get some feedback:

image

All good so far. The CSS used to achieve this is rather simple:

div#pageContainer div.homepage-sections div.benefits a {
    background:transparent url(../yellow-button.png) no-repeat scroll 0%;
}

div#pageContainer div.homepage-sections div.benefits a:hover {
    background:transparent url(../yellow-button-hover.png) no-repeat scroll 0%;
}

Firing up a copy of Fiddler shows pretty clearly that IE loads both the normal image, and hover image as part of the page load:

image 

When you mouse-over the button, IE has the hover image cached so you get the rollover effect instantly.

This is where the problem starts…

When you mouse-off, IE re-loads the normal image. Even though the image is in the client side cache, IE still decides that it needs to call the server to check for any updates. During this time, you get a noticeable flash where there is no image displayed at all. You can see the calls in Fiddler, and the Flash is noticeable even on most localhost tests.

image 

The Solution

It took me a few years of web development to come across a solution for this that didn’t involve the messy pre-loaders and ugly JavaScript hover scripts prominent throughout the 90s. In the end, the solution was remarkably simple and provided a number of other benefits. They’re called sprites.

First up, combine the two image assets into one. For this example, the buttons were 215x98px so I placed them on a 215x196px canvas in Photoshop:

image 

Even if you have unused pixels in one of the grid spaces, keep them both the same size.

This can now be applied using even less CSS then we had before:

div#pageContainer div.homepage-sections div.benefits a {
    background: transparent url(../yellow-button.png) no-repeat left top;
}

div#pageContainer div.homepage-sections div.benefits a:hover {
    background-position: left -98px;
}

You’ll notice that in the hover rule, instead of changing the image I’m now just sliding the image further up within the element.

Loading both styles in the one resource solves the flashing issue, as the image is never actually changing – it’s just moving. We also get some instant IO savings as a result:

image

While 1.7KB might not seem like much, you need to remember that in this scenario that equates to an almost 30% saving. And that doesn’t even include the 798 bytes of HTTP headers we saved too.

Extending this idea and including all six states (two states each for three buttons) as a 3×2 grid gives us even better savings.

The next thing to remember is that you can only have two simultaneous HTTP connections to the same server. Considering the homepage of TFS Now! currently triggers a total of 25 requests, there’s a lot of blocking going on there. Just by optimizing these buttons, we could reduce that by 20%.

The Benefits

All up, with this simple change we:

  • Removed the blank flash on mouse-out
  • Reduced data transfer by ~30%
  • Reduced server requests by 20%, and thus noticeably improved page load time
  • Removed almost duplicate code from the CSS

At this point, I’d say it isn’t worth changing, however it’s a useful technique to keep in mind next time you’re building out a site.

Particularly in cases where you have lots of similar-sized images, such as toolbar icons for a web-interface, you can get significant benefits to both real and perceived performance by loading them as a single image rather than as a series of blocking requests.

Update: Following up from Mitch’s concern that different browsers may have issues moving the image, I can confidently reply that this technique has worked everywhere I’ve ever tried to use it. At Squeeze Creative, everything is tested in IE6 PC, IE7 PC, FF1.5 PC, FF2.0 PC, S3.0 PC, O9 PC, S1.3 Mac, S2.0 Mac, S3.0 Mac, FF1.5 Mac, FF2.0 Mac, and O9 Mac. All up, that’s a pretty exhaustive set of tests. Historically we have tested even more platforms (like FF1.0) and this technique stood up pefectly then too. I focussed on IE7 throughout the post as it shows the clearest ‘flash’, however the problem and the solution are relevant to all browsers.

— 

Was this post useful for you? Let me know!

Which time zone should I set for my server?

This was the question I was faced with last week. I was configuring a VPC image in Sydney, that I was later going to load on to a physical box in San Diego, then use it from anywhere in the world. So which time zone to use?

I decided upon UTC, and now, the more that I think about it, I don’t know why all servers aren’t configured that way.

Advantages to using UTC:

  • No weird shifts/jumps around daylight savings
  • Accidental usage of DateTime.Now in server-side code returns that same value as DateTime.UtcNow anyway
  • Migrating applications and their data, or entire VMs across physical locations becomes easy

Disadvantages to using UTC:

  • Everything is stored in UTC, and thus hard for most humans to deal with mentally. The solution: tools like the Windows Event Viewer and SQL Management Studio should allow you to apply a time zone offset to visible results.

My recommendation: Run all your servers in UTC (that’s “GMT – Greenwich Mean Time” with “Automatically adjust clock for DST” turned OFF).

Maybe Adam can add this to his standards? 😉

Strange server behavior – huge I/O other count for csrss.exe (SOLVED!)

I can’t say this feels right to me:

image

Notice the 603 terabyte value in the “I/O Other Bytes” column for csrss.exe. Oh yeah, and the box has only been up for 24 hours. The number seems to climb by 50 or 60 GB a second when I’m connected via RDC. If I logoff then back on, the count starts again.

I’ve tried Googling it, but found no solutions.

Anybody got any ideas? It doesn’t actually seem to be causing a problem, it’s just out of the ordinary.

The box is a 1.8GHz Core2 Duo running Windows Server 2003 R2 Standard x64 Edition.

Update – I (think) I found the solution!

Following some suggestions in this post’s comments and on Google that this might be a virus, Edward suggested I fire up a copy of SysInternals Process Explorer on the server. I’ve used the SysInternals tools in the past, but not being a server guy by trade meant that I’d forgotten about them.

After finding the process, I was quickly comforted that this wasn’t a virus. The signature was verified, it was executing from the correct path (system32) and it was running under NT AUTHORITY\SYSTEM where I expected it to.

image

If we recall the original symptoms, the count was reset if I logged off RDC and back on again. This didn’t really prove much other than the fact that the process restarted at login. Swapping to the “Performance Graph” tab is Process Explorer did however show some interesting results.

image 

Whenever I minimized my RDC window, it stopped requesting data and the I/O bytes graph was stagnant. As soon as I brought the RDC window back up and started interacting with it, the values skyrocketed. My screenshot doesn’t show it, but that graph was peaking to 80GB!

The end result would seem to be: “that’s just how much stuff is going on to support RDP”.

To me that’s an incredible amount on information being shuffled around and process, but I guess this is why the actual data on the wire is so efficient. Unless anything else comes up, I’m happy to file this in the “amazing but normal” pile.

See you at TechEd 2007

Assuming nobody at Microsoft Australia rolls over in bed tonight and changes their mind, I’m pleased to announce that I’ll be presenting at TechEd Australia this year.

The official session blurb (missing a zillion ™ and ® symbols) is:

Utilising Windows Live Web Services Today

Windows Live represents a collection of opportunities to integrate with a new generation of online services. This session provides an overview of the Windows Live family – outlining the business and technical advantages, the range of integration options that exist for each service, real world examples and live demos.

My plan is to focus on:

  • Live ID
  • what’s available now
  • an overview of how to integrate it with demo (partly pre-cooked)
  • why to integrate (stats on number of users)
  • Spaces
    • what integration options exist
  • Messenger
    • what integration options exist (activities / bots)
    • why to integrate
  • Contacts
    • control vs. API
    • demo of the Windows Live Data API
    • I’ve been building out a library in this space which makes it easy to call from.NET – will hopefully be releasing this open source soon
  • why to integrate (stats on number of users)
  • Virtual Earth
    • less here – John + Bronwen will cover this in their session
    • overview + show some of the partner sites already out there

    Other topics that could also be covered:

    • Live Toolbar Custom Buttons
    • how to build them
    • I’ve been building out a library in this space which makes them really easy to publish from ASP.NET – will hopefully be releasing this open source soon too
  • Live.com Gadgets
    • been around a while so probably not do much more than mention them

    If you’re interested in know more about the Windows Live family, or don’t really understand what live is about, come along and check it out. If there’s anything you can think of that you’d especially like to see, drop me a note and I’ll try and get it in.

    See you on the Gold Coast!

    ASP.NET AJAX … continued

    Last week I was angry. Today I’m disappointed.

    ASP.NET as a platform rocks, and being the Microsoft zealot I am, we chose to use ASP.NET AJAX behind visitscandinavia.com.au. It’s no longer in beta. It supports IE6, IE7, Firefox 1.5 and Safari 2.0. Good to go – right? Unfortunately, that hasn’t been our experience.

    Luckily, the actually AJAX requirements of this project are very minimal (only two parts of the whole site), but the headaches we’ve experienced along the way are starting to outweigh the benefits. I trusted the framework, and on tight timelines it has come back to bite me in the ass. Even the bits that are supposed to work for everyone don’t seem to work for me (see “Any Browser + UpdatePanel = Nothing” below). This made me angry.

    Here are some of my thoughts at the moment …

     

    Safari 1.3.2 = Neglect

    No support for Safari 1.3.2 at all. This was documented, so this is ok. Considering the number of alternatives on the market that do work with Safari 1.3.2, this disappoints me, but they documented it so it’s ok. For this project, Safari 1.3.2 is a very small sliver of the userbase. I think there should be at least some partial support here for things like the client side library – Sys.UI.DomElement, etc.

     

    Safari 2.0 = Boom

    The first problem was pretty major for us. As I documented in my previous post, the framework failed to do anything useful (like actually load) on Safari 2.0 through 2.0.3. The word I’m hearing from ASPInsiders is that the documentation is actually wrong and should read 2.0.4 not 2.0 – I’m waiting to hear the final response on this. Really, if the documentation gets fixed this could be ok, as most Mac users are up-to-date with minor builds. I found this especially painful though, as one of the first things I did when trying to debug it was start with a blank project. When even that failed, it wasn’t a great confidence boost.

     

    Safari 2.0.4 + <fieldset /> + border:none; + ModalPopupExtender = Big Boom

    This bug requires a pretty specific set of elements to be in place, however I’m still surprised it wasn’t picked up earlier. In most of the examples where I’ve seen a ModalPopupExtender used, it is displaying some nice little gadget like this:

    Currency Converter Gadget

    Considering the gadget, like most others, contains a group of related fields the contents of the gadget should be wrapped in some semantic HTML like so:

    <fieldset>
        <legend>Currency Converter<legend>
        ... form elements ...
    </fieldset>

    Now, the rendered appearance of a fieldset is really rather ugly and also inconsistent between browsers. Lets fix that with some CSS:

     fieldset { border: none; }

    Trigger the updated panel and big boom – Safari itself just crashed after ending up in an infinite loop of updateScrollInfoAfterLayout() calls in the com.apple.AppKit library. This is 100% replicable for me on Safari 2.0.4.

    image

    The workaround is currently to use this rather ugly CSS instead:

    fieldset { border: solid 1px transparent; margin: -1px; } /* Needs to be a transparent border to avoid crashing Safari 2.0.4 — border:none in conjunction with ajaxToolkit:ModalPopupExtender blows up the browser. */
    * html fieldset { border: none; }

    This could be a Safari problem, it could be a bug in the extender that Safari isn’t handling very well. Either way, it’s pretty broken.

     

    Any Browser + UpdatePanel = Nothing

    This one’s the cracker.

    To me, this is pretty basic code:

    <asp:UpdatePanel runat=”server” ID=”UpdatePanel”>
        <ContentTemplate>
            <asp:Label ID=”Proof” runat=”server” />
            <asp:Button ID=”Trigger” runat=”server” OnClick=”Trigger_Click” />
        </ContentTemplate>
        <Triggers>
            <asp:AsyncPostBackTrigger ControlID=”Trigger” EventName=”Click” />
        </Triggers>
    </asp:UpdatePanel>

    Apparently it’s there’s something wrong with my particularly little setup, because it’s rendering out broken JS that the browsers are refusing to even parse:

    <input type="submit" name="ctl00$ContentPlaceHolder$Trigger" value="" onclick="javascript:WebForm_DoPostBackWithOptions(new WebForm_PostBackOptions(&quot;ctl00$ContentPlaceHolder$Trigger&quot;, &quot;&quot;, true, &quot;&quot;, &quot;&quot;, false, false))" id="ctl00_ContentPlaceHolder_Trigger" />

    Notice the escaped “&quot;” within the onclick parameter?

    None of my client-side code can be the culprit as the rendered output is just plain wrong.

    I’ve got working instances of the UpdatePanel in plenty of other places (even some already within this same application), so there’s just something quirky in this particular setup. I’m still trying to debug this one …

     

    ASP.NET AJAX Control Toolkit

    Unfortunately the control toolkit experience on a Mac is pretty pathetic, however it’s pretty clear why. The Control Toolkit is largely developed by people external from Microsoft – people like Paul Glavich. Having vented to Paul quite a bit about all this lately, it became clear that him and most of his co-developers don’t actually have access to a Mac to test any of their bits – they just have to poke blindly in the dark and hope it all works. My experiences with a such an approach over the years haven’t proved very successful – there are even noticeable differences between the same version of Firefox on two different platforms. In the end I spent $80 each for two G4 Mac towers. They hum away on my local network controlled by VNC, one running Mac OS X 10.3.9 with Safari 1.3.2 and the other running Mac OS X 10.4.9 with Safari 2.0.4. When I’m out of the office, I can VNC over the the VPN.

    I personally don’t understand how any professional web developer can justify not having such a setup. In the mean time though, I think the core ASP.NET AJAX team seriously needs to investigate some kind of hosted testing setup like this, with logins issued to the Control Toolkit developers. Obviously it’s a lot easier if you can point it across the local LAN to your development IIS, however a hosted setup like this accessing a CI output would still make a substantial dent in the number of Mac related bugs.

    Right now I’m trying to write a patch for the CalendarExtender that puts it on the right z-index in Safari… Everything else in the extender works perfectly, you just can’t always see it.

     

    The People

    I do have to say, Scott Gutherie continues to amaze me as a person. He seems to be responsible for every useful chunk of developer technology getting released from Microsoft at the moment, yet still manages time to actually connect with the community, listen then actually help people out. This is a rare trait across any workplace, and requires an incredible amount of dedication.

    I still love your work Scott, I just wish it’d love me back from time to time. 🙂

     

    (More to come … Sorry for the delay in getting this follow-up post out; I was busy vomiting between extended periods of sleep.)

    Debugging SSL/TLS problems in .NET

    In my previous post I discussed  some issues I discovered with SSL client certificates.

    For this application, I’m dealing with PayPal. Rather annoylingly, if your client certificate doesn’t check out, they don’t even bother sending an error – they just drop the TCP connection mid-SSL-handshake. Depending on how fast the connection loss is realised, .NET can return a few different errors; none of which are actually much use when it comes to debugging.

    Luckily, System.Net (as with most other areas of the framework) has wonderful tracing capabilities. Particularly with a complex process like an SSL handshake, these capabilities become critical to debugging.

    Better yet – they’re incredibly easy to use. Just add a block like this to the end of your app.config!

    <system.diagnostics>
      <trace autoflush="true"/>
      <sources>
        <source name="System.Net" maxdatasize="1024">
          <listeners>
            <add name="TraceFile"/>
          </listeners>
        </source>
        <source name="System.Net.Sockets" maxdatasize="1024">
          <listeners>
            <add name="TraceFile"/>
          </listeners>
        </source>
      </sources>
      <sharedListeners>
        <add name="TraceFile" type="System.Diagnostics.TextWriterTraceListener"
          initializeData="trace.log"/>
      </sharedListeners>
      <switches>
        <add name="System.Net" value="Verbose" />
        <add name="System.Net.Sockets" value="Verbose" />
      </switches>
    </system.diagnostics>

    Now, run your app again and take a look at the wonderful lines being added to your bin\Debug\trace.log file.