Patterns-Based Silverlight Development – Part I – Getting Started

Introduction

During the summer I put together a session on Patterns-Based Silverlight Development that we delivered across the West Region as part of MSDN events.  The session was structured around building a Silverlight application from the ground up that illustrated the use of the following design patterns: 1) Repository, 2) Pipeline, 3) Service Agent and 4) Model View ViewModel.  The goal was not to build a final productionized version of the application, rather illustrate how one might take advantage of these patterns in a Silverlight application.  I will follow the same plan here, albeit in more detail. 

Environment and Demo Choices

Development Environment:

• Visual Studio 2008 sp1
• .NET Framework 3.5 sp1
• Silverlight 3
• The Silverlight 3 tools for Visual Studio 2008 sp1
• The Silverlight 3 Toolkit – July 2009 drop

* There is no reason you wouldn’t be able to follow along using our free tools

Demo Choices:

Whenever you put together a sample application, you have to make some choices: Language, Web Services Stack, REST vs. SOAP, data access strategy, etc.  The following is a listing of some of the choices I made for this sample to start with (I may swap in and out as things progress):

• Language: C#
• Data Access: LINQ to SQL – This was a random decision.  I had done a bunch of work with the EF lately.  Thought I’d give it a go.  I may swap this layer out later for the EF when .NET 4 hits Beta 2
• Web Services: WCF, SOAP, New custom Silverlight binding

Setting up the Project

I set up Visual Studio to use solution folders to clearly illustrate what is server code and what is client code.  Here is what it will look like eventually:

We are going to start with creating a blank solution and add the 2 solution folders.  We will then add the HelpDesk.Data project to the Server solution folder.

1. Create a blank solution
   • On the File menu, select New and then click Project
   • Under Project types, select Other Project Types and then select Visual Studio Solutions
   • Select Blank Solution
   • Name it HelpDesk
2. Add the ‘Client’ and ‘Server’ solution folders
   • Right-Click on the Solution
   • Choose Add -> New Solution Folder -> Name it ‘Client’
   • Repeat for ‘Server’
3. Add the HelpDesk.Data class library 
   • Right-Click on the Server solution folder
   • Choose Add -> New Project
   • Under Windows -> Choose Class Library
   • Name it HelpDesk.Data
   • Delete Class1

Data Model

I am using a VERY simple data model I crufted up.  There is a Ticket table with a SeverityLevel Lookup table.  See below:

Building the LINQ to SQL Model

I assumed that this part of the application would take under 5 minutes.  As it turns out, that was a poor estimate.  The issue I had surrounded serializing the generated L2S objects.  It took me hours to realize that the Visual Studio designer did not support configuring the serialization in the way I needed it (at least I couldn’t figure it out).  Truthfully, I could have skipped all of that pain and manually decorated the classes with the appropriate attributes up front, but I wanted to use the designer, if for no other reason than it would save me from re-decorating the classes every time I re-generated the classes.  In the end, I was able to get partway there with the designer, but did have to make a change in the code, as well.  Let’s walk through the process:

Adding the Tables to the Designer

1. Add a LINQ to SQL named HelpDesk.dbml to the HelpDesk.Data project. 
   • Right-Click the HelpDesk.Data project –> Add –> New Item
   • Choose Visual C# Items –> Data –> LINQ to SQL Classes
   • Name it HelpDesk.dbml
2. Add the ‘Ticket’ and ‘SeverityLevelLookup’ tables to the L2S designer
   • From the ‘Server Explorer’, expand the Tables Node from the HelpDesk Database
   • Drag the ‘Ticket’ and ‘SeverityLevelLookup’ tables onto the designer
3. Make some simple adjustments to the model
   • Rename the ‘SeverityLevel’ property to ‘Level’
   • Rename the ‘SeverityLevelLookup’ Class to be ‘SeverityLevel’

   

Configure the Classes to be Serializable

By default the “Serialization Mode” for the Model is set to None:

What does that mean?  Well, the default serializer for WCF is the DataContractSerializer.  When it was first released (Fx 3.0), it required you to apply the DataContractAttribute to classes you wanted to serialize and to apply the the DataMemberAttribute to fields and properties you wanted to be serialized.  WCF 3.5 sp1 lifted this restriction, allowing for the serialization for POCOs (Plain Old C# CLR Objects).    The Serialization Mode of None simply indicates that the designer will not add any DataContractAttributes or DataMemberAttributes to your classes or properties, respectively.

Do I really need to decorate my classes?  I mentioned POCO serialization support.  The reality is that if you have any relationships between your classes, you will need to decorate your classes.  So the first step is to set this Serialization Mode to Unidirectional.  Incidentally, you can click on the properties pane and then click on any white space in the designer to view the properties for the DataContext.

This means that the classes in the model will be decorated with a DataContractAttribute and some public properties will be decorated with a DataMemberAttribute.  So which properties are decorated.  To start with, all, public primatives will be decorated.  What we have to now understand is which complex types will be decorated.  This is where the term ‘Unidirectional’ takes on importance.  Essentially, if you take a look at the generated classes, the SeverityLevel class contains a collection of Tickets and the Ticket class has a SeverityLevel object.  Hopefully, you can see that if we serialized both sides of the relationship, we will create an endless loop.  What Unidirectional then means is that we will support serializing one side of the relationship.  By default, by choosing Unidirectional, we will serialize the parent class and the child collection.  So, in this case, if we serialize the SeverityLevel class, we will by default serialize it’s collection of Tickets.  Have a look at the designer (I clicked on the relationship arrow in the designer):

If you take a look at the HelpDesk.designer.cs file, you can see the actual code that gets generated.

Notice below that the Tickets collection is decorated with a DataMemberAttribute in the SeverityLevel class.  Also notice that the SeverityLevel property is not decorated in the Ticket class.

This default behavior does not suit the needs of my application.  It does not make sense to fetch a particular SeverityLevel (like Low, Medium or High) and with it a collection of Tickets.  It does, however make sense to return a SeverityLevel type when fetching a Ticket.  I need to change the direction of serialization.  This is the point where spent all of my time.  I was not able to find the appropriate settings in the designer to accomplish this task.  Clearly, I could have simply taken the 30 seconds to delete the DataMemberAttribute above the Tickets property and added one above the SeverityLevel property.  However, as I stated, every time the code was regenerated, I would have to remember to make this change.  (*** It is highly likely that there is a way to do accomplish this in the designer.  If you know how, please leave me a comment and I will update this post ***).

I was able to get partially there in the designer.  By setting the Child property to ‘Internal’, I was able to prevent the Tickets collection from being decorated:

That left decorating the SeverityLevel property in the Ticket class.  I simply did that manually.  Now my LINQ to SQL model is ready to go.

What’s Next

Tomorrow I will build a repository layer and illustrate the value of that pattern.

Introducing Parallelism into your programs

Overview

A little over a month ago, I volunteered to develop a session for the Windows 7 / Windows Server R2 launches that are being delivered across the country and beyond  The session I worked on I affectionately call “An Introduction to Parallel Programming”.  In that session I wanted to illustrate various approaches you can take when introducing parallelism to your applications: 1) Fine-grained parallelism, 2) Structured parallelism  and 3) PLINQ.

The approach I chose was to start with an a sequential application and parallelize it using each of the 3 approaches.  The application I wrote simply creates a thumbnail image for every image in a specific directory.  This post provides a link to a screencast I did on this subject, provides a brief overview of each approach and illustrates how my sample program looks with each approach.

Screencast

Click here to view the screencast on channel9

Approaches for introducing parallelism into your applications

Fine-grained parallelism (aka imperative task parallelism)

This approach provides you with the most control.  Accordingly it requires the most work.  The following are some attributes of this approach:

• Express potential parallelism via expressions and statements that take the form of lightweight tasks
• Affords fine-grained control
• The semantic is similar to how threads and the threadpool work today
• You tend to use it when the other approaches do not suit your problem

Structured parallelism (imperative data parallelism)

Structured parallelism offers a higher level of abstraction.  With this approach you tend to think in terms of parallelizing ‘blocks of code’.  Daniel Moth did a great job of explaining this in his PDC 08 talk this way:  One thread goes into that block. Within that block, many threads can execute the code within the block and one thread comes out.  To this end, we provide a ‘Parallel’ type that allows you to easily parallelize for loops, foreach loops and arrays of actions.  The following are some attributes of this approach:

• Mechanisms used to express common imperative data-oriented operations
  • For loops
  • Foreach loops
  • Invoke
• Think in terms of blocks of code
• Parallelize loops

PLINQ (Declarative Data Parallelism)

As a declarative approach, you simply define what you want done, not how you want it done.  You do this with PLINQ,  an implementation of LINQ To Objects.  PLINQ replaces LINQ to Objects extension methods with new ones that internally use parallelism.  In many cases, it is as simple as adding a call to the AsParallel extension method to your query.  The following are some attributes of this approach:

• Implementation of LINQ-to-Objects that executes queries in parallel
• Express what you want to accomplish, rather than how you want to accomplish it
• Minimal impact to existing queries

Illustrating the 3 Approaches

In order to illustrate these 3 approaches, I decided to take a simple console application that was implemented sequentially and convert it to use each approach.  The sample application simply creates a thumbnail image for each image in a specified directory.  The process I will take is to show you the code for the sequential implementation, as well as each approach to parallelism.  I will then explain the code and lastly show you the timings for each approach.  I am running this on a dual core laptop using the Beta 1 of Visual Studio 2008 and .NET 4.  I will run the code 3 times for each approach. 

* Please note that the results in this post should not be used to evaluate which of the 3 parallel approaches is generally more performant than the others.  That will be scenario driven.  The numbers should simply be used to illustrate the relative performance gains the parallel approaches provide over the sequential approach.

The Sequential Version of the application

The Code

The following is the serial version of the application:

    static void Main(string[] args)
    {
        Stopwatch sw = Stopwatch.StartNew();

        CreateThumbnails();

        Console.WriteLine("Elapsed = " + sw.ElapsedMilliseconds.ToString());

        Console.ReadLine();
    }

    private static void CreateThumbnails()
    {
        string[] pics = Directory.GetFiles(imageDir, imageExt);

        foreach (string pic in pics)
        {
            CreateThumbnail(pic);

            Console.WriteLine(pic + TID);
        }

        Console.WriteLine("Done");
    }

Code Summary

As you can see, in CreateThumbnails, we create a string array of image paths.  We then iterate over that array, calling CreateThumbnail for each item, as well as writing out the path and the thread id of the executing thread to the console.  Lastly, we write out ‘Done’ to the console when complete.  The implementation of CreateThumbnail is relatively unimportant to the theme of this post.  It uses the Bitmap and Graphics types from the GDI+ libraries to create a thumbnail image (max height / width of 128 PX).  If you are interested, you can download the sample code here.

Performance

Fine Grained Parallelism

The Code

   1:      private static void CreateThumbnails()

   2:      {

   3:          string[] pics = Directory.GetFiles(imageDir, imageExt);

   4:   

   5:          var tasks = new Task[pics.Length];

   6:          for (int i = 0; i < pics.Length; i++)

   7:          {

   8:              tasks[i] = Task.Factory.StartNew(state =>

   9:              {

  10:                  CreateThumbnail((string)state);

  11:                  Console.WriteLine(state + TID);

  12:              }, pics[i]);

  13:          }

  14:          Task.WaitAll(tasks);

  15:   

  16:   

  17:          Console.WriteLine("Done");

  18:      }

Code Summary

The first thing you might notice is that this code is a bit more involved.  If you watch the screencast I did on Channel9 on introducing parallelism into your applications, I describe in more detail how we get to this code from the sequential implementation.

Again, the first thing we do is to create a string array of image paths.  The next thing we do (line 5) is to create an array of Tasks (the Task is the type we use to implement fine grained parallelism).  We will need this array later.  Next we iterate i number of times (once for each path in the array of image paths).  Now comes the interesting part.  On line 8, we call Task.Factory.StartNew, assigning the result to the appropriate item in the Task array.  StartNew is a helper method that is analogous to creating a Task and then calling Start.  In code terms, it is similar to doing the following, yet a bit more efficient:

            tasks[i] = new Task(state =>
            {
                CreateThumbnail((string)state);
                Console.WriteLine(state + TID);
            }, pics[i]);
            tasks[i].Start();

You will notice that we are using an overload of StartNew where we explicitly pass the state into the Task using the state object (pics[i] is passed in line 12).  This allows us to ensure that the Task is, in fact, working on the image path that we intended it to.  If you are new to this, that may sound confusing.  Let’s re-write the code for a moment to illustrate what may happen if we didn’t use the state object (or a proper closure):

        string[] pics = Directory.GetFiles(imageDir, imageExt);

        foreach (string pic in pics)
        {
            Task t = Task.Factory.StartNew(() =>
                CreateThumbnail(pic));

            Console.WriteLine(pic + TID);
        }

At first glance, this code doesn’t appear to have any issues.  However, to quote Inigo Montoya from The Princess Bride “I do not think it means what you think it means”.  In this example, the pic variable is not captured correctly in the closure.  What this ultimately means is that by the time the Task wakes up and begins calling CreateThumbnail on the pic variable, the pic variable may no longer have the original value from the outer loop. 

Getting back to the proper implementation, you will notice that in lines 10 and 11 both CreateThumbnail and the Console.WriteLine call are operating on the state object.  The last thing to point out is the call to Task.WaitAll in line 14.  As you can see, we pass the Task array we created in line 5 and assigned to in line 8.  This call will ensure that all of the Tasks will complete prior to the call to write ‘Done’ to the console.

Performance

As you can clearly see, this is a dramatic performance increase.  This is due to the fact that we were able to distribute the work across the 2 cores on my machine, instead of just one like the sequential implementation.  However, the code was a bit involved.  Let’s take a look at a couple of other options that provide a higher level of abstraction.

Structured Parallelism

The Code

   1:      private static void CreateThumbnails()

   2:      {

   3:          string[] pics = Directory.GetFiles(imageDir, imageExt);

   4:   

   5:          //foreach (string pic in pics)

   6:          Parallel.ForEach(pics, pic =>

   7:          {

   8:              CreateThumbnail(pic);

   9:   

  10:              Console.WriteLine(pic + TID);

  11:          });

  12:   

  13:          Console.WriteLine("Done");

  14:      }

Code Summary

The first thing you will notice is that this code is much simpler.  In fact, it is semantically similar to the sequential implementation.  Line 5 contains the original foreach construct that I commented out.  I replaced it with a call to Parallel.ForEach in line 6.  Notice how similar they are.

The ForEach method takes the IEnumerable you want to iterate over as its first argument and a delegate as the second.  The Action delegate gets run for each iteration of the loop.  In our case, we pass a lamda that calls CreateThumbnail and Console.WriteLine.

Performance

Here we see similar performance increases over the sequential solution.

PLINQ

LINQ Code

   1:      private static void CreateThumbnails()

   2:      {

   3:          string[] pics = Directory.GetFiles(imageDir, imageExt);

   4:   

   5:          var output = pics.Select(pic =>

   6:          {

   7:              CreateThumbnail(pic);

   8:              return pic + TID;

   9:          });

  10:   

  11:          foreach (var line in output) Console.WriteLine(line);

  12:   

  13:   

  14:          Console.WriteLine("Done");

  15:      }

PLINQ Code

   1:      private static void CreateThumbnails()

   2:      {

   3:          string[] pics = Directory.GetFiles(imageDir, imageExt);

   4:   

   5:          var output = pics.AsParallel().Select(pic =>

   6:          {

   7:              CreateThumbnail(pic);

   8:              return pic + TID;

   9:          });

  10:   

  11:          foreach (var line in output) Console.WriteLine(line);

  12:   

  13:   

  14:          Console.WriteLine("Done");

  15:      }

Think back to when you were a kid.  Remember the game where you were shown 2 pictures and you were tasked with identifying the differences between them.  In that spirit, the only difference between the LINQ and PLINQ code is the call to ‘AsParallel’.  By simply adding the call to this extension method, you have introduced parallelism into your solution (see description above).

Performance

Sample Code

You can download the sample application for introducing parallelism into your applications here (you will have to register and login first).

Acknowledgements

I would like to thanks Steven Toub, Rick Brewster, Huseyin Yildiz, Chris Dern and Aaran Layman for their help answering my questions and offering guidance.  Further, it should be noted that this sample was inspired by and was based upon Daniel Moth’s PDC 2008 talk.

Summary

As you can see, there are various approaches to introducing parallelism into your applications, each with their own pros and cons.  Some provide greater control (Fine-Grained), while others provide a simpler model (Structured, PLINQ).  Hopefully one of them will meet your needs and allow your application to take advantage of the hardware it is running on.

Taking My Medicine

I woke up this morning from a terrible dream.  A dream where I had punted a presentation in front of all of my peers.  Laying there in bed, I started thinking to myself ‘how could I punt a presentation?  I’ve done hundreds of presentations in front of tens of thousands of people.  It must have been a dream’.  Or was it?  I crawled out of bed, walked across the hall into the office and cracked open my laptop.  Much to my dismay, when I looked down, there it was.  The residue from a single tear.  Not just any tear.  The presenters tear.  The purest of all tears.  A tear that can only be shed after punting a presentation in front of one’s peers.

Here is my story…

The Situation

I was at a team offsite that was being held at the Hard Rock Hotel in San Diego.  Prior to the trip, my boss sent along a note to all evangelists asking them to be prepared to do a 5 minute presentation to the team.  So, who was the team?  Along with me were all of my peers, my boss and his peers, his boss and, yes, his bosses boss – or as I call him, my great-grandboss.  The general feel you should be getting here is that there were more than a few people in the room who could fire me.  Oddly, I still can’t decide who it’s worse to punt a presentation in front of: a) senior management or b) a room full of evangelists who are going to make your life miserable for the foreseeable future.  I suppose time will tell.

I digress.  The note from my boss actually read that we should be prepared to do a 5 minute presentation on a topic that we are passionate about.  I decided upon .NET RIA Services.  I should be clear here that I am not new to .NET RIA Services.  In fact, it is now sitting on top on ADO.NET Data Services which I am very familiar with and ADO.NET Data Services sits on top of WCF REST with which I am intimately familiar.  I should also note that I have presented on .NET RIA Services dozens of times and wrote ALL of the code that I was going to present.  Lastly, I should mention that I am not the type of guy to “wing it”.  I spent a good deal of time preparing for this, ensuring that I could get my message across in 5 minutes (not an easy task, by the way).  In other words, I was ready for success.

The presentations were split across 2 days.  7 or so of my peers went on day 1.  Each one made himself and the team proud.  There may have been a minor glitch or 2, but they were minor – nothing like what was about to occur on day 2.  Anyway, at the end of the day 1 presentations, our GM stood up, thanked all of the presenters and then said those fateful words… something to the effect of ‘you might consider adding a little flair to your demos’.  As my friend Jason Mauer would say – enter ominous music here.

The Mistake

There are certain rules that you should never break.  I’m not talking about wimpy rules like don’t go swimming for a half hour after you eat.  I’m talking about rules which, if you break, there will be epic consequences.  Think Bobby Brady and “mom always said don’t play ball in the house” or Maverick and “never leave your wingman”.  Well there are a few of those for presenters.  One such rule is “never change your presentation just before your present it”.  Hi, I’m Rob and I broke that rule.

I wanted to add a bit of flair to my demo.  What I did was to add a simple animation to my XAML.  I wanted to make a DataForm spin in a circle (using perspective 3D) every time a different record was selected in the DataGrid.  It seemed harmless enough.  All in all, it was 9 lines of XAML and 4 lines of code.  The change was so insignificant, I thought “what could go wrong?”.

The Demo

Things started out ok.  I felt relaxed and was ready to go.  I started out explaining the context for the presentation.  I got into the demo and added the DomainService (if you are not familiar with .NET RIA Services or some of the other technical mumbo-jumbo I discuss a bit later, it is not really necessary to get the rest of this story – however, I do suggest looking into it).  The next part of the demo was to add a DomainDataSource control into the XAML – the same XAML I added to animation to.  Unfortunately, I opened up the wrong XAML file.  Had I not made the “simple change” and added the animation, I would likely have noticed that I opened the wrong page – but I did…  I did make that change.  It is kind of hard to describe what happened next.  I suppose the best way to describe it is I had a mental blue screen.  My entire memory dumped.  Things started getting all slow.  That is, everything but the 5 minute countdown timer.  I think it actually sped up. 

I did finally figure out that I had opened the wrong page, so I closed it and opened the proper page.  However, somehow, the effect of those extra 9 little lines of XAML on my now weakened intellect were devastating.  That page (that I had authored myself less than a week ago) could have just as easily been Beowolf.  At this point, I knew I was hosed.  The timer was at less than a minute and I was flailing.  Unfortunately, only thing came to my mind and it went out unfiltered to the audience.  If I know my colleagues (and I do), I am sure that this will live on for a long, long time.  I looked up at the audience, nodded slowly and said “this is kinda like watching your dad get beat up, isn’t it?”.  I’ll leave the audience’s response to your imagination.  However, I will tell you that I am almost certain that, as I was walking back to my seat, I could hear playing on the Hard Rock background music from the hall “ha ha ha ha ha ha ha ha, wipeout”.

The Medicine

I named this post ‘Taking My Medicine’.  Part of that medicine is sharing that horrifying moment with you, in the hope that you will not repeat it.  However, there is a second dose of medicine.  At the very end of my presentation (from here on out I’ll just refer to it as ‘the incident’), I told the audience that I would record the presentation properly.  I thought I’d share it with all of you, as well.  Click here to watch my screencast of .NET RIA Services in 5 Minutes.

The Summary

1. Never leave your wingman
2. Don’t play ball in the house.
3. Never change your presentation just before you present.

Working with Azure Table Storage from PHP

Introduction

Windows Azure Table Storage is a non-relational structured storage system in the cloud that offers massive scalability, durability and high availability.  The service is exposed with a RESTful API.  As such, it is easily consumable from a variety of platforms, including PHP.  In this post, I will illustrate how to consume Azure Table Storage via the RESTful API.  It is important to note that on July 31, 2009 we will reach Milestone 2 on the PHP SDK for Windows Azure.  Milestone 2 focuses on support for Azure Table Storage.  Accordingly, in all of the code you see in this blog post and the accompanying screencasts (Part I and Part II) illustrate accessing Azure Table Storage the “hard way”.  Much of the work I had to do by hand in the accompanying example here will be taken care of for you by the SDK.  That said, let’s dig in…

PHP Interop Series

This is part of a small series of blog posts and screencasts that I am doing on interoperability between PHP and the Microsoft stack.  My goal is to illustrate some features of the MS stack that PHP developers can take advantage of, as well as to illustrate how it is done.

Watch the Screencasts

I have created 2 screencasts that accompany this blog post that you can watch.  You can view them here:

• Working with Azure Table Storage From PHP – Part I
• Working with Azure Table Storage From PHP – Part II

What is Windows Azure Table Storage

As stated in the intro, Windows Azure Table Storage is a non-relational structured storage system in the cloud that offers massive scalability, durability and high availability.  While the goal of this post is not to dig deep into what Azure Table Storage is, I think it is best to at least provide a high-level overview of what it is and why you should care.  Looking at the first sentence in this paragraph, there are 4 major points that need to be covered: 1) non-relational storage system, 2) in the cloud, 3) massive scalability and 4) durability, high availability.

Non-Relational Structured Storage System

It is important to note that while we refer to this storage abstraction as tables, they do not conform to tables in the traditional sense.  To start with, the container for tables is not a database, rather a Storage Account.  You establish this storage account in the Azure Services Developer Portal (see this blog post for full details).  The Storage Account acts as the parent namespace for Table Storage, therefore you will not have any naming collisions with others that have similarly named tables.

A second major difference is that Azure tables do not conform to the traditional rows and columns structure.  Azure tables contain entities.  I think of entities as property bags.  These entities contain 3 or more properties which are typed name-value pairs.  I say 3 or more properties because every entity in Azure Tables must contain a PartitionKey, a RowKey and a TimeStamp property (I will go into a bit more detail on the role of these properties later in this post).  An interesting byproduct of this design is that tables are completely schema-less.  For instance, you are not precluded from adding 2 entities into the same table that have completely different schemas.  I do not want to address the inevitable debate that surfaces whenever this statement comes up, but suffice it to say that it is up to the developer to protect the schema in their tables.  The figure below illustrates how it is structured.

A third major difference is that there are no relational capabilities in Windows Azure Tables.  Many people look at these differences (between this structured storage system and a traditional RDBMS system) and focus on what is missing.  The simple reality is that Azure Table Storage is what it is… which is a structured storage system that provides massive scalability.

In the Cloud

Windows Azure is a cloud platform.  The data in Windows Azure Table Storage is stored in Microsoft Data Centers around the world and are accessible via HTTP.

Massive Scalability

The typical means of scaling databases is to scale up – buy bigger and bigger boxes.  However, after a while you run out of either bigger boxes or money.  At that point you have to scale out.  In the database world, this process is know as horizontal partitioning or sharding.  In RDBMS tables, this means that differing rows of data are stored on different physical partitions.  For example, I might store a subset of customers and their orders on one partition and another subset on another (perhaps customers with last names A-M on one and N-Z on another – or perhaps customers in Europe on one, customers in Asia on another …).

You might correctly infer that the more flexible you are in your ability to move data from one partition to another, the more scalability your system will have.  For instance, at one extreme, if all of your data must be on the same partition, you can only scale up (not out) and when you reach that threshold, you can scale no more.  At the other extreme, if you are able to store each record / entity on a differing partition, you have infinite scalability.  Now don’t go getting excited.  When data is stored across different partitions, it is quite costly to query it.  You are forced to query each partition independently.  As a result, you need to take special care to ensure that data that you want to query together remains on the same partition, while still allowing for the scalability required.

As mentioned earlier, every entity in an Azure Table is required to have a PartitionKey property.  This PartitionKey provides you with the control to determine which records must be stored on the same partition.  Windows Azure is free to move Entities with different PartitionKeys to differing nodes when it needs to, for scaling or load balancing purposes.  If you think about it, that is pretty cool.  You don’t have to worry about moving the data about.  Azure will handle that for you.  However, you do need to realize that if you query data across PartionKeys, it will be rather expensive.  It is the classic give-take.

Durability, High Availability

Data in Windows Azure is replicated at least 3 times.  Further, as described above, Windows Azure will move data about for load balancing purposes (as well as for scaling).

The REST API

I published a blog post a month or so ago that described ad nauseum the REST API for Windows Azure Table Storage.  I would urge you to take a look at that post, or at least watch the first screencast that accompanies this post.  The understanding of the REST API, as well as the message signing process was key to me to be able to put together the following code.

Writing the PHP Implementation

Caveat (again)

I realize I have stated this before, but I want to mention again that on July 31, 2009, we will be releasing Milestone 2 of the PHP SDK for Windows Azure.  This milestone will include code to greatly ease the process of working with Azure Table Storage.  Much of the following code will be unnecessary.  That said, I think it is very cool to illustrate accessing Table Storage from PHP without the SDK.  It illustrates the benefits of exposing an open RESTful API.

Wine.php

In my code, I want to work with classes.  To that end, I first created a Wine class as seen here:

<?php
        class Wine
        {
            public $ShortWineName = "";
            public $WineLabelUrl = "";
            public $BottlePrice = 0.00;
            public $Description = "";
            public $Vintage = "";
            public $WineID = 0;

            public $PartitionKey = "";
            public $RowKey = "";
            public $TimeStamp;

            public function __construct()
            {

            }
        }
?>        

PHPTableStorageHelper.php

This class is a little helper I wrote.  It performs the following functions: 1) Calculates the appropriate URI for my Table (and page) 2) Signs the message, 3) Creates the HTTP Headers for the Request (including the Authorization header containing the signed message, 4) Issues the Request using CURL, 5) Parses the Response HTTP Headers and 6) Returns the Reponse Body.

GetResponse Method Signature

    public static function GetResponse($account_name,
             $shared_key,
            $table_name,
            $pageSize,
            &$nextPartition,
            &$nextRow){

        //Get the base uri
        $service_uri = self::getUri($account_name, $table_name);

        //Limit the results by the pageSize
        $query = '';
        if ($pageSize > 0)
            $query = '?$top='.$pageSize;

        //Add a filter for paging (if the partition and row were passed
        if (strlen($nextPartition) > 0 && strlen($nextRow) > 0)
        {
            //See if we have already created a query
            if (strlen($query) > 0)
                $query .= '&';
            else
                $query .= '?';

            //Add the NextPartitionKey and NextRowKey QueryString parameters
            $query .= 'NextPartitionKey'.'='.rawurlencode($nextPartition);
            $query .= '&NextRowKey'.'='.rawurlencode($nextRow);
        }

        $service_uri .= $query;

        //Get the date (GMT)
        $date = self::GetGMTDate();

        //Get the signature
        $signed_message = self::GetSignature($date, $account_name, $table_name, $shared_key);

        //Get the http headers array
        $headers = self::GetHttpHeaders($date, $signed_message);

        $session = curl_init($service_uri);
        curl_setopt($session, CURLOPT_HEADER, 1);

        curl_setopt($session, CURLOPT_HTTPHEADER, $headers);
        curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
        $response = curl_exec($session)
            or die('Curl error: ' . curl_error($session));

        //Split the response into headers and the actual response body
        list( $outheaders, $body )= explode( "\r\n\r\n", $response, 2 );

        //Split the outheaders into an array and set the ref arguments
        $outheadersArray = self::http_parse_headers($outheaders);
        if (array_key_exists('x-ms-continuation-NextPartitionKey', $outheadersArray))
            $nextPartition = $outheadersArray['x-ms-continuation-NextPartitionKey'];
        else
            $nextPartition = "";
        if (array_key_exists('x-ms-continuation-NextRowKey', $outheadersArray))
            $nextRow = $outheadersArray['x-ms-continuation-NextRowKey'];
        else
            $nextRow = "";

        curl_close($session);

        return $body;
    }

The core of this helper class is the GetResponse method.  This method takes in the account name containing the table to query, the table name, the shared key for use in signing the method and some arguments used for paging.  I realize that this method doesn’t allow you to really query a table, rather just return pages of records, but that is ok for the purposes of this post.

Calculate the appropriate URI

The class declares a constant used as a template for the URI to our table.  It is as follows:

    const uriTemplate = "http://%s.table.core.windows.net/%s";

The first placeholder (%s) in the template is replaced with the account name, while the second is replaced with the table.  The first line of code in the method simply does this replacement:

    $service_uri = self::getUri($account_name, $table_name);

The next bit of code simply adds a few querystring parameters for paging:

    //Limit the results by the pageSize
    $query = '';
    if ($pageSize > 0)
        $query = '?$top='.$pageSize;

    //Add a filter for paging (if the partition and row were passed
    if (strlen($nextPartition) > 0 && strlen($nextRow) > 0)
    {
        //See if we have already created a query
        if (strlen($query) > 0)
            $query .= '&';
        else
            $query .= '?';

        //Add the NextPartitionKey and NextRowKey QueryString parameters
        $query .= 'NextPartitionKey'.'='.rawurlencode($nextPartition);
        $query .= '&NextRowKey'.'='.rawurlencode($nextRow);
    }

    $service_uri .= $query;

Sign the message

The next 2 lines of code get the signed message that is to be used for the Authorization HTTP header.

    //Get the date (GMT)
    $date = self::GetGMTDate();

    //Get the signature
    $signed_message = self::GetSignature($date, $account_name, $table_name, $shared_key);

Firstly, we get the current date in GMT format.  Next we call a helper method that creates a string to sign and signs it with the private key.  Here is that helper method:

    public static function GetSignature($date, $account_name, $table_name, $key){

        $message_to_sign = "$date\n/$account_name/$table_name";
        $signed_message = base64_encode(hash_hmac('sha256',
                                                  $message_to_sign,
                                                  base64_decode($key),true));

        return "SharedKeyLite $account_name:".$signed_message;
    }

Create the HTTP Headers

The next line after returning the signature simply calls a helper method to return an array containing the HTTP headers:

    //Get the http headers array
    $headers = self::GetHttpHeaders($date, $signed_message);

Here is the helper method:

    public static function GetHttpHeaders($date, $signed_message){
            return array("x-ms-date:$date",
                         "Authorization:$signed_message",
                         "Accept:application/atom+xml,
                         application/xml");
    }

Issue the Request

The next few lines use CURL to issue the HTTP GET Request to the URI we calculated, passing the HTTP headers

    $session = curl_init($service_uri);
    curl_setopt($session, CURLOPT_HEADER, 1);

    curl_setopt($session, CURLOPT_HTTPHEADER, $headers);
    curl_setopt($session, CURLOPT_RETURNTRANSFER, true);
    $response = curl_exec($session)
        or die('Curl error: ' . curl_error($session));

Parse the Response HTTP Headers

The next few lines parse the HTTP Headers from the response.  CURL doesn’t give you access to an array of headers.  You have to do this yourself.  We have to in order to parse out the continuation headers that are returned.  These provide a pointer to us for paging.  Remember earlier when we added the querystring parameters?  We used the values passed here from a previous Response.

    //Split the response into headers and the actual response body
    list( $outheaders, $body )= explode( "\r\n\r\n", $response, 2 );

    //Split the outheaders into an array and set the ref arguments
    $outheadersArray = self::http_parse_headers($outheaders);
    if (array_key_exists('x-ms-continuation-NextPartitionKey', $outheadersArray))
        $nextPartition = $outheadersArray['x-ms-continuation-NextPartitionKey'];
    else
        $nextPartition = "";
    if (array_key_exists('x-ms-continuation-NextRowKey', $outheadersArray))
        $nextRow = $outheadersArray['x-ms-continuation-NextRowKey'];
    else
        $nextRow = "";

Here is the helper method http_parse_headers:

    static function http_parse_headers($headers=false){
        if($headers === false){
            return false;
        }

        $headers = str_replace("\r", "", $headers);
        $headers = explode("\n",$headers);

        foreach($headers as $value){
            $header = explode(": ", $value);

            if(count($header) == 1){
                $headerdata['status'] = $header[0];
            }
            elseif($header[0] && $header[1]){
                $headerdata[$header[0]] = $header[1];
            }
        }

        return $headerdata;
    }

Return the Response Body

The last 2 lines simply clean up and return the response body.

    curl_close($session);

   
    return $body;

WineContext.php

This class calls our helper class above, parses the ATOM Formatted response and returns a collection of the Wine class.  I’m not going to go into any detail, as it is pretty straightforward:

    public static function GetWinePage($table_name, $pageSize, &$nextPartition, &$nextRow)
    {
        include_once('config.php');

        $wines = array();
        $response = PHPTableStorageHelper::GetResponse($accountName, $sharedKey, $table_name, 
	$pageSize, $nextPartition, $nextRow);

        //Load the xml into simplexml and register the namespaces
        //referenced in the xml
        $dom = new SimpleXMLElement($response);

        //Create vars for the namespaces.
        //Calls to children takes the namespace (if exists)
        $metadataNS = "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata";
        $dataservicesNS = "http://schemas.microsoft.com/ado/2007/08/dataservices";

        //Iterate over the entity nodes
        foreach($dom->entry as $entry) {
            //Get a reference to the properties node
            $properties = $entry->content->children($metadataNS)->children($dataservicesNS);

            //Create the wine
            $wine = new Wine();
            $wine->WineID = (int)($properties->WineID);
            $wine->ShortWineName = (string)($properties->ShortWineName);
            $wine->Description = (string)($properties->Description);
            $wine->BottlePrice = (float)($properties->BottlePrice);
            $wine->Vintage = (string)($properties->Vintage);
            $wine->WineLabelUrl =(string)($properties->WineLabelUrl);

            //Add the wine to the array
            $wines[] = $wine;
        }

        return $wines;
    }

Catalog.php

Now on to consuming our classes.  I’m not going to get into all of the paging logic and UI code.  At the end of this post is a link where you can download all of the files.  The paging code should be pretty self-explanatory.  I do want to show you the code that calls our context and writes the HTML, though.  Firstly, I included the context and the wine classes:

<?php
    include('Wine.php');
    include('WineContext.php');
?>

Next, at the appropriate place in the HTML, I call our context and return the array of wines.  I then iterate over the results, outputting the appropriate HTML.

<?php

 

  //Get the data
  $table_name="Wines";

  $wines=CohoStorageData::GetWinePage($table_name,2,$nextPartition,$nextRow);

  //Add the next row returned from the service to the array   $nextRowArray[] =$nextRow;

  //Iterate over the results   foreach ($wines as $wine)   {


        echo'<tr>';

      echo   '<td>';

      echo      '<table width="100%" border="0" align="center" cellpadding="8" cellspacing="0" class="thinblueline" style="background-color: White;">';

      echo           '<tr>';

      echo               '<td valign="top">';

      echo                   '<img src="'.$wine->WineLabelUrl .'" />';

      echo               '</td>';

      echo               '<td valign="top">';

      echo                   '<strong>';

      echo                        $wine->ShortWineName;

      echo                   '</strong>';

      echo                   '<br />';

      echo                   '<span>';

      echo                        $wine->Description;

      echo                   '</span>';

      echo                   '<br />';

      echo                   '<br />';

      echo               '</td>';

      echo               '<td width="80" align="right" valign="top">';

      echo                   '<strong>';

      echo                       '$'.number_format($wine->BottlePrice,2,'.',' ');

      echo                   '</strong>';

      echo                   '<br />';

      echo                   '<br />';

      echo                   '<img id="ImageButton1" productid="'.$wine->WineID .'" cssclass="noUnderline" onclick="ItemSelected" src="images/selectbutton.gif" />';

      echo               '</td>';

      echo           '</tr>';

      echo       '</table>';

      echo     '</td>';

      echo'</tr>';                                                }
?>

Get the code

You can download the code here.  Don’t forget to add the appropriate account name and key in config.php.  Also, this code has no provisions for creating the tables or adding the records.  If you want direction on that, please follow this blog series.  It will illustrate how to do it (in .NET not PHP).  Otherwise, you can wait until July 31 (or shortly thereafter) and I will illustrate these features in the SDK for PHP.

Hosting a PHP Application in Windows Azure

Introduction

At MIX ‘09 we (Microsoft) announced support for FastCGI in Windows Azure.  FastCGI allows us to host applications like PHP both safely and performantly (warning – performantly is not a real word).  In this post, I will begin with a very brief overview of what Windows Azure is and why you might consider hosting your PHP apps on Windows Azure.  I will then provide you with a couple of resources that you can use to easily publish your PHP application to Windows Azure, including a link to a screencast I created that illustrates just how to publish a PHP application to Windows Azure.

PHP Interop Series

This is part of a small series of blog posts and screencasts that I am doing on interoperability between PHP and the Microsoft stack.  My goal is to illustrate some features of the MS stack that PHP developers can take advantage of, as well as to illustrate how it is done.

What is Windows Azure (from 30,000 feet)

Simply put, Windows Azure is a cloud platform.  The platform provides the ability to host applications (Web Applications, Web Services and Services that don’t accept HTTP Requests) and to store and retrieve data in the cloud.  The important part of the last sentence is “in the cloud”.  Clearly, Windows Server 2008 (along with a multitude of other operating systems) provides the services to host applications and store and retrieve data.  The “in the cloud” part is the clear differentiator for Windows Azure.

You might then think that the cloud to Microsoft would simply be one or several datacenters filled with servers running Windows Server 2008 either virtually or on physical servers.  However, it is much more than that.  What differentiates Windows Azure from that environment is something known as the Windows Azure fabric.  This fabric ties all of the Azure nodes together into a cohesive whole.  This fabric provides several benefits to the applications running in Windows Azure, including manageability and health monitoring.

Windows Azure provides a portal where the application owner can simply configure how many instances of an application are running in the cloud.  In the case of web applications running on multiple instances, you need not worry about configuring the load balancer.  The Windows Azure fabric handles that for you.  For example, if you decide that you want your application running on 5 instances instead of 2, you simply change a single setting in a configuration file.  Windows Azure does the rest.  You application is deployed on 3 additional instances that are wired appropriately to the load balancer.

What happens when one of these instances is no longer healthy?  Another of the functions of the fabric is to monitor the health of the applications and servers.  Windows Azure will simply bring up another instance of your application and re-route traffic to that new, healthy instance.  Think of the complexity of building this kind of tolerance yourself.

Why Host my PHP Application in Azure?

Some of the benefits of hosting your application in Azure should be apparent from the preceding discussion.  They include scalability, availability and manageability.  Other benefits that may not be so apparent are the fact that you only pay for what you use and there is no lead-time required to bring up instances of your applications.

Pay for what you use / Lead time required

A key differentiator between all cloud platforms (not just ours) and hosted environments is that with cloud platforms you only pay for what you use and do not have to commit to future use.  In a typical hosted environment, you will likely have to commit to a defined term of use.  Further, in hosted environments, there is often a lead time required to get up and running.  There are obviously exceptions to this, notably in shared hosting scenarios.  However, if you are looking for dedicated hosting on a number of nodes, you will probably find that a significant lead time is required, along with a long-term contract. 

With Windows Azure, you simply pay for what you use and the only lead-time that is required is the amount of time required to spin up the new node.  This provides you with the flexibility to spin up new instances when demand requires and scaling back when that action is warranted. 

Scalability and Availability

This pay for what you use model allows you to scale up when demand requires and scale back when demand is down.  This can be great for startups that are intellectual property rich and cash poor.  They do not want to invest heavily in the infrastructure required to meet their forecasted demand.  However, they do want to be able to meet demand if their product or service takes off.  Further, because the Windows Azure fabric is monitoring the health of your application instances, re-routing requests to new instances where required, you can be self-assured that your application will be available to your customers.

Manageability

The goal of any abstraction layer is to hide the complexity of a given task from the end user, allowing them to focus on business-specific problems.  By hosting your PHP application in Windows Azure, you need not worry about provisioning servers, configuring the operating systems, administering patches, deploying instances of web applications and wiring them to load balancers.  All of this is neatly handled for you.  All you have to do is simply deploy your application to the Azure portal and change some simple configuration settings.

Resources for Publishing your PHP Application to Azure

The following are some resources that you can take advantage of.  They make it easy to publish your PHP application to Windows Azure.  They include a screencast I put together that details every step, a comprehensive blog post by Wade Wegner that lists the steps for publishing (this is what I used when I first did it), along with a link to a blog post I put together that illustrates how to get started with Azure.  The last post shows you all of the steps required to sign up for Windows Azure.

• Screencast on Hosting a PHP Application in Windows Azure
• Wade Wegner’s Post on ‘Running a PHP application on Windows Azure’
• Post on Getting Started with Windows Azure  

Comments Please

Please leave me a comment, letting me know whether this post / series is helpful to you or not.  I would really appreciate it!

« Previous Page — Next Page »