Tuesday, 8 September 2009

BDC Becomes BCS

It looks like the new version of Sharepoint 2010 will rebadge the Business Data Catalog (BDC) as Business Connectivity Services (BCS).

This will be providing read/write access to external data from not only Line Of Business (LOB) systems but also web services, databases or other data in sharepoint.

The BCS tools to do this will be in the Sharepoint Designer 2010 and VS2010 - lets hope it's easier than '07 where you ended up crafting lovely metadata xml files by hand or buying 3rd party tools.

For a look at the BCS and other new features of Sharepoint 2010, check out the sneak peaks...

Saturday, 28 March 2009

Asynchronous ASP.Net

This entry is actually the one of a write up of my notes from previous DevWeeks. Although from 2007, this is still relevant today...
These notes follow on from a Jeff Prosise talk on this subject at DevWeek 2007.

Asynchronous ASP.NET Programming – Jeff Prosise
Most sites that access external data (databases, web services etc) are written are done so in a synchronous way, and when the number of users increases the site can suddenly slow right down to a crawl, or even fail totally.

When ASP.NET receives a request for a page, it grabs a thread from a thread pool and assigns that request to the thread. A normal, or synchronous, page holds onto the thread for the duration of the request, preventing the thread from being used to process other requests.

If a synchronous request becomes I/O bound - for example, if it calls out to a Web service or queries a database and waits for the call to come back - then the thread assigned to the request is stuck doing nothing until the call returns. That impedes scalability because the thread pool has a finite number of threads available.

If all request-processing threads are blocked waiting for I/O operations to complete, additional requests get queued up waiting for threads to be free. At best, throughput decreases because requests wait longer to be processed.

At worst, the queue fills up and ASP.NET fails subsequent requests with 503 "Server Unavailable" errors.

ASP.NET 2.0 however supports asynchronous pages.

When the request arrives, it's assigned a thread by ASP.NET. The request begins processing on that thread, but when the time comes to hit the external data, the request launches an asynchronous request and returns the thread to the thread pool. When the query completes, the asynchronous request calls back to ASP.NET, and ASP.NET grabs another thread from the thread pool and resumes processing the request.
While the query is outstanding, zero thread pool threads are consumed, leaving all of the threads free to service incoming requests. A request that's processed asynchronously doesn't execute any faster. But other requests execute faster because they don't have to wait for threads to become free. Requests incur less delay in entering the pipeline, and overall throughout goes up.

Asynchronous HTML Handler Pages
The second asynchronous programming model featured in ASP.NET is the asynchronous HTTP handler. An HTTP handler is an object that serves as an endpoint for requests. Requests for ASPX files, for example, are processed by an HTTP handler for ASPX files. Likewise, requests for ASMX files are handled by an HTTP handler that knows how to deal with ASMX services.

You can extend ASP.NET to support additional file types by writing custom HTTP handlers. But even more interesting is the fact that you can deploy custom HTTP handlers in ASHX files and use them as targets of HTTP requests. This is the proper way to build Web endpoints that generate images on the fly or retrieve images from databases. You simply include an <img> tag (or Image control) in the page and point it to an ASHX that creates or fetches the image. Targeting an ASHX file with requests is more efficient than targeting an ASPX file because an ASHX file incurs much less overhead at processing time.

By definition, HTTP handlers implement the IHttpHandler interface. Handlers that implement that interface do their processing synchronously
HTTP handlers don't have to be synchronous. By implementing the IHttpAsyncHandler interface, which itself derives from IHttpHandler, an HTTP handler can be asynchronous. When used correctly, an asynchronous handler utilizes ASP.NET threads more efficiently. This is done in the same manner as an asynchronous page. In fact, asynchronous pages leverage the asynchronous handler support that predated asynchronous pages in ASP.NET.

See http://msdn.microsoft.com/en-gb/magazine/cc163725.aspx and
http://msdn.microsoft.com/msdnmag/issues/07/03/WickedCode/default.aspx for more information.

Friday, 27 March 2009

New Profile Pic

Quick post to say thanks to my friend and colleague Rob Baines, who is not only a talented architect, but also a great cartoonist, and did the "creator" avatar I'm using here.

Check out
his site for more details.

LINQ in .Net

This entry is actually the one of a write up of my notes from previous DevWeeks. Although from 2008, this is still relevant today...

These notes follow on from Tim Ewald talk on this subject at DevWeek 2008.

Introduction to LINQ

LINQ stands for Language INtergrated Query, and extends the .Net framework to provide query, set and transform operations. It allows the working with of collections of objects in a SQL like fashion.


Lambda Expressions

Is the next step past anonymous methods (which were introduced in .Net 2.0). They are symbolised by the => operator and can have a simple test, or an entire procedure following them. They can be useful when working with any collection that is enumerable. Below is an example:


Person[] people = GetPeople();

IEnumerable result = people.Where(p => p.LastName == lastname)


Extension Methods

These allow you to extend any existing type without breaking any implementors. Extending is just like writing a new method, however for the parameters you specify "this ". The keyword "this" identifies it as an extension method, followed by the data type (could be an interface as well as a object) followed by a variable name to use in the method. The methods must be static.


C# 3.0 uses this technique to extend IEnumerable with the Enumerable class. This adds many functions including Sum, Avg, Min, First, Union, Where, OrderBy

public static StringExtension


public static string addHi(this string str)


return str + "Hi";




String name = "test";

name = name.addHi();

^Would return testHi



This is used in IEnumerable collections to simplify the creation of a list of one type of object from the enumeration of a list of another type of object. It's used in the Where() and Select() extension methods of IEnumerable.


Anonymous Types

These are structural unnamed types, that infer their type from the first assignment to them. They differ from VB6/VBA object type though as they enforce type safety from the first assignment onwards. It uses the var keyword before the name of the variable.


Type Inference

Type inference is a way to refer to an anonymous type. The type is inferred and not dynamic (hence type safety is enforced, as stated above)


They can only be used at a local scope.

var i = 5;

i = "Hello";

^this would produce an error as i is inferred as an int


Object Initialisers

Object Initialisers allow the property/field values of a new object to be set on instantiation. It's primarily used in the .Select() method of Ienumerable, via a lambda expression. An example is below:


Person[] people = GetPeople();

IEnumerable result = people.Where(p => p.LastName == lastname)

.Select(p => new PersonSimple{


LName = p.LastName});

LINQ Simplified Syntax

To simplify the use of the above features, new syntax is also supported which is more SQL like. Below is an example of the above code rewritten in this syntax


Person[] people GetPeople();

Var result = from p in people

where p.LastName == lastname

Select new {p.Name,LName = p.LastName}


LINQ Providers

The following providers are available:

LINQ to Objects

See examples above


Provides an enumerable tree model to walk through XML


Maps Database schemas to classes

Supports queries, updates and stored procedures

Supports delegated execution (passing an ad hoc procedure to SQL to be run)

Should be used with caution - you most likely end up pulling masses of data to the server

Behind the Scenes of WCF

This entry is actually the one of a write up of my notes from previous DevWeeks. Although from 2008, this is still relevant today...

These notes follow on from Aaron Skonnard talk on this subject at DevWeek 2008.

Message Oriented WCF

WCF provides a message-oriented programming model, with flexibility in its use.


WCF Messages

All WCF messages are modelled with System.ServiceModel.Channels.Message.


They can be encoded as XML, JSON, binary etc. They can also optionally be mapped to/from .Net objects (see serializing, below).


The MessageVersion class allows the specifying of the versions of SOAP and WS-Addressing that you wish to use (if any).


As the message class has been designed to support streaming, the body of a message can only be processed once for a particular instance. You can copy the body if you need to process the message more than once. The message has a state property which will be either Created, Read, Written, Copied or Closed.


The message has a Headers property which can hold values about the message. It is typically the job of intermediaries to process the message headers. Intermediaries often need to store the results of their processing somewhere for future use in the processing pipeline, which is why a message also have a message properties collection. The properties are usually only used during local processing, and don't usually affect what happens to the message on the wire, but they are able to.


Message bodies can be read from by calling either GetBody or GetReadyAtBodyContents. The body can be written to by calling WriteMessage or WriteBody, passing in either an XmlDictionaryWriter or XmlWriter object.


Action values of the OperationContract can be used to map messages to methods, and these can be either universal one-way or two-way operations. Operations can either be "*" or named specifically.


Data and Message Contracts

The data contract defines the format of the message body that will be passed in the message. It is done by annotating classes with the [DataContract] and [DataMember] attributes. An example is shown below:



public class Person


public string PersonID; //This property will not be in the message


public string Name;


public int Age;




A data contract will break if any of the following changes are made:

  • Changing the type Name or Namespace

  • Changing the Order of data members

  • Changing the Name of a data member

  • Changing the type of a data member

  • Changing IsRequired property from false to true


Message contracts map data contracts to SOAP envelopes. They allow full control to model SOAP messages and headers, by annotating the class with [MessageContract], [MessageBodyMember] and [MessageHeader] attributes on members.



Messages can be serialised into .Net objects using one of the supported serializers. This allows objects to be sent easily using WCF. Messages can be "typed" to .Net types by annotating the types with special mapping attributes. At runtime a serializer maps the .Net objects into messages.


The XmlSerializer is the original serializer from .Net 1.0, and is still fully supported, and offers backwards compatibility to web (ASMX) services and other Non-WCF services.


The DataContractSerializer is a new serialiser in .Net 3.0 that supports most .Net types including:

  • CLR built in types,

  • Byte Array, DateTime, TimeSpan, GUID, XmlQualifiedName, XmlElement, XmlNode arrays

  • Enums

  • Types marked with the DataContract or CollectionDataContract attribute

  • Types that implment IXmlSerializable

  • Arrays/Collection classes including List, Dictionary and HastTable

  • Types marked with Serializable attribute or implement Iserializable


Note that the DCS only supports a subset of XSD


In .Net 3.5 the DataContractJSONSerializer was added, that perfoms the same as the DCS but with serializing to JSON instead of XML.

Message-oriented Design Techniques

Pure XML messaging approaches are gaining more traction these days. This is very like REST.


Schema-first design is common in integration scenarios, specially where collaboration and parallel development is required. Data/message contracts can be generated from schemas using SvcUtil.exe. XmlSerializer (and xsd.exe) is the recommended approach here.


Code-first design is more common when collaboration isn't needed, where the developer writes data contracts and lets WCF generate the schemas. DataContractSerializer (and SvcUtil.exe) is the recommended approach here.

Code Access Security (CAS) in .Net 3.5 (+ Other Security Bits)

This entry is actually the first in a write up of my notes from previous DevWeeks. Although from 2008, this is still relevant today...

These notes follow on from Dominick Baiers talk on this subject at DevWeek 2008.

Code Access Security & Sandboxing

Most of the important base features were added in 2.0, including a simple sandboxing API, security transparency and manifest based activation. These have been used in several current and future technologies such as ClickOnce, XBAP, and SilverLight.


Sandboxing is all about loading code dynamically yet restricting what it can and cannot do. This is supported using the AppDomain class with restricted permissions. The permissions that the code in the sandbox would use were defined by the permission set in Code Access Security (CAS).

Code Access Security Overview

CAS defines permissions and permission sets that represent the right to access various system resources. The permissions are grouped into security policies by associating them with groups of code. It allows code to request the permissions it requires in order to run, as well as the permissions that would be useful to have, and specifies which permissions the code must never have. The restrictions are enforced at run time.

When a method that is called demands a permission, CAS traverses the stack of assemblies up, checking if the method is allowed. The diagram below illustrates this:

Setting up permissions and policies however in CAS was very cumbersome and not very readable without in depth knowledge of CAS, and even then a challenge.


The CAS model has now been simplified for sandboxing, so that the PolicyLevel attribute is no longer required, and that the AppDomain is independent from the security policy (which is a very painful document to work with). Instead, loaded assemblies either get permissions specified in grantSet or full trust if part of fullTrustAssemblies list.


Windows Security Integration

Much of the work to provide Windows security integration was done in 2.0 with SIDs, ACLs and tokens.


Role based security has been integrated since 1.0 using IIdentity and IPrincipal along with Thread.CurrentPrincipal and PrincipalPermission. 3.0 introduced authorisation systems based on claims (Claim type, Resource, Right) which are grouped into claim sets. A claim set has an issuer and the issuer itself is a claim set.


A new "convenience" library has been added for common tasks called System.DirectoryServices.AccountManagement. This allows amongst other things:


    • Adding/modifying/deleting user/group/machine accounts

    • Password validation

    • "is user x in group y" scenarios


It supports local, domain and ADAM user stores.



The symmetric cryptography and hashing algorithms have been updated in 3.5.


Also, support has been added for Crypto Next Generation (CNG) API, which is the next generation of crypto standards which are compliant with the NSA "Crypto Suite B" set, and elliptic curve based algorithms.


Note, the new API is only supported by Vista and Windows Server 2008 however


Network Protocols

The most important additions here were made in 2.0, which are SslStream and NegotiateStream, which allow the layering of SSL and SPNEGO over arbitrary network streams. This gets used by higher level frameworks live WCF.


Two new secure protocols were also added in 3.5 which were Named Pipes (anonymous and named) and Peer technologies including peer to peer (P2P), peer name resolution protocol (PNRP) and people near me (PNM)



New support is added for Event Tracing for Windows (ETW) and Windows Eventing 6.0 with new classes including EventSchemaTraceListener, System.Diagnostics.Eventing and System.Diagnostics.Eventing.Reader.


Windows Management Instrumentation (WMI) support has also been rewritten under System.Management and System.Management.Instrumentation.

Windows Home Server now on MSDN

Well, after all the refusals to do it, it seems that Microsoft have finally made Windows Home Server available for MSDN subscribers!

See this link for more details...

Wednesday, 25 March 2009

It has begun...

Well I finally thought I'd start a blog to save all my thoughts and discoveries whilst I code away at and outside of work!

My plan is to blog about anything to do with coding, development, architecture and all those wonderfull geek like things.

Anyway, a little about my experience in case people are interested. I am currently employed as technical/solutions architect with a large property services company. I've been here over 10.5yrs. For more details on that, you can check out my linkedin profile

I've been a true style geek since the age of 8 when I wrote my first logo program at school, then got an Amstrad CPC6128 (go 128kb of ram!) and was writing games on that as I wasn't impressed with the ones I could buy.

I've worked with .Net since it's initial release, and have followed it through to the current upcoming 2010 version. Also I've worked a lot with Oracle databases and writing/tuning apps for it. But I'm not adverse to dabbling with Java or any other language that intrigues me (currently having a play with Ruby & Python).

Technologies & trends I am specifically enjoying playing with at present are Message Broker, WCF, Security (geneva etc) and REST. Oh and I'm sure some cloud-play will come about too.

I love the challenge of problem solving applications, be that with performance, or those hard to trace bugs, plus designing/architecting new solutions. I like to think I've got a good grounding in the work I do, but I always try to remember what a great man (an ex boss in fact) once told me - and that is that "you don't know what you don't know!" so every day working on code is potentially a lesson - don't get compacent!