SharePoint

How to create an adapter for the TFS Integration Platform - Part V: Items (IMigrationItem & IMigrationItemSerializer)

Submitted by Robert MacLean on Tue, 06/01/2010 - 09:23

Note: This post is part of a series and you can find the rest of the parts in the series index.

An important class in your implementation is your item implementation, which is used to identify what an item (be that file, directory, list item, bug etc…) is. This class needs to implement IMigrationItem. This class is more important for VC than WIT but in both cases you need to put in all the properties you need to know about. You must also make sure that all properties are serialisable by the default XML serialiser in .NET.

Power Tip: VC stands for Version Control. This refers to an adapter that works with the source control aspects of the system. WI, work items, and WIT, work item tracking, are the same thing. File attachments in WI are NOT regarded as VC and must be handled by your WI adapter.

IMigrationItem

WIT

The WIT adapter is slightly smaller than the VC one since it is just a set of properties. I do not care too much about the inherited DisplayName property here either and the Download method is just logging. The only interesting part is the SimpleDictionary<T,V> I use. SimpleDictionary<T,V> is just to store Key/Value pairs of column information from SharePoint (because you may have customised the columns in SharePoint, I cannot hard code them). The reason I use this rather the Dictionary<T,V> which .NET provides, is because that class is not able to be serialised using the default XML serialiser. I will cover SimpleDictionary<T,V> in a later post.

public class SharePointListItem : IMigrationItem
{
    public SharePointListItem()
    {
        this.Columns = new SimpleDictionary<string, object>();
    }

    public string Id { get; set; }
    public DateTime ModifiedOn { get; set; }
    public string AuthorId { get; set; }
    public SimpleDictionary<string, object> Columns { get; set; }    
    public string DisplayName { get; set; }

    public void Download(string localPath)
    {
        TraceManager.TraceInformation("WSSWIT:MI:Download - {0}", localPath);
    }
}

VC

The VC adapter is a little more complex with more properties but the key difference is the DisplayName property, which I return the filename. Even more important though it the Download method which is used to get the actual file or folder to a specific location, provided by the localpath parameter, on disk for the platform to use.

Power Tip: When you are downloading files in the IMigrationItem, you are responsible for the creation of the path too. So make sure you are creating directories and also checking what directories exist too.

string IMigrationItem.DisplayName
{
    get { return Filename; }
}

void IMigrationItem.Download(string localPath)
{
    TraceManager.TraceInformation("WSSVC:Item:Download:From {0} to {1}", this.AbsoluteURL, localPath);
    if (this.ItemType == SharePointItemType.File)
    {
        TraceManager.TraceInformation("\tType is file");
        string targetDir = Path.GetDirectoryName(localPath);
        if (!Directory.Exists(targetDir))
        {
            TraceManager.TraceInformation("\tCreating Directory for file - {0}", targetDir);
            Directory.CreateDirectory(targetDir);
        }

        HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(this.AbsoluteURL);
        webRequest.Credentials = this.Credentials;
        using (Stream responseStream = webRequest.GetResponse().GetResponseStream())
        {
            using (FileStream fileStream = new FileStream(localPath, FileMode.CreateNew, FileAccess.ReadWrite))
            {
                byte[] buffer = new byte[1024];
                int bytesRead;
                do
                {
                    // Read data (up to 1k) from the stream
                    bytesRead = responseStream.Read(buffer, 0, buffer.Length);

                    // Write the data to the local file
                    fileStream.Write(buffer, 0, bytesRead);
                } while (bytesRead > 0);
            }
        }

        TraceManager.TraceInformation("\tFile downloaded successfully");
    }

    if (this.ItemType == SharePointItemType.Directory)
    {
        TraceManager.TraceInformation("\tType is directory");
        if (!Directory.Exists(localPath))
        {
            TraceManager.TraceInformation("\tCreating Directory - {0}", localPath);
            Directory.CreateDirectory(localPath);
        }
    }
}

IMigrationItemSerializer

The migration item serializer is just a way to get your item to XML using .NET serialisation. My implementation of this is the exact same for both VC and WIT.

public class SharePointWITMigrationItemSerializer : IMigrationItemSerializer
{
    public IMigrationItem LoadItem(string itemBlob, ChangeGroupManager manager)
    {
        TraceManager.TraceInformation("WSSWIT:S:LoadItem");
        if (manager == null)
        {
            throw new ArgumentNullException("manager");
        }

        if (string.IsNullOrEmpty(itemBlob))
        {
            throw new ArgumentNullException("itemBlob");
        }

        XmlSerializer serializer = new XmlSerializer(typeof(SharePointListItem));

        using (StringReader itemBlobStringReader = new StringReader(itemBlob))
        {
            using (XmlReader itemBlobXmlReader = XmlReader.Create(itemBlobStringReader))
            {
                return (SharePointListItem)serializer.Deserialize(itemBlobXmlReader);
            }
        }
    }

    public string SerializeItem(IMigrationItem item)
    {
        if (item == null)
        {
            throw new ArgumentNullException("item");
        }

        TraceManager.TraceInformation("WSSWIT:S:SerializeItem - {0}", item.DisplayName);

        XmlSerializer sharePointTaskSerializer = new XmlSerializer(item.GetType());

        using (MemoryStream memoryStream = new MemoryStream())
        {
            sharePointTaskSerializer.Serialize(memoryStream, item);
            memoryStream.Seek(0, SeekOrigin.Begin);
            using (StreamReader streamReader = new StreamReader(memoryStream))
            {
                return streamReader.ReadToEnd();
            }
        }
    }
}

How to create an adapter for the TFS Integration Platform - Part IV: IProvider

Submitted by Robert MacLean on Mon, 05/31/2010 - 09:10

Note: This post is part of a series and you can find the rest of the parts in the series index.

IProvider is the first class we will look at implementing for both adapters (WI and VC) as it provides the core information for the platform to talk to our adapter. The first thing you provider needs is the ProviderDescriptionAttribute, which has three properties ID, Name and Version.

[ProviderDescription("{7F3F91B2-758A-4B3C-BBA8-CE34AE1D48EE}", "SharePoint TIP Adapter - Version Control", "1.0.0.0")]
The ID must be unique and you will need a record of it somewhere as it is used in configuration for the platform. The name and version are potentially used to make it easier for users, but I have not seen them used anywhere (maybe in future/different tools).

The only  method in the provider is the GetService method which is used to get the implementations of the interfaces/classes we will be building later. Put another way this allows the platform to request a class which implements a specific interface using this method:

object IServiceProvider.GetService(Type serviceType)
{
    TraceManager.TraceInformation("WSSVC:Adapter:GetService - {0}", serviceType);

    if (serviceType == typeof(IAnalysisProvider))
    {
        if (analysisProvider == null)
        {
            analysisProvider = new SharePointVCAnalysisProvider();
        }
        return analysisProvider;
    }
    
    if (serviceType == typeof(IMigrationProvider))
    {
        if (migrationProvider == null)
        {
            migrationProvider = new SharePointVCMigrationProvider();
        }
        return migrationProvider;
    }

    if (serviceType == typeof(IServerPathTranslationService))
    {
        if (transalationProvider == null)
        {
            transalationProvider = new SharePointVCAdapterTranslation();
        }
        return transalationProvider;
    }        

    return null;
}

Above is the implementation is what I used in the SharePoint VC adapter, the WI adapter is the same except it does not have the server path translation service at the end.

Power Tip: Using Visual Studio 2010’s new “Generation from usage” features makes this stage of development much easier. 

How to create an adapter for the TFS Integration Platform - Part III: Overview of adapters

Submitted by Robert MacLean on Fri, 05/28/2010 - 10:42

Note: This post is part of a series and you can find the rest of the parts in the series index.

The TFS Integration Platform has two types of adapters, a WI (for work items, tasks,  bugs etc…) and a VC (version control) adapter and they are nothing more than a .NET assembly made up of a number of classes which, mostly, you will inherit from interfaces in the Toolkit project. For both adapters the key interfaces you need to implement are:

  • IProvider: Gives the platform the way to invoke your adapter.
  • IMigrationProvider: This is used for writing to the adapters source system, so for me SharePoint.
  • IMigrationItemSerializer: This provides support for converting the item to XML.
  • IAnalysisProvider: This is used for reading from the adapters source system.

as well as they both need to implement the ChangeActionHandlers abstract class.

The VC adapter also needs:

  • IServerPathTranslationService: Used to translate the path (i.e. directories and such) from other adapters to this adapter and visa versa.

 

image 

While the WIT adapter needs:

  • IConflictHandler: Provides support for handling conflicts during the migration.

image

So in both adapters you will need a minimum of 6 classes you will implement, excluding any extra ones you will need for your specific requirements.

The core concepts of the adapters are all explained in the interfaces and so it appears that it is very simple to implement, and indeed it is – however there are some weird things which may catch you up which we will cover in detail in future posts.

TraceManager

Something very nice in the platform is the TraceManager class which is really just a wrapper around System.Diagnostics.Trace but it has some extras in that wrapper, such as being what is included in the console windows and log files. You will see this sprinkled through out my code because it is useful to to have when trying to debug later on.

Power Tip: The TraceManager puts all information written to it in the log files, so please make sure you do not put any sensitive information in there.

How to create an adapter for the TFS Integration Platform - Part II: Setup of the environment

Submitted by Robert MacLean on Wed, 05/26/2010 - 16:34

Note: This post is part of a series and you can find the rest of the parts in the series index.

Getting started with the adapter development is not the easiest task because you are stuck a little in the wild, so this part will serve as as a quick start guide for getting what you need to become a TFS Integration Platform developer.

SQL Server

The TFS Integration Platform requires Microsoft SQL Server so you need to install an instance of that.

TFS

It goes without saying, or maybe it doesn’t, that if you plan to write an adapter to integrate to TFS you will need TFS. Even if you don’t care about TFS, you will want to test and the TFS 2010 adapters are of the highest quality and so they make a great test target (so testing between your adapter and TFS). Thankfully with TFS 2010 you can now install on Windows 7 natively so this means as a developer you can have a great easy environment.

Target system (SharePoint for me)

Since I was developing for SharePoint that meant I needed a SharePoint installation, which meant a 20Gb Windows 7 Virtual Machine :( Hopefully for you this will be less of an issue.

TFS Integration Platform

The TFS Integration Platform, is a software component and database which runs on your machine and handles the actual moving of data around. You can get it from http://tfsintegration.codeplex.com/releases - however it may not be obvious which is the one you want since the team has so many download options, you want the tools:

image

During install you will get an option to install the service, which is recommended for production environments when you want to have the synchronisation running continuously. However for development this is not needed.

Power Tip: Once you have completed the tools install, go into to SQL Server and backup the TFSIntegrationPlatform database immediately. There are not only a few odd bugs that roam around the platform (it’s still in beta) which may cause you to need a restore of the database but if you want to test on a clean environment then a restore is quicker than a reinstall.

Platform Source

To build adapters you will also need the source code for the TFS Integration Platform which you can also get from CodePlex. Best is to get the latest drop of the code which you can get from the Source Control page and then by clicking on the Download link in the latest version box on the far right.

image

In there you will find the IntegrationPlatform folder which contains all the code from Microsoft.

image

Power Tip: Make a common root for the TFS code and yours (in my case I used RangersCode) and then create sub directories in there for platform and your code (so I had My Production and MS production folders under RangersCode). This helps keep the items close, which makes things easier later plus keeps them separate so you can identify them.

The code itself is for Visual Studio 2008, however you can be just like me and use Visual Studio 2010 and it will work just fine. Once you have done all of this you are finally ready for writing your adapter!

How to create an adapter for the TFS Integration Platform - Part I: Introduction

Submitted by Robert MacLean on Tue, 05/25/2010 - 14:29

Note: This post is part of a series and you can find the rest of the parts in the series index.

Since September 2009 I have been engaged in a ALM Rangers project, namely the TFS Integration Platform. Which is:

The TFS Integration Platform is a project developed by the Team Foundation Server (TFS) product group and the Visual Studio ALM Rangers to facilitate the development of tools that integrate TFS with other systems. Currently, the scope of this project is to enable TFS to integrate with other version control and work-item/bug tracking systems, but the eventual goal of this project is to enable integration with a broader range of tools/systems (i.e. build). This platform enables the development of two major classifications of tools: tools that move data unidirectionally into TFS, and tools that synchronize data bidirectionally.

So in short it is a integration system, like BizTalk or SSIS but specially built for version control and work items. I have not said TFS there because, it can work to migrate between other source control and work item systems provided adapters exist. Adapters are the logic which allows the TFS platform to connect to a variety of technologies, and my goal has been to build two of them – one for SharePoint lists and one for SharePoint document libraries.

You may have noticed that SharePoint isn’t a version control or work item system, so why integrate? Well lots of companies do use it for ALM related items, such as the lists being used for tracking work items and the document libraries are used to store content which should be in a source control system. This is the first post in a series which will give you an idea of what is involved in building adapters, show you what to avoid and hopefully give you a few laughs at my expense. 

Now I want to be clear this series will not covering usage of the platform or any of the core concepts in it. For those please see the links below in particular Willy-Peter’s blog. You do need to understand a bit about how the platform works before you attempt to build your own adapter.

As all my work was done for the ALM Rangers the source code for my adapters is included in the code which can be obtained from the CodePlex site.

To help you on your way let’s list a few links which are key for this:

Upgrading to SharePoint 2010: In the field experience

Submitted by Robert MacLean on Mon, 04/26/2010 - 12:03

sharepoint1Last week I was able to upgrade BB&D’s internal intranet site from SharePoint 2007 to SharePoint 2010! So you can properly understand what happened let me cover a little bit about the intranet first. Our intranet is a small deployment, just a single server deployment however it is kept up to date with technology so it is running on SQL Server 2008 with SP1, Windows 2008 R2 on a 64bit virtual machine. We have also not gone with heavy customisation, rather focusing on small tweaks and adjustments. A good example is we do not have a customised master page but rather use the theme options to get the colour scheme we want.

The first step I did was to download the pre-requisites for SharePoint 2010, and using the option on the installer this was a breeze. I’ve seen this before with Dynamics CRM and once again I am impressed by how a very simple feature makes such a big difference. Next step was the install, which was pain less and quick.

Once installed the configuration manager had to run and this is where I had two issues. The first problem was that I got stuck on task 1! The cause here is that a dialog box had appeared behind the main window (telling me to do the same to all servers in the farm) and won’t go until I clicked OK. This annoying little bug cost me a few minutes.

The second issue was that the SharePoint 2010 install needs to do some Active Directory queries and this meant my user account was not good enough. Not having a good enough user means that the installer produces a very unhelpful error at step 3:

System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Microsoft.SharePoint.Utilities.SPUtility.GetUserPropertyFromAD(SPWebApplication webApplicaiton, String loginName, String propertyName)
   at Microsoft.SharePoint.Administration.SPManagedAccount.GetUserAccountControl(String username)
   at Microsoft.SharePoint.Administration.SPManagedAccount.Update()
   at Microsoft.SharePoint.Administration.SPProcessIdentity.Update()
   at Microsoft.SharePoint.Administration.SPApplicationPool.Update()
   at Microsoft.SharePoint.Administration.SPProcessIdentity.UpgradeToV4ManagedAccount()
   at Microsoft.SharePoint.Administration.SPConfigurationDatabase.ResolveObjectAndClassVersions(SPLog log)
   at Microsoft.SharePoint.Upgrade.SPConfigurationDatabaseSequence2.Upgrade()
   at Microsoft.SharePoint.Upgrade.SPUpgradeSession.Upgrade(Object o, Boolean bRecurse)

sharepoint2It took ages to figure this out, mostly because it is not a documented requirement. To get the correct permissions you need to get a domain admin to do the following on the domain controller:

  1. Open up Active Directory Users and Computer.
  2. Select Advanced Features from the View menu. Failing to do this means that the tab in step four won’t be visible.
  3. Right-click the your AD account and select Properties.
  4. Select the Securities Tab.
  5. Select Authenticated Users in the Group or user names field.
  6. Allow Full permissions in the Permissions for Account Operators.
  7. Repeat this process for any SharePoint service accounts you may have created.
  8. Next make sure this change replicates to all domain controllers.
  9. Now connect to your SharePoint server, and open the command prompt (cmd.exe) and type gpupdate /force. This will force the changes to the machine as it may have a cached version.
  10. Finally reboot the SharePoint server and start the configuration wizard again.

After all that was done the configuration wizard completed and the upgrade process started in central admin. The upgrade process also took a while to do, but once done everything just worked.

Lastly we applied some theme tweaks and a quick run through of testing it and it was done. One thing that is important to remember about this process is that the entire time it was happening the current intranet was done so plan your deployment accordingly. This is easily the best experience I’ve ever had installing or upgrading SharePoint and shows that the product is maturing.

Death of a SharePoint Developer

Submitted by Robert MacLean on Wed, 04/21/2010 - 14:08

Originally from :http://rawsocket.org/pf/arquivos/2004_11_01_index.html I have had to explain many times in the last year why I, a normal developer, am involved with Information Worker which is (mostly) a SharePoint group? I am involved because I believe that the idea of a SharePoint Developer is a fast dying one and soon, people who call themselves a SharePoint Developer will be using it just as a way to justify higher consulting costs more than anything else.

I do not think this is because SharePoint usage is dying, rather the growth (maturity and adoption) of SharePoint is causing SharePoint developers to die off. This is not because SharePoint is so user friendly we no longer need custom code, because we still need custom code in SharePoint. The two reasons for my thinking this is based on two questions, “What SharePoint development really is?” and secondly “What Microsoft is doing about SharePoint development?”.

What is SharePoint development really? In SharePoint versions past (2007 and before) you would develop code for SharePoint using development concepts unique to SharePoint. Now that SharePoint has matured, development of the code for SharePoint involves concepts that are universal to development. There are two examples which come to mind which highlight the maturity of development concepts. First is web parts, which are now the same as ASP.NET web parts, and secondly is web services (and OData if you have SharePoint 2010). Both of these concepts are the exact same as used by many other products made by Microsoft and other companies. For example if you understand how to get data from Twitter, with OData in SharePoint 2010 you will understand how to get data from SharePoint. Yes, you will have some specific bindings/API’s/code that are SharePoint specific but the concepts, which is the difficult part to learn, are the same.

I mentioned two questions and the second is about Microsoft and it’s strategy for SharePoint development, in particular their 2010 strategy (Visual Studio 2010 + SharePoint 2010). A SharePoint developer used to have to go and download special files, install them, fixed issues, try installing again, fix more issues, have special machines or virtual machines to run SharePoint on and so on. The actual process of just writing code for SharePoint meant that you became elite because you had to go through a ritual of fire before you could start. Microsoft have really made SharePoint 2010 development simple and more importantly easy to start with, both by making SharePoint run natively on Windows 7 and also by including everything you need for development within Visual Studio 2010 from day one. It is as hard start wring code for SharePoint now as it is to make a WPF application!

What I am trying to convey is that previously a SharePoint developer had “paid their school fees” by learning so much that was so specific to the process of SharePoint development that they actually had earned a special title. Now that all those barriers have been removed, the title of SharePoint developer no longer applies, we are all just developers now!

Note worthy

Submitted by Robert MacLean on Wed, 11/18/2009 - 22:48

I have been very focused during the day on a project and my evenings have been taken up a lot with VSTS Rangers work so the blog has lagged a bit so here are some things you should be aware of (if you follow me on Twitter, then you probably have heard these in 140 characters or less):

I was awarded the title of VSTS Rangers Champion - this is a great honour since it is a peer vote from VSTS External Rangers (no Microsoft Staff) and MVP’s for involvement in the VSTS Rangers projects.

The VSTS Rangers shipped the alpha of the integration platform for TFS 2010 - this is important for me because it means some of the bits I have worked on are now public and I am expecting some feedback to get them better for beta and release next year. It is also important since my big contribution to the integration platform, which is an adapter I will cover in future blog posts, has a fairly stable base.

Dev4Dev’s in coming up in just over a week. This is one of my favourite events because it really is event for passionate developers since they have to give up a Saturday morning for it (no using an event to sneak off work). I will be presenting on Visual Studio 2010! Which should be great, based on my first dry run to an internal audience at BB&D last week. Two more of my BB&D team mates will be presenting Zayd Kara on TFS Basic and (if memory serves me) Rudi Grobler on Sketchflow!

The Information Worker user group is really blowing my mind with it’s growth, on Tuesday we had 74 people attend our meeting. For a community that only had a 100 or so people signed up on the website at the beginning of the year that is brilliant. Thanks must go to my fellow leads: Veronique, Michael, Marc, Zlatan, Hilton and Daniel. We will be having a final Jo’burg event for the year on the 2nd and it will be a fun ask the experts session.

Outlook 101 Poster

Submitted by Robert MacLean on Sat, 07/25/2009 - 15:14

poster

I created this poster (A3 in size) which covers the 8 key areas of Outlook

  • Email
  • Contacts
  • Outlook Web Access
  • SharePoint Integration
  • Calendar
  • Tasks
  • Misc Outlook Features, like out of office assistant and inline preview
  • RSS

The poster also has three areas where you can write key information that will be specific for your organisation like mailbox size limit, SharePoint & OWA URL’s.

Direct Download

More posters at www.drp.co.za