Biz Talk : Dynamically creating receive locations

post by Brian Loesgon

Scenario:

There is a need to retrieve current tournament standings from the PGA Web site, pulling scores and statistics from them by FTP every 2 minutes while the tournaments were in progress. Ultimately, the standings would trickle through to a data-driven Web site. The interesting part here was the pickup location, there was a predefined FTP folder structure, with a separate folder for each tournament (the files were always the same name, but the locations would vary). We could have potentially multiple receive locations that become active during a given week, but only for a finite period of time, yet all pointing at different FTP locations. Biz Talk has dynamic send ports of course, but no dynamic receive locations. How to handle this? Clearly, although manually managing all this was possible, it really wasn't a viable solution given the amount of on-going effort it would require.

There is a overall tournament schedule file that will be posted on a weekly basis, and that's how the solution is driven. The solution basically:

• retrieves a schedule file (on a weekly basis) and publishes the schedule to the message box
• a CreateReveiveLocations orchestration subscribes to schedule files, and picks it up
• the orchestration maps the schedule to a structure that is easier to work with (we map inside the orchestration as we thought we may have other interesting things we could do with the schedule file in a future release)
• the orchestration removes any previously-created dynamic receive locations (based on a standard naming pattern, and they all belong to a known receive port)
• the orchestration calls a helper class that creates all the FTP receive locations. We know when the tournament starts, so:
  • we only create receive locations for current and future tournaments
  • we set the start date for the tournament to be the day of the tournament, so no polling will happen before the tournament starts

How cool is that? It's very cool to drop an XML document into BizTalk, and watch it create a bunch (and there are a LOT) of receive locations that it will subsequently use to drive a process. I'm all in favor of self-configuring environments!

We were in POC mode, and this took me a couple of days to get it all done (using good naming conventions, etc of course), and get it all working end-to-end. I'm sure I could have come up with a pretty good way to do this just in code, but it would have taken a LOT longer that 2 days.

Here's what the orchestration looks like:

 

There are two calls to a helper class above. The first removes any receive locations that were previously created (I use a naming convention and a unique name suffix, so I know which ones are dynamic). The second call passes in all the information required to create the new batch of receive locations.

The schedule file contained a tournament date. Because of that, we can skip creating FTP receives for tournaments that have ended.

Drilling into the CreateFtpReceiveLocation code, this snippet :

• first gets a reference to the receive port (which has a well know name)
• creates a new receive location
• sets the transport type to FTP

Note that if this were not a POC, I would have cached the ReceiveHandler and done other optimizations, however, even without optimizations, this creates hundreds of receive locations in seconds.

This last snippet:

• assigns the pipeline to use for this receive location
• set receive locations
• saves the changes to the BizTalk management database

There you have it. With a trivial amount of code, I eliminated what would have been significant ongoing manual intervention in the process. I fully automated what would have been a very tedious, and error-prone, task. The client was thrilled as this was a very elegant solution to an ongoing problem.

For anyone after the code rather that pretty pictures of it :), please see below.

-----------------------------

public static class LocationManagement
{
    const string CONNECTION_STRING = "Integrated Security=SSPI;Persist Security Info=False;Initial Catalog=BizTalkMgmtDb;Server=(local)";

    public static void CreateFtpReceiveLocations(string receivePortName, XmlDocument receiveLocations, string prefix, string server, string addressTemplate, string uid, string pwd)
    {
        XmlNodeList locs = receiveLocations.SelectNodes("//ReceiveLocation");
        DateTime cutoffDate = System.DateTime.Now.Add(new TimeSpan(-7,0,0,0));
        foreach (XmlNode loc in locs)
        {
            // if start date is more than 7 days ago, we don't need this one any more
            if (DateTime.Parse(loc.Attributes["StartDate"].Value) > cutoffDate)
            {
                CreateFtpReceiveLocation(receivePortName, prefix + loc.Attributes["Address"].Value, string.Format(addressTemplate, loc.Attributes["Address"].Value), server, uid, pwd, DateTime.Parse(loc.Attributes["StartDate"].Value));
            }
        }

    }

    public static bool CreateFtpReceiveLocation(string receivePortName, string receiveLocationName, string address, string server, string uid, string pwd, DateTime startDate)
    {
        try
        {
            BtsCatalogExplorer root = new BtsCatalogExplorer();
            root.ConnectionString = CONNECTION_STRING;

            //GetReceive Port
            ReceivePort receivePort = root.ReceivePorts[receivePortName];

            //Create a new receive location and add it to the receive port
            ReceiveLocation myreceiveLocation = receivePort.AddNewReceiveLocation(); ;
            myreceiveLocation.Address = address;
            myreceiveLocation.Name = receiveLocationName;

            //Receive Handler
            foreach (ReceiveHandler handler in root.ReceiveHandlers)
            {
                if (handler.TransportType.Name == "FTP")
                {
                    myreceiveLocation.ReceiveHandler = handler;
                    break;
                }
            }

            //Associate a transport protocol and URI with the receive location.
            ProtocolType protocol = root.ProtocolTypes["FTP"];
            myreceiveLocation.TransportType = protocol;

            Pipeline pipeline = root.Pipelines["Microsoft.BizTalk.DefaultPipelines.PassThruReceive"];
            myreceiveLocation.ReceivePipeline = pipeline;
            myreceiveLocation.StartDate = startDate;
            myreceiveLocation.StartDateEnabled = true;

            string ReceiveLocationTransportTypeData = "<CustomProps><AdapterConfig vt=\"8\">&lt;Config xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\"xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"&gt;&lt;uri&gt;ftp://BTS2006Dev:21/ClientName.PGA/*.xml&lt;/uri&gt;&lt;serverAddress&gt;BTS2006Dev&lt;/serverAddress&gt;&lt;serverPort&gt;21&lt;/serverPort&gt;&lt;userName&gt;administrator&lt;/userName&gt;&lt;password&gt;******&lt;/password&gt;&lt;accountName/&gt;&lt;fileMask&gt;*.xml&lt;/fileMask&gt;&lt;targetFolder&gt;ClientName.PGA&lt;/targetFolder&gt;&lt;representationType&gt;binary&lt;/representationType&gt;&lt;maximumBatchSize&gt;0&lt;/maximumBatchSize&gt;&lt;maximumNumberOfFiles&gt;0&lt;/maximumNumberOfFiles&gt;&lt;passiveMode&gt;False&lt;/passiveMode&gt;&lt;firewallType&gt;NoFirewall&lt;/firewallType&gt;&lt;firewallPort&gt;21&lt;/firewallPort&gt;&lt;pollingUnitOfMeasure&gt;Seconds&lt;/pollingUnitOfMeasure&gt;&lt;pollingInterval&gt;60&lt;/pollingInterval&gt;&lt;errorThreshold&gt;10&lt;/errorThreshold&gt;&lt;maxFileSize&gt;100&lt;/maxFileSize&gt;&lt;/Config&gt;</AdapterConfig></CustomProps>";

            XmlDocument docTransportTypeData = new XmlDocument();
            docTransportTypeData.LoadXml(System.Web.HttpUtility.HtmlDecode(ReceiveLocationTransportTypeData));

            docTransportTypeData.SelectSingleNode("//userName").InnerText = uid;
            docTransportTypeData.SelectSingleNode("//password").InnerText = pwd;
            docTransportTypeData.SelectSingleNode("//uri").InnerText = address;
            docTransportTypeData.SelectSingleNode("//serverAddress").InnerText = server;

            myreceiveLocation.TransportTypeData = ReceiveLocationTransportTypeData;

            //Enable the receive location.
            myreceiveLocation.Enable = true;

            //Save changes
            root.SaveChanges();

            return true;
        }
        catch (Exception ex)
        {
            throw new Exception("Error creating receive location", ex);
        }
    }