BizTalk How To : Calling A Receive Pipeline From Orchestration - Debatching Messages Inside Orchestration Using Pipeline

Post by: Stephen Thomas

For starter, why would you want to call a Receive Pipeline from within an Orchestration?  I had to struggle for a bit to come up with a good reason… I can find it useful in debatching Scenarios that require mapping prior to debatching or for debatching into smaller batches using a map.  I could also find it useful when working with flat file.

Limitations: Calling a Receive Pipeline inside the Orchestration does not support recoverable interchanges (more on this later) and it must be run inside an Atomic Scope.

Super Cool: Supports receiving multiple messages returned from the pipeline and can use enumeration to process each message.

The sample shown below receives a message of type XmlDocument into the Orchestration.  A Receive Pipeline is called to Debatch the message using an Envelope Schema.  A loop shape is used to enumerate over the resulting messages and send each single message.  In addition, references are needed to Microsoft.XLANGs.Pipeline and

Microsoft.BizTalk.Pipeline.

The CallPipeline Expression Shape contains the following line of code:

InputPipeline = Microsoft.XLANGs.Pipeline.XLANGPipelineManager.ExecuteReceivePipeline

(typeof(CallReceivePipeline.ReceivePipeline),msgFullMessage);

With InputPipeline defined as an Orchestration Variable of type Microsoft.XLANGs.Pipeline.ReceivePipelineOutputMessages

GetEachMessage will loop the collection using MoveNext like this:

InputPipeline.MoveNext();

Finally, the single messages are assigned inside the Message Assignment shape like this:

msgSingle = new System.Xml.XmlDocument(); 

InputPipeline.GetCurrent(msgSingle);

With msgSingle defined as an Orchestration Message of a specific schema type.

It is that simple!  In about 5 lines of code the Receive Pipeline can be executed inside on Orchestration in BizTalk 2006!

Read more ...

Biztalk : Using XPaths in Message Assignment

You can use the xpath function to assign an XPath value to a message part, or to assign a value to an XPath that refers to a message part. For more information on assigning to messages and message parts, see Constructing Messages.

Note  The use of the xpath function is not limited to message assignment. You can also use it in any expression, for example:

If ((System.Double) xpath(_RequestMessage.part, "number(//book[last()]/price)") == 75.00 && (System.Boolean) xpath(msgBoolean, "string(//boolean)") == false)...

Note  If you want to assign a value to a string, use the XPath string() function. For example:

myString = xpath(msg, "string(/*/book[1]/title)");

Note  The engine is not schema-aware, so you can only read values from or write values to a node that exists in the containing message (the complete path must exist), or the engine will raise an exception. This is true even if you supply a default value.

Assigning to an XPath in a message part

Consider the following schema:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="catalog">
    <xs:complexType>
      <xs:sequence>
        <xs:element minOccurs="1" maxOccurs="unbounded" name="book">
          <xs:complexType>
            <xs:sequence>
              <xs:element name="title" type="xs:string" />
              <xs:element name="author">
                <xs:complexType>
                  <xs:sequence>
                    <xs:element name="FirstName" type="xs:string" />
                    <xs:element name="LastName" type="xs:string" />
                  </xs:sequence>
                </xs:complexType>
              </xs:element>
              <xs:element name="price" type="xs:string" />
            </xs:sequence>
            <xs:attribute name="country" type="xs:string" />
          </xs:complexType>
        </xs:element>
      </xs:sequence>
    </xs:complexType>
  </xs:element>
</xs:schema>
You can use the XPath function as follows to set values on a document instance of that schema type:
construct _ResponseMessage
{
_ResponseMessage.part = _RequestMessage.part;
xpath(_ResponseMessage.part, "/*/book[1]/@country") = "USA";
xpath(_ResponseMessage.part, "/*/book[1]/title") = "Legends";
xpath(_ResponseMessage.part, "/*/book[1]/author/FirstName") = "A";
xpath(_ResponseMessage.part, "/*/book[1]/author/LastName") = "B";
xpath(_ResponseMessage.part, "/*/book[1]/price") = 50;
}

Assigning to a message part from an XPath

construct objMessage
{
objMessage.BooleanPart = xpath("false()");
objMessage.IntPart = xpath("100");
objMessage.StringPart = xpath("'Hello'");
objMessage.StringPart2 = xpath("'World'");
}

Using XPath to assign from nodes and node sets

You can also use XPath to assign XML nodes and node sets to an XML element, a class, or a schema-based or class-based message.
Suppose you have an XML-serializable class called Book, and consider the following examples:

[Serializable]
Class Book {...}

Example One — select the fourth book element from the catalog, and assign it to an XML element variable:

myXmlElement = xpath(myMsg, "/catalog/book[3]");

Example Two — select the fourth book element from the catalog, and convert it using XML deserialization into a Book class instance:

myBook = xpath(myMsg, "/catalog/book[3]");

Example Three — select the fourth book element from the catalog, and convert it a message of type Book:

myBookMsg = xpath(myMsg, "/catalog/book[3]");

Example Four — select all book elements in the catalog, where MyMethod takes an XmlNodeSet as a parameter:

MyMethod(xpath(myMsg, "/catalog/book"));

Example Five — add a book element to the "BookOfTheMonth" container:

xpath(MyMsg2, "/RecommendedBooks/BookOfTheMonth") = myBook;

Example Six — add all books that are priced at twenty or less to a set of recommended books:

xpath(MyMsg2, "/RecommendedBooks/BestPriceBooks") = xpath(MyMsg, "/catalog/book[@price <= 20]");

Example Seven — call user code that returns an XML element:

xpath(MyMsg2, "/RecommendedBooks/AdvertisedByPartner") = GetPartnerAdvertisedBook();

Before applying examples five and seven:

<RecommendedBooks>
       <BookOfTheMonth/>
       <BestPriceBooks/>
       <AdvertisedByPartner/>
</RecommendedBooks>

After applying examples five and seven:

<RecommendedBooks>
       <BookOfTheMonth>
              <Book country="USA">
                     <title>McSharry</title>
                     <author>
                            <FirstName>Nancy</FirstName>
                            <LastName>Jensen</LastName>
                     </author>
              </Book>
       </BookOfTheMonth>
       <BestPriceBooks/>
       <AdvertisedByPartner>
              <Book country="USA">
                     <title>The Rooster</title>
                     <author>
                            <FirstName>Mindy</FirstName>
                            <LastName>Martin</LastName>
                     </author>
              </Book>
       </AdvertisedByPartner>
</RecommendedBooks>

Source : MSDN

Read more ...

BizTalk Q & A: Process / Debatch Multiple Flat Files in a Single Receive Pipeline

Hi all, 

I have several different flat files I have to consume.  I would like to be able to drop them all into one receive location and use one flat file pipeline to disassemble them.  I created a receive pipeline and put in several flat file disassembler components, configuring each for one of the appropriate document schema.  I assumed one could do this from the language in BTS 2006 R2 help for the SchemaResolverComponent.  It reads: 

The flat file disassembler component normally requires you to define a parsing schema at design time. So if you expect to receive different flat file documents on the same receive location, you typically include several flat file disassemblers in the receive pipeline, one for each schema. At run time, the correct disassembler component is selected using a pipeline probing mechanism. However, this approach is expensive if you have many flat file schemas because probing for every corresponding disassembler component degrades pipeline performance.

 After deploying my pipeline, however, I got errors like the one below. 

Event Type: Error
Event Source: BizTalk Server 2006
Event Category: BizTalk Server 2006 
Event ID: 5753
Date:  7/12/2008
Time:  22:32:27
User:  N/A
Computer: 
Description:
A message received by adapter "FILE" on receive location "Receive Location_EMDEON.RPT4" with URI "D:\HIPAA_HOME\EMDEON_RPT4\*.*" is suspended. 
 Error details: An output message of the component "Flat file disassembler" in receive pipeline "HMOS.BizTalk.Pipelines.FF_ReceivePipeline.FlatFileReceivePipeline, HMOS.BizTalk.Pipelines.FF_ReceivePipeline, Version=1.0.0.0, Culture=neutral, PublicKeyToken=27ef7ae893a91467" is suspended due to the following error: 
     Unexpected data found while looking for:
'\n'
The current definition being parsed is Root. The stream offset where the error occured is 268. The line number where the error occured is 7. The column where the error occured is 12. .
 The sequence number of the suspended message is 1.  
 MessageId:  {F7B3BE2F-5504-40AE-8973-A08F073D6131}
 InstanceID: {3BB9BC41-2394-4C1B-A9F2-9DD97D7A7641}

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Evidently, the "pipeline probing mechanism" mentioned in BTS help is not part of the pipeline already?  How do I enable this one pipeline with multiple flat file dis-assembler components to be able to resolve which dis-assembler component to use for a given message?  (The SchemaResolverComponent in the SDK is one obvious answer, but I was really hoping to avoid it, if possible).

Sol:

I just tried to do the mimic you did

I created 3 – flat file schema  (delimited) with one receive pipeline with 3-dis-assemblers

It works fine with no extra configuration , the BTS will successfully recognize all message types

So I do not think your problem related to enable probing “it is already their”

But I notice the following points in this scenario

Lets say I have 3-flat file schema's (f1 , f2 , f3)

Where f3 instances also valid on f1 schema 

Pipeline dis-assembler process the list of dis-assembler in the same order you apply it on the design time

I created 2-pipeline with the following orders

Pipeline 1 (f1 , f2 , f3) this pipeline consider the f3 files as f1 files 

Pipeline 2 (f3, f2 , f1) this pipeline process all the messages successfully

Read more ...

FIX Biztalk Error: 'Microsoft.XLANGs.BaseTypes.BPELExportable': inconsistent duplicate module attribute

FIX Biztalk Error: 'Microsoft.XLANGs.BaseTypes.BPELExportable': inconsistent duplicate module attribute

While working on a solution we suddenly encountered the following error message while trying to build the code.

'Microsoft.XLANGs.BaseTypes.BPELExportable': inconsistent duplicate module attribute

It can be a nuisance to find the problem if you don't work with Biz Talk BPEL capabilities that often. It is however quite easy to fix. The error message is due to one orchestration having a different Module Exportable setting than the other orchestrations in the project. Making sure that the setting match between all orchestrations will make the error message disappear.

Read more ...

ROLLBACK FOR Store Procedure called From BizTalk || Implement Optional Transaction in StoreProcedure

A simple trick that you could try would be to test for an exisitance of a
transaction before starting a new one in the proc. But this would require
minor modifications to your stored procedure. For example:

 

At the top of the proc,

 

Declare @MyTran bit

 

-- If Caller Started the Transaction, let the caller handle it
-- otherwise control it in the proc.
If @@Trancount = 0
Begin
    Begin Transaction
    Set @MyTran = 1
End

 

At the bottom of your proc, to rollback or commit:

 

If @MyTran = 1 and @@Trancount > 0
 Begin
          Rollback Transaction -- Or Commit Transaction
 End

 

Read more ...

Composite Operations on BizTalk WCF- SQL Adapter - Calling Multiple Store Procedures

The new SQL Adapter in the WCF Adapter Pack 2.0 supports composite operations, that is performing multiple SQL operations in response to a single input message. The purpose of this blog post is to provide a walkthrough of how this works. I am using the public beta of BizTalk Server 2009, the public beta of the WCF Adapter Pack 2.0 and released version of SQL Server 2008. See below for an important caveat, as well as a link at the end of the post to my test solution.

To start, as with most things in BizTalk’s contract-first world, we need a schema. In order to do this, choose “add generated items” from a BizTalk project in Visual Studio. Then, choose “Consume Adapter Service”. If you don’t see that Visual Studio template, then you haven’t installed the Adapter Pack, as that’s where it comes from.

Next, select the sqlBinding specify a server, and press “Configure”. Set the client credential type to Windows (assume appropriate SQL login rights), and then on the URI tab, specified the server and database to use:

Af6ter doing this, press “Connect”, and the metadata will be populated.

For the purpose of this walkthrough, I have created 2 stored procedures: the first one inserts a record into a table, the second returns all rows in that table. Those are shown in the UI below.

Note that in the category we have “Procedures” and “Strongly-Typed Procedures”. The distinction is that “Procedures” will create un-typed schemas, whereas “Strongly-Typed Procedures” will generate schemas that you can work with inside BizTalk for mapping, promoting properties, etc.

The “Filename Prefix” will be used as a prefix for all the generated schemas.

After that was configured, I clicked OK and all the schemas were generated for me.

Next step is that you need to create a composite schema that will define the message you send to the adapter. I’m not quite sure why this one wasn’t generated for me, it’d be nice (hint hint), but it’s trivial to do.

How I did this for the walkthrough:

• create a new schema
• rename the root to SQLMsg (or whatever you like, this is unimportant)
• add a sibling record called SQLMsgResponse (this name does matter, it is the name of the request, with “Response” appended)
• add two child records under SQLMsg, and another two under SQLMsgResponse (names don’t matter, they’ll get renamed below)
• right-click the topmost “<schema>” node, and in the “Imports” property, add the “CompositeTypedProcedure.dbo.xsd” schema
• in the first child under SQLMsg, set the “Data Structure Type” property to InsertIntoDestination (this is a reference that you just imported above)
• in the second child under SQLMsg, set the “Data Structure Type” property to SelectAllDestination
• in the first child under SQLMsgResponse, set the “Data Structure Type” property to InsertIntoDestination (this is a reference that you just imported above)
• in the second child under SQLMsgResponse, set the “Data Structure Type” property to SelectAllDestinationResponse

Your schema should now look like this:

 

The, create an instance of the new composite schema to use as a test message, and populate the request. Here’s mine:

I then created a simple orchestration that would receive a request, call the adapter, and persist the response from the adapter. The request and response messages are of the type we just created in the composite schema:

Build and deploy the solution. After deploying it, note that there was a binding file generated along with the schemas, which is awesome, as this means you don’t need to manually create the send port. So, import the binding file which exists in you Visual Studio project.

HOWEVER… pretty big caveat here…. after importing the binding, you need to change the action mapping. If you use the default value, it will fail. You need to replace what is generated with the magic keyword “CompositeOperation”. This tells the adapter that it needs to call multiple operations, which it will resolve based on the schemas and namespaces. I believe the reason this works the way it does is that it allows you to import multiple operations in a single pass, and then use some subset of those operations in a composite operation, thereby enabling re-use of the generated schemas to potentially cover multiple different combinations of composite operations. Either way, watch out for this one. The error message tells you exactly what the problem is, however it won’t tell you about the keyword.

As an aside, and for the benefit of those who have not worked with this adapter yet, here are the binding configuration properties you have access to:

Then:

• create an inbound file drop location
• create an outbound file drop folder
• bind everything
• start the application
• drop your instance doc into the file drop location, triggering the orchestration

Lastly, here’s the output file:

In closing, I think this is an awesome new capability, and I am really liking the new SQL adapter. In case you haven’t heard, the old SQL adapter is being deprecated, so you really should be working with this one going forward.

Read more ...