Moving 32-bit Biztalk Applications to 64 -nit Environiment

Q: Hi,

I have a Biztalk Application developed in a 32 bit OS and a 32 bit BizTalk server 2009 .Now my client wants to upgrade the OS and BizTalk Server to 64 bit, is it possible that my old application written in to work in 32 bit environment, work fine in a 64 bit OS and BizTalk Server(2009) ? I have tried to find answers to these myself, but they are not 100% sure. Are there any restrictions in doing so ? Any help is appreciated.

Thanks!

Ans:

From a development perspective, ensure that all of your assemblies are compiled to MSIL (Platform Target 'Any CPU' in Visual Studio), that way they are architecture independent - when they are run for the first time, the .Net JIT compiler will compile them down to the correct base architecture for execution; there will be a short time-delay while this occurs, but it will only happen the first time the assembly is executed. For reference see this thread.

You could still run your host instances as 32-bit hosts on a 64-bit installation.

Here is 32 and 64 bit support adapter's link http://msdn.microsoft.com/en-us/library/aa560166(v=bts.20).aspx

similar thread http://social.msdn.microsoft.com/Forums/en-US/biztalkgeneral/thread/2f85ccf6-fe79-4343-a43b-19e0ee678272/

HTH

Regards,

Read more ...

Setting Host throttling on Biztalk Server - Message Publishing - 1 Message Per Minute

Q:

Guys,

In my application, Biztalk receives files through file adapter. Each file has 500 records and I'm using xml disassembler to split them in to records in pipeline.

Now I have a question, if I want to receive ~ one file per minute, in Message Publishing Throttling Settings (For File Adapter Host), to set Minimum number of samples, Sampling window duration (milliseconds), Rate overdrive factor (percent), Maximum throttling delay (milliseconds), should I consider records as samples or files as  samples?

Means, Considering Files as Samples____Minimum number of samples == 1, Sampling window duration (milliseconds) == 60,000, Rate overdrive factor (percent)==100%, Maximum throttling delay (milliseconds) == 300,00,000 or ~8.5 hour.

Or I need to consider Records as sample and put Minimum number of samples == 500 and all other things as shown above.

Thanks, Rik

Ans:

Hi Rik,

Minimum number of messages BizTalk Server will sample for the Sampling window duration before considering rate-based throttling. If the actual number of samples in a sampling window fall below this value then the samples are discarded and throttling is not applied. This value should be consistent with a rate at which messages can be published under a medium load. For example, if your system is expected to handle 1,000 documents per second under a medium load, then this parameter should be set to 1,000 * Sample window duration in seconds (or more precisely, 1 * Sample window duration (milliseconds)). If the value is set too low, then the system may experience a throttling condition under low load. If the value is set too high, then there may not be enough samples for this technique to be effective.

For reference see http://msdn.microsoft.com/en-us/library/dd761093.aspx

It is true that it does clarify (make a distiction) frequency (number of documents) and size (number of records). So you will have to test values for having one document per minute i.e. 1 document per minute that results in publishing 500 records, so I would say you assumption is correct (Records as sample and put Minimum number of samples == 500). Still I would test it first.

HTH

Regards,

Read more ...

How to Cluster Host 's in Biz talk Environment

Post by Kent weare

Clustering BizTalk Hosts

I went through this configuration a few years ago before I started blogging. Since I had to build a parallel environment recently, I have decided to post this information.

So why would I want to Cluster a BizTalk Host?

If your BizTalk group needs to be Highly Available, but you want to limit a Host to only have 1 Host instance. The reason that you may want to do this is to prevent message duplication. Message duplication may occur when you have *some* adapters running in multiple Host Instances. Adapters such as POP3, FTP, MSMQ/MSMQT and database adapters in polling scenarios.

For example, if you have two BizTalk servers and they both have a Host Instance on them called "FTP", both Host Instances will trigger a connection to this FTP server. If a file exists on the FTP server that matches your Receive Location's file mask, both of these host instances will attempt to retrieve a copy of this file. The reason for this is that the FTP protocol does not support file locking.


The solution to this problem is to cluster the "FTP" Host so that only one Host instance is capable of running at one time. Since the Host is clustered, it is capable of running on the servers that are configured in your cluster administration configuration. In this "Active/Passive" scenario, the Host instance does exist on the "Passive" node, it is just stopped. The core Windows 2003 Cluster functionality takes care of determining whether the Host Instance is online and if it is not, it will try to start it on another node.

So there is a little background on why you need to do this, I will now go through the process of setting this up inside of BizTalk.

The first thing that you need to do is create a BizTalk Host. You do this the same way you would create a Non-Clustered Host.
You then need to create a Host Instance on every node in your cluster. Otherwise you will get presented with the following error:
Once you have created all of the host instances, go into the "Hosts" menu, find the Host that you want to cluster, right mouse click and select "Cluster".

You will then be prompted to provide the Clustered Resource Group that you want this Host to belong to. A resource group basically allows you to group services, or resources. You can then spread these Resource groups across multiple servers.

Once this is complete, you will see a new "Generic Service" added to the Clustered Resource group that was previously selected.



So what does this look like inside of the BizTalk Admin console? I have included a non-clustered host instance "ServiceProcess" just to illustrate that the icons are different. The next clue is that one "EnterpriseClusteredSQLReceive" Host instance has the caption of (Active) beside it. What this means is that this Host Instance is actively running on node "01A". It is not possible to try and start this host instance on multiple servers at the same time.

So in this post I have over simplified the Windows 2003/2008 clustering process. I have attacked it from the perspective that the OS level clustering has already been put into place. Don't under estimate the effort required to do this. To learn more about this process, check out the following link: Improving Fault Tolerance in BizTalk Server 2006 by Using a Windows Server Cluster

Read more ...

Determining the optimum number of BizTalk Hosts

post by Marcel

This post is written to get an overview of the different aspects one should take into consideration when creating multiple BizTalk hosts. There is no one-size-fits-all optimum for each BizTalk project. Generally it is a good practice to have a separate host for Receiving, Sending, Processing and Tracking and then add additional hosts as machine resources allow and tuning and managing require.

Hosts
The BizTalk Host object represents a logical set of zero or more runtime processes in which you can deploy services, pipelines, and other artifacts. The Host object also represents a collection of runtime instances (zero or more) where the deployed items physically run.

After you create a host (a logical container), you can add physical BizTalk servers (host instances) to the host.

Items—such as adapter handlers, receive locations (including pipelines), and orchestrations—contained in BizTalk hosts can perform the following functions:

Receiving. These items do the initial processing of messages after they are picked up in a receive location. When a host contains a receiving item, such as a receive location or pipeline, it acts as a security boundary, and the message decoding and decrypting occurs in a pipeline within the host.
Sending. These items do the final processing of messages before they are sent out to the send port. When a host contains a sending item, such as a send port or pipeline, the host acts as a security boundary, and the message signing and encryption occurs in a pipeline within the host.
Processing. These items process messages based on the instructions in an orchestration.
One BizTalk Host can contain items that receive, send, and process messages. It is recommended that you create different hosts for each function to create security boundaries and facilitate management. In particular, it is recommended that you use different hosts for processing and for receive/send, and that you separate trusted and non-trusted items.



Separating Host Instances by Functionality

In addition to the high availability aspects of the host instance configuration, you should separate sending, receiving, processing, and tracking functionality into multiple hosts. This provides flexibility when configuring the workload in your BizTalk group and is the primary means of distributing processing across a BizTalk group. This also allows you to stop one host without affecting other hosts. For example, you may want to stop sending messages to let them queue up in the MessageBox database, while still allowing the inbound receiving of messages to occur. Also, orchestration and adapter functionality should be separated into different BizTalk Server hosts to minimize resource contention.

Separating host instances by functionality also provides the following benefits:

· Each host instance has its own set of resources such as memory, handles, and threads in the .NET thread pool.
· Multiple BizTalk Hosts will also reduce contention on the MessageBox database host queue tables since each host is assigned its own work queue tables in the MessageBox database.
· Throttling is implemented in BizTalk Server at the host level. This allows you to set different throttling characteristics for each host.
· Security is implemented at the host level; each host runs under a discrete Windows identity. This would allow you, for example, to give Host_A access to FileShare_B, while not allowing any of the other hosts to access the file share.

Redundancy
This host should be run on at least two computers running BizTalk Server (for redundancy in case one fails). Depending on the type of host either fail-over or clustering should be used.



Disadvantages of additional hosts
While there are benefits to creating additional host instances, there are also potential drawbacks if too many host instances are created. Each host instance is a Windows service (BTSNTSvc.exe), which generates additional load against the MessageBox database and consumes computer resources (such as CPU, memory, threads).

Several performance counters are reported per host.

Host consumes memory (at least 200MB (?) per host)

Polling receive adapters will poll for each host that they are enlisted in

SQL connections: Each host instance will poll the db at intervals of 500ms.

Too many hosts will:

· Cause a heavy load on the SQL Server and possibly will prevent processing of messages
· Consume too much memory on the BizTalk servers leading to a throttling situation (reduces performance)
· Too many receive hosts that poll the receive location will degrade the performance
· Too much granularity in performance counters. This has negative impact on the overall view an administrator has.

Tune the Host polling interval
BizTalk Server uses a polling mechanism to receive messages from its host queues in the MessageBox. The MaxReceiveInterval value in the adm_ServiceClass table of the BizTalk Management (BizTalkMgmtDb) database is the maximum value in milliseconds that each BizTalk host instance will wait until it polls the MessageBox. The adm_ServiceClass table contains a record for the following service types:

XLANG/S – for BizTalk orchestration host instances
Messaging InProcess – for in-process host instances
MSMQT – for MSMQT adapter host instances
Messaging Isolated – for out of process host instances, used by the HTTP, SOAP, and certain WCF receive adapter handler

By default, this value is set to 500 milliseconds, which is optimized for throughput rather than low-latency. In certain scenarios, latency can be improved by reducing this value.

Note
Changes to this value impact all instances of the associated service type, therefore, take care to evaluate the impact on all host instances before changing this value.
Note
This value is only used if the MessageBox has no remaining unprocessed messages. If there is a constant backlog of unprocessed messages in the MessageBox, BizTalk Server will attempt to process the messages without waiting on the polling delay. After all messages are processed, BizTalk Server will begin polling using the value specified for MaxReceiveInterval.
Note
In a BizTalk Server environment with a high ratio of host instances to MessageBox database instances, decreasing the value for MaxReceiveInterval may cause excessive CPU utilization on the SQL Server computer that houses the MessageBox database instance. For example, if the MaxReceiveInterval is decreased to a low value (< 100) in a BizTalk Server environment with a single MessageBox and > 50 host instances, CPU utilization on the SQL Server may climb above 50%. This phenomenon can occur because the overhead associated with continually polling host queues is significant. If you reduce MaxReceiveInterval to a value less than 100, you should also evaluate the impact that this has on your SQL Server computer’s CPU utilization.

Quote: “(having many BizTalk hosts) seems like a bad solution. They will most likely see serious performance issues in the SQL layer as each host instance is “polling” for work. I have seen situations where many (maybe around 40) host instances were creating so much chatter with the MsgBox database that even when NO messages were being processed the CPU on the SQL box was at about 60%. Now, the customer did have the MaxReceiveInterval turned way down as well.”



Configuring a Dedicated Tracking Host
BizTalk Server is optimized for throughput, so the main orchestration and messaging engines do not actually move messages directly to the BizTalk tracking or BAM databases, as this would divert these engines from their primary job of executing business processes. Instead, BizTalk Server leaves the messages in the MessageBox database and marks them as requiring a move to the BizTalk Tracking database. A background process (the tracking host) then moves the messages to the BizTalk Tracking and BAM databases.

Advantages of Using a Dedicated Tracking Host
A BizTalk Host that hosts tracking is responsible for moving the DTA and BAM tracking data from the MessageBox database to the BizTalk Tracking (DTA) and BAM Primary Import databases. This movement of tracking data has an impact on the performance of other BizTalk artifacts running in the same host that is hosting tracking. Thus, you should use a dedicated host that does nothing but host tracking.

Using a dedicated tracking host also allows you to stop other BizTalk hosts without interfering with BizTalk Server tracking.



Best practices
Make your plan on how to separate hosts beforehand and how to name them. It is not possible to rename hosts.

Create separate hosts for different functions.

· Receiving
· Sending
· Processing
· Tracking



Add hosts for protocol or application specific requirements.

· Prioritization
· Batching vs. messaging
· Low latency
· Protocols that require clustering
· Protocols that require scale-out
· Separate hosts for applications with known or expects problems
When problematic applications are in a separate host, crashes will not affect applications in other hosts.
· Licensing of adapters
Certain 3rd party adapters are licensed separate from BizTalk Server. Separating per host can be a cost-effective deployment scheme.



Resources
MSDN: Hosts
http://msdn.microsoft.com/en-us/library/aa578695.aspx

MSDN: Host instances
http://msdn.microsoft.com/en-us/library/aa560673.aspx

Microsoft BizTalk Server Operations Guide
http://msdn.microsoft.com/en-us/library/cc296643.aspxMicrosoft BizTalk Server Performance Optimization Guide
http://msdn.microsoft.com/en-us/library/cc558617.aspx

Read more ...

post by Marcel


Determining the optimum number of BizTalk Hosts

This post is written to get an overview of the different aspects one should take into consideration when creating multiple BizTalk hosts. There is no one-size-fits-all optimum for each BizTalk project. Generally it is a good practice to have a separate host for Receiving, Sending, Processing and Tracking and then add additional hosts as machine resources allow and tuning and managing require.

Hosts
The BizTalk Host object represents a logical set of zero or more runtime processes in which you can deploy services, pipelines, and other artifacts. The Host object also represents a collection of runtime instances (zero or more) where the deployed items physically run.

After you create a host (a logical container), you can add physical BizTalk servers (host instances) to the host.

Items—such as adapter handlers, receive locations (including pipelines), and orchestrations—contained in BizTalk hosts can perform the following functions:

Receiving. These items do the initial processing of messages after they are picked up in a receive location. When a host contains a receiving item, such as a receive location or pipeline, it acts as a security boundary, and the message decoding and decrypting occurs in a pipeline within the host.
Sending. These items do the final processing of messages before they are sent out to the send port. When a host contains a sending item, such as a send port or pipeline, the host acts as a security boundary, and the message signing and encryption occurs in a pipeline within the host.
Processing. These items process messages based on the instructions in an orchestration.
One BizTalk Host can contain items that receive, send, and process messages. It is recommended that you create different hosts for each function to create security boundaries and facilitate management. In particular, it is recommended that you use different hosts for processing and for receive/send, and that you separate trusted and non-trusted items.



Separating Host Instances by Functionality

In addition to the high availability aspects of the host instance configuration, you should separate sending, receiving, processing, and tracking functionality into multiple hosts. This provides flexibility when configuring the workload in your BizTalk group and is the primary means of distributing processing across a BizTalk group. This also allows you to stop one host without affecting other hosts. For example, you may want to stop sending messages to let them queue up in the MessageBox database, while still allowing the inbound receiving of messages to occur. Also, orchestration and adapter functionality should be separated into different BizTalk Server hosts to minimize resource contention.

Separating host instances by functionality also provides the following benefits:

· Each host instance has its own set of resources such as memory, handles, and threads in the .NET thread pool.
· Multiple BizTalk Hosts will also reduce contention on the MessageBox database host queue tables since each host is assigned its own work queue tables in the MessageBox database.
· Throttling is implemented in BizTalk Server at the host level. This allows you to set different throttling characteristics for each host.
· Security is implemented at the host level; each host runs under a discrete Windows identity. This would allow you, for example, to give Host_A access to FileShare_B, while not allowing any of the other hosts to access the file share.

Redundancy
This host should be run on at least two computers running BizTalk Server (for redundancy in case one fails). Depending on the type of host either fail-over or clustering should be used.



Disadvantages of additional hosts
While there are benefits to creating additional host instances, there are also potential drawbacks if too many host instances are created. Each host instance is a Windows service (BTSNTSvc.exe), which generates additional load against the MessageBox database and consumes computer resources (such as CPU, memory, threads).

Several performance counters are reported per host.

Host consumes memory (at least 200MB (?) per host)

Polling receive adapters will poll for each host that they are enlisted in

SQL connections: Each host instance will poll the db at intervals of 500ms.

Too many hosts will:

· Cause a heavy load on the SQL Server and possibly will prevent processing of messages
· Consume too much memory on the BizTalk servers leading to a throttling situation (reduces performance)
· Too many receive hosts that poll the receive location will degrade the performance
· Too much granularity in performance counters. This has negative impact on the overall view an administrator has.

Tune the Host polling interval
BizTalk Server uses a polling mechanism to receive messages from its host queues in the MessageBox. The MaxReceiveInterval value in the adm_ServiceClass table of the BizTalk Management (BizTalkMgmtDb) database is the maximum value in milliseconds that each BizTalk host instance will wait until it polls the MessageBox. The adm_ServiceClass table contains a record for the following service types:

XLANG/S – for BizTalk orchestration host instances
Messaging InProcess – for in-process host instances
MSMQT – for MSMQT adapter host instances
Messaging Isolated – for out of process host instances, used by the HTTP, SOAP, and certain WCF receive adapter handler

By default, this value is set to 500 milliseconds, which is optimized for throughput rather than low-latency. In certain scenarios, latency can be improved by reducing this value.

Note
Changes to this value impact all instances of the associated service type, therefore, take care to evaluate the impact on all host instances before changing this value.
Note
This value is only used if the MessageBox has no remaining unprocessed messages. If there is a constant backlog of unprocessed messages in the MessageBox, BizTalk Server will attempt to process the messages without waiting on the polling delay. After all messages are processed, BizTalk Server will begin polling using the value specified for MaxReceiveInterval.
Note
In a BizTalk Server environment with a high ratio of host instances to MessageBox database instances, decreasing the value for MaxReceiveInterval may cause excessive CPU utilization on the SQL Server computer that houses the MessageBox database instance. For example, if the MaxReceiveInterval is decreased to a low value (< 100) in a BizTalk Server environment with a single MessageBox and > 50 host instances, CPU utilization on the SQL Server may climb above 50%. This phenomenon can occur because the overhead associated with continually polling host queues is significant. If you reduce MaxReceiveInterval to a value less than 100, you should also evaluate the impact that this has on your SQL Server computer’s CPU utilization.

Quote: “(having many BizTalk hosts) seems like a bad solution. They will most likely see serious performance issues in the SQL layer as each host instance is “polling” for work. I have seen situations where many (maybe around 40) host instances were creating so much chatter with the MsgBox database that even when NO messages were being processed the CPU on the SQL box was at about 60%. Now, the customer did have the MaxReceiveInterval turned way down as well.”



Configuring a Dedicated Tracking Host
BizTalk Server is optimized for throughput, so the main orchestration and messaging engines do not actually move messages directly to the BizTalk tracking or BAM databases, as this would divert these engines from their primary job of executing business processes. Instead, BizTalk Server leaves the messages in the MessageBox database and marks them as requiring a move to the BizTalk Tracking database. A background process (the tracking host) then moves the messages to the BizTalk Tracking and BAM databases.

Advantages of Using a Dedicated Tracking Host
A BizTalk Host that hosts tracking is responsible for moving the DTA and BAM tracking data from the MessageBox database to the BizTalk Tracking (DTA) and BAM Primary Import databases. This movement of tracking data has an impact on the performance of other BizTalk artifacts running in the same host that is hosting tracking. Thus, you should use a dedicated host that does nothing but host tracking.

Using a dedicated tracking host also allows you to stop other BizTalk hosts without interfering with BizTalk Server tracking.



Best practices
Make your plan on how to separate hosts beforehand and how to name them. It is not possible to rename hosts.

Create separate hosts for different functions.

· Receiving
· Sending
· Processing
· Tracking



Add hosts for protocol or application specific requirements.

· Prioritization
· Batching vs. messaging
· Low latency
· Protocols that require clustering
· Protocols that require scale-out
· Separate hosts for applications with known or expects problems
When problematic applications are in a separate host, crashes will not affect applications in other hosts.
· Licensing of adapters
Certain 3rd party adapters are licensed separate from BizTalk Server. Separating per host can be a cost-effective deployment scheme.



Resources
MSDN: Hosts
http://msdn.microsoft.com/en-us/library/aa578695.aspx

MSDN: Host instances
http://msdn.microsoft.com/en-us/library/aa560673.aspx

Microsoft BizTalk Server Operations Guide
http://msdn.microsoft.com/en-us/library/cc296643.aspxMicrosoft BizTalk Server Performance Optimization Guide
http://msdn.microsoft.com/en-us/library/cc558617.aspx

Read more ...