Pages

Showing posts with label Performance. Show all posts
Showing posts with label Performance. Show all posts

BizTalk : How To : Cluster BizTalk Server Databases

Wednesday, March 20, 2013

If the BizTalk Server databases become unavailable, the BizTalk Server environment will not function correctly. To provide high availability, you can create a Microsoft SQL Server cluster for the BizTalk Server databases, as shown in the following figure.

BizTalk Server Database Tier

To create a highly available solution for the BizTalk Server databases, you must have at least two computers that are running SQL Server and a shared disk array in the cluster.
Determine the best cluster configuration for the BizTalk Server databases for your business needs. Here is a list of the options:
  • Active/passive. High availability for the BizTalk Server databases typically consists of two or more database computers configured in an active/passive server cluster configuration. These computers share a common disk resource (such as a RAID 1+0 SCSI disk array or storage area network) and use Windows Clustering to provide backup redundancy and fault tolerance.
  • Active/active. Windows Clustering and SQL Server allow you to run SQL Server in Active/Active mode where each node of the cluster is “active” and running one or more SQL Server instances. For example, this would allow you to have the MessageBox database on one node and all other BizTalk Server databases on the other node. This allows you to maximize cluster hardware usage, but an active/active SQL Server configuration should be used with care.

    Can each node simultaneously handle the load of all SQL Server instances during a SQL Server cluster node failover scenario? Are there enough CPU resources? Is there sufficient memory? What about network bandwidth? How about disk I/O contention?

    These are just some of the questions that need to be answered in order to determine if an active/active SQL Server cluster is right for your BizTalk applications. If it is determined that one node cannot handle all SQL Server instances in a failover scenario, an alternative is to use active/active/passive clustering.
  • Active/active/passive. The run-time processes write to the BizTalk Management database, MessageBox databases, Tracking Analysis Services database, BAM Analysis database, BAM Star Schema database, BAM Primary Import database, and BAM Archive database. Therefore, these databases are especially important if a disaster occurs, and must have greater priority when determining what databases to cluster. Only users or tools write to the other databases. For the MessageBox databases, you can consider an active/active/passive or active/active/active/passive configuration to minimize the hardware needed.
Gg634455.note(en-us,BTS.70).gifNote
SQL Server 2008 R2 and SQL Server 2008 SP1 Standard Edition supports 2-node failover clusters. If you decide to use the active/active/passive configuration on SQL Server 2008 R2 or SQL Server 2008 SP1, you must use the Enterprise Edition of SQL Server.
Make sure you meet the following prerequisites before you start clustering the BizTalk Server databases.
  • When you create the domain groups for your BizTalk Server environment, you must create global domain accounts.
  • Configure the SQL Server cluster before you install and configure BizTalk Server. For more information about clustering SQL Server 2008 R2, see Getting Started with SQL Server 2008 R2 Failover Clustering (http://go.microsoft.com/fwlink/?LinkId=156820).
  • If you are also clustering the master secret server, configure that server first. For more information about high availability for Enterprise Single Sign-On, see High Availability for the Master Secret Server.
  1. Install BizTalk Server 2010 on a runtime server.
  2. Launch the BizTalk Server Configuration program. Click Start, point to Programs, point to Microsoft BizTalk Server 2010, and then click BizTalk Server Configuration.
  3. To apply a custom configuration, follow the steps in Working with the Custom Configuration Manager (http://go.microsoft.com/fwlink/?LinkId=156822) in BizTalk Server Help. To specify the SQL Server cluster for the BizTalk Server databases enter the name of the SQL Server cluster in the Databases dialog of the configuration.
  4. Complete the BizTalk Server configuration by following the instructions in Custom Configuration (http://go.microsoft.com/fwlink/?LinkId=156823) in BizTalk Server Help.
For more information about clustering BizTalk Server databases, see Improving Fault Tolerance in BizTalk Server by Using a Windows Server 2008 Failover Cluster or Windows Server 2003 Server Cluster (http://go.microsoft.com/fwlink/?LinkId=156819). 
For more information about behavior of BizTalk host instances during SQL Server failover, see Behavior of BizTalk Server Host Instances during SQL Server Failover(http://go.microsoft.com/fwlink/?LinkId=151287).
For more information about using SQL Server database mirroring with respect to BizTalk Server database clustering, see Use of SQL Server database mirroring or the Volume Shadow Copy service (http://go.microsoft.com/fwlink/?LinkId=151288) in BizTalk Server help.
Read more ...

Advanced Users Biztalk WIKI : Useful How to Links Part - 3

Thursday, June 14, 2012
Useful How to Links Part - 3

BizTalk Administrator's Checklist Compiled by Microsoft BizTalk Support

Muenchian Grouping and Sorting in BizTalk Maps

Xslt Template StyleSheet For Biztalk Mapping : Query in Biztalk Map

AppFabric-enabled WCF Data Service Walkthrough (C#)

Querying DataSets – Introduction to LINQ to DataSet

Accessing BRE RuleEngine Using .Net Framework
Few links that may help:
http://msdn.microsoft.com/en-us/library/aa995566.aspx                                                                

(MSDN: Walkthrough: Executing the Policy Programmatically)

Read more ...

Advanced Users Biztalk WIKI : Useful How to Links Part - 4

Friday, June 8, 2012
Useful How to Links Part - 4

Iterate ArrayList in BRE Business Rules

Programming with Business Rules Framework

How to Script Import/Export and Deployment of Business Policies using FileRuleStore

SSO Configuration Store Application





Tool Source Code:

Here are a few TechNet Wiki articles that have become available recently. I have usually found it difficult to find relevant content on the TechNet Wiki so I thought it would be good to highlight a few of these.
Improving performance when executing a BRE rule:

Implement ordered messaging with BizTalk and SSIS:

BizTalk load testing in VS 2010:

Load testing for simultaneous BizTalk unit tests:

Read more ...

Advanced Users Biztalk WIKI : Useful How to Links Part -2

Wednesday, June 6, 2012
Useful How to Links Part -2


Implementing Biztalk Pipeline Trace

Biztalk Orchestration Tracing Using Biztalk Diagnostics Library

High Performance Message Transform Alternative to XslTransform vs Xsl 
CompileTransform

Dynamic Mocking With N-Mock

Biztalk 2010 Documentation

Validating Incoming Xml using BRE

BizTalk: Delivery Notification in Direct Send Ports

BizTalk XSLT Reuse (xsl:include) by using MS Build



BizTalk Server: Performance Tuning & Optimization
http://social.technet.microsoft.com/wiki/contents/articles/7253.biztalk-server-2010-database-biztalkdtadb.aspx
Read more ...

BizTalk Performance Analysis Using PAL Tool, Walk Through Using PAL Tool

Friday, June 1, 2012
Post By Clint Huffman

BizTalk Performance Counters

Microsoft BizTalk Server 2006 shipped with about 294 performance counters. This means a BizTalk Server implementation with at least 2 servers for redundancy means there are at least 588 BizTalk performance counters that may need to be analyzed. Therefore, a performance monitor log analysis tool is helpful with analyzing BizTalk performance counters.

Consider the Operating System Performance

Many BizTalk performance issues can be narrowed down by analyzing the resources of the operating system (CPU, disk, memory, and network). For example, if the tracking (DTADB) database file is heavily using a disk, then reducing the amount of tracking would be a logical step towards alleviating the bottleneck.

Overview 

The PAL (Performance Analysis of Logs) tool is a powerful tool that reads in a performance monitor counter log and analyzes it using known thresholds.  A template is created from the PAL tool and imported to the server’s Performance tool as a new counter.  This includes relative BizTalk and SQL metrics that will be tracked.  The resulting blg files are analyzed by the PAL tool and thresholds are applied once questions are answered regarding number of CPU, memory, etc.  A report is produced by the PAL tool in an easy to read format with graphs.  Tallan will use this information to make recommendations for performance improvements. 

Features 

Threshold files take the questions answered in the tool and apply them to the results.  This is a quick and easy way to see where the weaknesses are without lengthy analysis. 
An easy to use GUI interface which makes creating batch files for the PAL.ps1 script.  This GUI also creates a template Perfmon template and allows the user to answer questions specific to the environment such as CPU count, RAM, etc. 
A GUI editor for creating or editing your own threshold files.  This allows the user to modify, add, and remove threshold values and files for a more custom implementation. 
Creates a HTML based report for ease of copy/pasting into other applications. 
Analyzes performance counter logs for thresholds using thresholds that change their criteria based on the computer’s role or hardware specs. 

Required Downloads 

Microsoft .NET Framework 3.5 Service Pack 1 (Partial package – internet access required)
http://www.microsoft.com/downloads/details.aspx?familyid=AB99342F-5D1A-413D-8319-81DA479AB0D7&displaylang=en
Microsoft .NET Framework 3.5 Service Pack 1 (full package – no internet access required)
http://download.microsoft.com/download/2/0/e/20e90413-712f-438c-988e-fdaa79a8ac3d/dotnetfx35.exe
PowerShell v2.0 (Windows Management Framework (Windows PowerShell 2.0, WinRM 2.0, and BITS 4.0))
http://support.microsoft.com/kb/968929 

Installation 

Download PAL from http://pal.codeplex.com/releases/view/51623 : developed by Clint Huffman
Unzip contents and run the installation msi.  Note, only supported on x32 machines – this requirement does not restrict using product on x64 machines because the log files can be processed locally. 

Usage 

Open PAL from installation directory 

 

Click on ‘Threshold File’ tab, select ‘Microsoft BizTalk Server 2006/2009/2010’ in the Threshold File Title drop down.  Click ‘Export to Perfmon Template File’.  Note, optionally one of the features is to edit the template file – click ‘Edit’ and add/remove metrics.

 
Select the file location: \\servername\e$\PAL\perfmonTemplate or the central directory on the server environment you will be testing on.


 
Create Perfmon Log from Exported Template 

Remote Desktop into the server you want to performance the analysis on (or continue on local machine) and open Perfmon tool – expand Performance Logs and Alerts.


 
Right click on ‘Counter Logs’, ‘New Log Settings From…’



Select the template that was previously created.

Click Ok

A new dialogue box will open, Click ‘Log Files’ tab and click ‘Configure’ to set the location of the Perfmon log (blg) files.


Set the location of the log files.  IMPORTANT – use a network drive so that there is no drive IO interference.  Here we used E:\PAL\PerfmonLogFiles directory.  Click OK.


Click on ‘Schedule’ tab and set when the Perfmon should run.  Once the log file is created, we will go back to our local environment and load it with PAL tool.


After the log completes, go to the directory that was set up above and verify blg file exists.

Run PAL Against Log File
Go back to your PAL tool on local machine and click on ‘Counter Log’ tab.  Click on path button and select the log file that was created in the above step by the server.


Click Open

Click tab ‘Questions’.  This will allow you to answer preset questions that will provide the threshold variables by the tool.  Answer the questions by entering values for each.


Next, Click ‘Output Options’ and set the time slice Analysis interval.  Use AUTO to automatically separate into 30 time slices.  This will show as a data point for each metric (30 in this case).

Next, Click ‘File Output’ tab and select the Output Directory to store the HTM report. 


Click tab ‘Execute’, make sure ‘Execute: Execute what is currently in the queue’ and click ‘Finish’ to start the powershell scripts.  After scripts are completed, browse to the report directory set up previously if report does not automatically open.


Open Report

View Report and Analyze
A Report with thresholds and alerts will open. 

Usage For Windows Server 2008 PerfMon

The PAL tool is can be used to create a Microsoft Performance Monitor (perfmon) template file from PAL threshold files and can analyze the perfmon log after its collection period. In this example, we will create, gather, and analyze a BizTalk performance monitor log file.

Collect a Microsoft Performance Monitor (perfmon) log

1. Export a Microsoft Performance Monitor Log from the BizTalk Server threshold file.
a. Selecting the Microsoft BizTalk Server Server 2006 threshold file and clicking the Export… button, then save the file. This is a perfmon template file. 

image

2. Copy the perfmon template file to the BizTalk server(s) you have chosen for analysis. This includes the Microsoft SQL Servers hosting the BizTalk databases.

3. Create a new performance monitor log using the template exported in step 

 

 


 
Then Right Click on User Defined template And
START

Let PerfMon to Run Adequate Time And then collect the PerfMon report

1. For more information on how to create performance monitor logs from a template file, please refer to the Windows help documentation.
a. Adjust the perfmon log settings if needed.

4. Start the new performance monitor log. Stop the perfmon log when the collection period is over.

5. Copy the perfmon log to the installation directory of the PAL tool.

Filling out the PAL Wizard Form
image 
  1. Counter Log Path: Specify the path to the Microsoft Performance Monitor (perfmon) log. The log can be in any of the known perfmon log formats such as BLG (binary) or CSV (text). Use the ellipsis button to browse for the perfmon log. If the perfmon log file is in the installation directory of PAL, then use the drop down arrow to select one of the perfmon log files.
    Note: Multiple log files can be merged by separating them with semicolons, but this may produce unpredictable results. PAL is best used with logs containing data from a single computer.
  2. Date/Time Range: This is the date/time range that you can restrict the analysis to. For example, if you did load testing on BizTalk during a specific time.
  3. Threshold File Title: This is the threshold file that you want to use to determine thresholds that will be analyzed by PAL. If you do not see a product specific threshold file, then consider using the default System Overview threshold file. The System Overview threshold file analyzes the basic operating system performance counters only.
  4. Question Variable Names: The question variables will be different depending on which threshold file is selected. Click on each question variable name and answer the question in respect to the perfmon log chosen earlier.
  5. Analysis Interval: Specify the time (in seconds) that you want the PAL tool to analyze the perfmon log. Choose “AUTO” to have PAL automatically detect and choose an appropriate interval size. Choose “ALL” to have PAL analyze all of the data in the perfmon log. Be careful about choosing “ALL” because it is extremely resource intensive. The Analysis Interval determines how the perfmon log will be broken up when analyzed. For example, if you gathered a 24 hour log and choose a 1 hour analysis interval, then PAL will analyze each hour in the log for minimum, maximum, average, and thread values for that time interval. The data points in the charts generated by PAL are based on the analysis interval. “AUTO” is the default and is recommended.
  6. Output Options: You can optionally specify an output directory for the output, the file name format for the HTML report, and an optional XML document output. The XML output is useful if you want to analyze the results using another tool.
  7. The Queue: The queue is really just a batch file with line feeds for readability.
  8. Execute:
    • Execute: Execute what is currently in the queue.
    • Add to Queue: Do not execute yet, but add the current configuration into the queue and restart the wizard allowing you to add more. This is useful if you intend to "batch" the processing while you go to lunch or overnight.
    • Execute and Restart: This executes what is in the queue now and restarts the wizard. This is useful if you want to start processing one of the logs right away while answering the questions for the other logs.
  9. Execute as a low priority process: This will run the PAL.vbs script at a low process priority so that when PAL.vbs is running, it will have the little to no impact on your computer. PAL processing can be very resource intensive. This option will make the processing take longer.
 
Interpreting the Report

The report generated by the PAL tool is simply an interpretation of the perfmon log data using generalized thresholds. The intention is to assist with performance analysis, but not to replace traditional performance analysis. Therefore, you should have a working knowledge of the BizTalk architecture and BizTalk performance analysis.
The thresholds used in the BizTalk threshold file include operating system thresholds (CPU, disk, memory, and network), Microsoft SQL Server counter thresholds, and BizTalk counter thresholds. The BizTalk thresholds generally focus on host throttling, adapter latency, service instance statistics (suspended, dehydrated, etc.), database sizes, and memory usage.
The PAL report is separated by categories and in each category is a collection of analyses. Each analysis focuses on a specific performance counter. If any of the thresholds are exceeded, then an alert is raised. The number of alerts in each analysis and the alert condition (typically Warning or Critical) typically indicate the severity of the results. For example, a critical alert in Pool Paged Bytes is a very serious condition and should be resolved immediately. When critical alerts occur, go to the analysis section associated to the alert. In the analysis section, there is typically a description of the analysis, a description of the thresholds used, and a link for more information on the topic.
The analyses in the report contain content describing the purpose of the analysis, why the thresholds are there, and references to more information. To learn more about interpreting the PAL report for BizTalk analysis, then read the following article:

Using the Performance Analysis of Logs (PAL) Tool
http://msdn.microsoft.com/en-us/library/cc296652.aspx
Read more ...