Flat files are extensively used in BizTalk Projects, In this article I will be talking about some of the scenarios in which Flat files are used

In this example the Flat file which I will be taking is of the following structure:

Case 1 : Message only scenario Input Flat File --> OutPut Flat File

 Your Flat file contains a number of records each separated by some delimiter character. For this you can have a schema like the one below, created from the “Flat File Schema Wizard”

Error Handling  

If any of the record is not according to Schema, then the same message is also there in its original format at the send Location. No messages suspended.  

Case 2 Message Only Scenario Input Flat File --> Output N Flat or N Xml Files or 1 Flat or 1 Xml File

In this case the Flat file schema is slightly different and the “CustomersDetails” Record is not set to be “Unbounded”. In this case BizTalk will process the Flat file in a different way.  

Observation:  

In this case there will be N messages created (not one as in the Case First).

N = Number of records in Flat Files

So after this

• If you have a PassThroughTransmit sendPipeline, you will see N Xml files (Not n Flat File) in the send location.

  N = Number of records in Flat Files

• If you have a custom send pipeline with Flat File Assembler and a map applied on it, the transformed messages (as Flat Files) are saved in the send location; the Map is applied on individual record.

• If you want the Flat files as a whole also in the send Location, just change the “Copy Mode” property to “Append”, this will create a Flat file with all the records. A lot of Disk Access, to write individual record. L

Error Handling

If any of the record is not according to Schema, then nothing happens and the record is appended to the Flat File in the second location. No messages suspended.

If how ever you look at the xml messages formed after the receive Pipeline, the message is really Scrambled.

     For ex.

Case 3 Orchestration Scenario Input Flat File --> Single Records to be consumed by Orchestration

In this case we will see the following features with Flat Files

If we want to use individual records in a Orchestration from a Flat File  

We have to create a Flat File Schema with its

Max occurrence set to 1

So if you have N records in a Flat File, then N Orchestrations instances will be created, passing one record to each of the Orchestration instance.  

Error Handling  

          In this case the benefit is that if any one of the record in the Flat file is not according to the Schema expected, only that message is suspended, and all other messages are received by the Orchestration.  

Just set the following properties of the Flat file Dissembler  

With the release of 3.0 there are some talks on the future of BizTalk server, how we will be integrating BizTalk with new technologies like WF and WCF in 3.0?

And so Microsoft is coming up with BizTalk Server 2006 R2. In this article I will be focusing on some of the new features of BTS 2006 R2 which helps us to integrate BizTalk Server R2 with 3.0. 

These are some good article on When to use "Windows Workflow Foundation" and when to use "BizTalk Server" ?

I am putting here a basic example in which BizTalk server is integrated with 3.0 (WF and WCF).

BizTalk Server 2006 R2 & Windows Workflow Foundation (WF)

There are various ways in which we can communicate b/w BizTalk Server 2006 R2 and Windows Workflow Foundation.

1. Hosting WF inside BizTalk Server
   1. Create the WorkFlow in Windows WorkFlow foundation
   2. Call the Workflow from the Assembly
   3. Use the Hosting Assembly (Class) in the BizTalk Server
2. Exposing WF as a Web Service hosted inside IIS
   1. Create the WF
   2. Publish rge WF as a Web Service
   3. Call the WS from the BizTalk using SOAP, http adapter
3. Hosting WF inside WCF Service
   1. Create the WF
   2. Host the WF inside the WCF Service
   3. Call the WCF Service from the BizTalk using WCF adapter
4. Consuming an Orchestration exposed as WS in WF
   1. Create an Orchestration and expose it as a WS
   2. Consume the WS as a simple WS in WF 

Hosting WF inside BizTalk server  

This option provides the simplest of the option to call WF from BizTalk.  

First, create any Workflow in Windows Workflow foundation. Don’t forget to strong name the assembly as we have to put it in the GAC (BizTalk picks dll’s from GAC). This WF assembly now needs to be called from a class which will then be used in the BizTalk.

In this class just create the Workflow Runtime, create an instance of the WF and start it.  

So the flow of development will be:  

WF-à Class--à BizTalk Server  Orchestration

Also, if you want to pass any parameters to WF from BizTalk, the flow will be like this  

BizTalk -à Class-à WF  

As WF is executed in an asynchronous manner, you cannot pass the parameters back to the BizTalk in a synchronous manner, for this what you can have do is,

First call the class from the Expression box in BizTalk. When ever the WF will complete, it will dump the information at Ftp, File, MSMQ etc and then have a Receive shape in BizTalk polling at the required location. You will have to consider Correlation and Sequential Convoy for this  

The following code an extract of the code which is ised in the Class

Exposing WF as a Web Service hosted inside IIS

First create the WF. You will be using some specific activities for exposing your WF as Web Service like WebServiceInput, WebServiceOutput. Publishing the WF as a WS is not that difficult as Microsoft has provided an option to publish. After publishing the WS, you can always use this WS in any application like ASP.NET, Windows Application and off course our BizTalk Orchestration also using SOAP, HTTP adapter.

This option is better as the WF then can be used by many application, and not only BizTalk. Also you can always scale the Web Server architecture.

This option has some advantages over the previous one, as we can take the advantage of Load Balancing Servers for the Web Service also. 

There are still some issues on integrating BizTalk Server and Windows Workflow Foundation.  

1. How can we integrate the tracking data in BTS and WF?
2. Tracking data in WF is stored in SQL Server in a separate Db just as BTS.
   1. But I think there should be some sort of difference in that?
3. Can we some how use HAT to have a consolidate view for both BTS and WF?
4. I think Microsoft should be doing some thing to integrate these?
5. Can we some how use BAM across WF and BizTalk for over all monitoring the Business?
6. Yes I know BAM API’s is a good option, but can’t we do this with out using it?
7. Just like in BizTalk from Tracking Profile editor?
8. Is Microsoft going to provide something like BAM in WF?
9. What’s the future of BizTalk with the release of WF?
10. Will Microsoft be pushing WF?
11. Don’t think so, as at present both the things are meant for different levels

BizTalk a Server product is to integrate diverse applications (that’s why there are so many of those adapters coming every release). Also the scalability which BizTalk provides can’t be compared with any thing else.

As far as WF is considered, it can be used in small workflows within ASP.NET, Windows applications, and as a sub process to BizTalk workflows.

Some good links on when to consider WF and when BTS

1. http://geekswithblogs.net/bloesgen/archive/2005/10/09/56481.aspx
2. http://blogs.msdn.com/scottwoo/archive/2005/10/10/479331.aspx
3. http://blogs.msdn.com/irenak/archive/2006/11/29/sysk-249-choosing-the-right-microsoft-integration-technology.aspx
4. http://pluralsight.com/blogs/matt/archive/2005/09/14/14818.aspx
5. http://blogs.msdn.com/lamonth/archive/2005/09/14/466222.aspx