Handling comma separated values inside a map, part I

by eliasen 22. juni 2009 20:55

Hi all

Some times people need to convert a list of elements into a comma separated list of values. In the forums this questions arises every now and then, so here goes a solution.

Suppose this input:

<ns0:MultipleElementsRoot xmlns:ns0="http://MapCommaSeparatedValue.MultipleElementsSchema">

And I want this output:

<ns0:CommaSeparatedRoot xmlns:ns0="http://MapCommaSeparatedValue.CommaSeparatedSchema">

Inside a map I would handle it like this (2 approached, that share the first two functoids):


The reoccurring element is used as input to a string concatenate functoid. The functoid takes a comma as the second input. The output of the concatenate functoid is then used as input for a cumulative concatenate functoid, effectively creating a comma separated list of values. The issue with this list is, that it will have a comma at the end of the line, which needs to be removed.

One way of doing this is to use three functoids to pull this last comma away. First, I take the length of the string, then I subtract one from that, and use the string extract functoid to extract the string beginning at position 1 and ending at (length - 1) and send that to the destination element. Works like a charm.

Now, plenty of people out there (Henrik, you know who you are) would have no moral qualms using a C# scripting functoid to get rid of the extra comma. Like this:

public string RemoveLastComma(string input)
  return input.Substring(0, input.Length-1);

That one scripting functoid would eliminate the last three functoids I added to remove the last comma. But I still prefer the functoid way of doing it, mostly for maintenance reasons. A new BizTalk developer looking at a map with lots of scripting functoids has no idea what is happening, and even after looking at the code in all the scripting functoids, he can’t remember what is happening in more than a couple of them. In my opinion; Always use the built-in functoids if possible.

Now, just for the kick, I thought that some times the opposite could be the case – going from a comma separated list to lots of elements. This, I cannot solve using built-in functoids, unfortunately, so the map ended up like this:


Just one scripting functoid. The type is an “Inline XSLT Call Template” and the script is:

<xsl:template name="MyXsltSplitTemplate">
  <xsl:param name="param1" />
  <xsl:if test="$param1 != ''">
    <xsl:element name="MyField">
        <xsl:when test="contains($param1, ',')">
          <xsl:value-of select="substring-before($param1, ',')" />
          <xsl:value-of select="$param1" />
    <xsl:call-template name="MyXsltSplitTemplate">
      <xsl:with-param name="param1" select="substring-after($param1, ',')" />

Basically, the function substring-before is used to get any string before a comma and output it. If no comma is in the string, the string is output. The string to the right of a comma is then used as a recursive call to the same template. Again; Works like a charm :-)

So, with any luck, this question can be answered by googling soon instead of asking in the forums :-)

Edit on 27’th June:



Assigning the value of an xs:integer to an integer faling

by eliasen 13. juni 2009 19:43

Hi all

I have received numeral questions from people getting a compile time error when compiling an orchestration that assigns a distinguished field of type xs:integer to a variable of type int32. Without looking into anything, you would expect this would work.

So lets say that we have a schema like this:


with this XSD:

<?xml version="1.0" encoding="utf-16"?>
<xs:schema xmlns:b="http://schemas.microsoft.com/BizTalk/2003" xmlns="http://IntegerDecimal.InputSchema" targetNamespace="http://IntegerDecimal.InputSchema" xmlns:xs="http://www.w3.org/2001/XMLSchema">
  <xs:element name="InputRoot">
          <b:property distinguished="true" xpath="/*[local-name()='InputRoot' and namespace-uri()='http://IntegerDecimal.InputSchema']/*[local-name()='MyInteger' and namespace-uri()='']" />
        <xs:element name="MyInteger" type="xs:integer" />

As you can see, it is a simple schema and the “MyInteger” field is of type xs:integer and it is marked as a distinguished field.

In my orchestration, I have a variable of type int32 called “MyInt” and in an expression shape I do this:

MyInt = InputMessage.MyInteger;

and it fails, which is quite surprising at first glance. At compile time I just get this: "The expression that you have entered is not valid.". That doesn’t help, so looking at my expression shape and reading the mouse over on the error in the expression shape I get this: “cannot implicitly convert type ‘System.Decimal’ to ‘System.Int32’”. Changing the line in the expression shape to this:

MyInt = System.Convert.ToInt32(InputMessage.MyInteger);

will make it work, because now you are explicitly casting the value to an integer.

The first times I saw this error, I thought that the compiler had an error, because I thought I was assigning an integer the value of another integer. It turns out, though, that I wasn’t really – only sort of…

If you look here: http://www.w3.org/TR/xmlschema-2/#integer you can see that the xs:integer type is actually a decimal type restricted to whole numbers. So the compiler actually thinks the field is a decimal and not an integer.

This is kind of silly, since the data type IS restricted to whole numbers. The actual error should be that the xs:integer can contain the number 4567234987623409763546387623476149823497862354845 which really is way to large for an int variable in an orchestration.

Anyway, the way to handle this is, naturally, to avoid the xs:integer type if possible and use the xs:int (or variants of this) instead. Sometimes it is needed, though, and in that case you just need to declare your variables of a type that will be able to contain all the possible values you receive. This can be an Int16 sometimes, if the people who have created the schema just didn’t know what the xs:integer type is. Sometimes the values received will be too large even for an Int64 variable, then consider using a decimal, a string or some other data type.



Promoting values from destination schema of map on receive port

by eliasen 10. juni 2009 23:30

Hi all

So, the other day, a guy asked a question on the online forums, and another guy tried helping out by stating, among other things, that the maps on receive ports are executed before the receive pipeline. This isn’t true, and I posted a post, where I tried to explain how things work. This ended up being slightly wrong, so I posted a correction, but now it seems I need to post another correction… and I ended up writing this post to explain how stuff works.

First of all, let me set one thing straight; When a message arrives on a receive location it is first sent through the receive pipeline that is specified on the receive location. This is needed before the map for several reasons, including converting the input to XML and promoting the MessageType, so the receive port can choose the correct map to execute. The receive pipeline also promotes all the distinguished fields and promoted properties that are specified on the schema.

Now, after the map has finished executing, the transformation engine will look up the schema for the output and it will instantiate the XMLDisassembler with this particular schema, so the disassembler doesn’t have to find the correct schema itself. After instantiating it, it will call it, so the XMLDisassembler will read the output from the map and promote all distinguished fields and promoted properties to the context of the message. Also, before doing the promotion, it will copy all the context from the original message, so you get all the properties from the adapter and so on copied to the destination message.

Now, there are a couple of issues to this, which most people don’t realize – mostly because they will only affect you on very very rare occasions:

Distinguished fields
I found that if you have an input message with a field marked as a distinguished field and then look at the context of the output from the map, then the output message also has the distinguished field from the input message in its context which really doesn't make sense, since you can't use it in any way. This has NO influence at runtime and NO influence at design time, so you will go through your life not realizing it. Also, usually we don’t set distinguished fields on the external schemas, because we don’t want to change them and because we don’t want to use schemas exposed to trading partners in our business processes, which is the only place we can use the distinguished fields.

Promoted Properties
If you have promoted a field from both input and destination schema to the same property, the value from input schema in the properties is overwritten by the value from the destination schema after the map. For the same reasons as for the distinguished fields, we rarely have promoted properties on the external schemas, and therefore you probably will never have an issue with this.

IF the schema for the destination message is marked as an envelope, the message will fail. This is because the disassembler will recognize the schema is an envelope and it will then debatch the message into several messages. Only the first is returned though, since the mapping engine which is calling the disassembler assumes this is not an envelope and therefore will only call the GetNext method once, whereas normally the GetNext emthod is called until it returns null. This first message is then looked at, but in the properties for the disassembler, the transformation engine has all ready set the only allowed schema, which was the envelope schema. So the disassembler only has the envelope schema as a possible schema, and the instance that comes out after debatching is not the output from the map anymore, meaning that it will fail with the standard error message “Details:"Document type "http://MyNameSpace.com#Record" does not match any of the given schemas."” As with the first to issues about distinguished fields and promoted properties, this should practically never happen, since you are most likely mapping the incoming message to some internal schema, for which there is usually no reason to mark as an envelope.

So, basically, when a message arrives, the receive pipeline is executed, then the map is executed and at the end, the XML disassembler is executed by the transformation engine to get all promoted fields in the destination message promoted. There are a couple of known issues with this, but they are either totally unimportant or very unlikely to occur.

I hope this helps someone out there.



Who moved my cheese?

by eliasen 6. juni 2009 16:49

Hi all

Just a quick book review, since I have just read the “Who moved my Cheese” book.

It only takes like an hour to read, and it is the story of two mice and two people who are searching for cheese, where the cheese is a metaphor for something you really want to have. At some point, they run out of cheese, and the four characters take very different approaches to the changes in the environment. The point of the story is, naturally, that you can most likely identify yourself as one of the characters and after reading the book, perhaps you have learnt something about how you react to changes and how you should react – and hopefully improve your life.

Easily read with lots of points. Read it…



Updated documentation for my BizTalk artifacts

by eliasen 24. maj 2009 18:48

Hi all

Just to let you all know, I have updated the documentation for my pipeline component library at http://eebiztalkpipelinecom.codeplex.com/ and the documentation for my functoid library at http://eebiztalkfunctoids.codeplex.com/

Happy reading :-)



BizTalk 24x7

by eliasen 23. maj 2009 19:36

Hi all

Saravana Kumar has launched version two of his great site http://www.biztalk247.com – it has loads of nice information about BizTalk, so go check it out.

Also, while you are checking it out, take a look at his other new site: http://blogdoc.biztalk247.com/ which is collecting information from lots of blog entries about BizTalk.



Two nodes on same level with same name but different namespace

by eliasen 6. maj 2009 22:36

Hi all

Today I encountered something I have never seen before, when creating a map.

The issue occurs because my customer had a schema that imports two other schemas, both of which have an element called “metadata” – but naturally the two schemas have different target namespaces.

The main schema imports both, and has two records just below the root, and these two records reference each of the two metadata elements in the two imported schemas.

So the first schema could look like this:


And the second schema could look like this:


So both have an element named “metadata” but one is in the namespace “http://TwoElementsDifferentNamespace.MetadataOne” and the other is in the namespace “http://TwoElementsDifferentNamespace.MetadataTwo”.

After that, I create the schema that impors both:


As you can see it imports the first two schemas, and has to elements that reference each of the metadata elements form the two first schemas.

Also, I crated an output schema that just has three elements and then I created this map:


Pretty simple. Now, the issue comes when compiling, because I get this error:

Exception Caught: The map contains a reference to a schema node that is not valid.  Perhaps the schema has changed.  Try reloading the map in the BizTalk Mapper.  The XSD XPath of the node is: /*[local-name()='<Schema>']/*[local-name()='Root']/*[local-name()='metadata']/*[local-name()='Field2']

o the first try was to reload the schema – but that didn’t help – it just broke one of my links.

In he end I found out, that the issue is that the links are stored like this in the .BTM file:

<Link LinkID="1" LinkFrom="/*[local-name()='&lt;Schema&gt;']/*[local-name()='Root']/*[local-name()='metadata']/*[local-name()='Field1']" LinkTo="/*[local-name()='&lt;Schema&gt;']/*[local-name()='OutputRoot']/*[local-name()='Field1']" Label="" />

<Link LinkID="2" LinkFrom="/*[local-name()='&lt;Schema&gt;']/*[local-name()='Root']/*[local-name()='SomeFields']/*[local-name()='Field3']" LinkTo="/*[local-name()='&lt;Schema&gt;']/*[local-name()='OutputRoot']/*[local-name()='Field2']" Label="" />

<Link LinkID="3" LinkFrom="/*[local-name()='&lt;Schema&gt;']/*[local-name()='Root']/*[local-name()='metadata']/*[local-name()='Field2']" LinkTo="/*[local-name()='&lt;Schema&gt;']/*[local-name()='OutputRoot']/*[local-name()='Field3']" Label="" />

So. basically, the .BTM file saves links as XPath expressions WITHOUT the namespaces. So naturally, this has to go wrong, when there are two “metadata” elements on the same level in the schema.

The way to solve this is to choose the properties of the map and disable the “Ignore Namespaces for Links” like this:


After setting this property, the links change having namespaces inside the .BTM file and everything is just fine.

One might wonder why the namespaces are not enabled as the default, since they do make the solution more robust. Well, the reason is simple; If the namespaces are in all the links, and you change for instance the namespace of the root node, then ALL links in the map gets broken. So actually, not having the namespaces in the links make the solution more robust.

So… I hope this can help someone.

You can download the solution here:



Now also a pipeline component library

by eliasen 25. april 2009 21:08

Hi all

Just as i have started developing a functoid library (found at http://eebiztalkfunctoids.codeplex.com/) I have also started developing a pipeline component library. Right now it contains three components:

  • DevNull. This pipeline component is quite simple. It will "swallow" everything that comes as input. This enables performance testing of stuff without concerns about adapter transport time at send port for instance.
  • SearchAndReplace. This component will perform a search and replace on the incoming stream, replacing some string with some other string. Optionally, you can decide to let the input string be a regular expression and replace based on that instead of normal string search and replace.
  • Promote. This component has three parameters, the name of a property, the namespace of the property and an XPath expression. The component will read in the value that corresponds to the XPath expression at runtime and promote it to the property given by the name and name space. This enables you to promote reoccurring elements.

You can find it at http://eebiztalkpipelinecom.codeplex.com/ – the url is weird, I know. But there is a limit to the length of the urls at codeplex, unfortunately.



Promoting reoccurring elements – Part III

by eliasen 25. april 2009 21:02

Hi all

So, first I had a post about promoting reoccurring elements where I discussed 4 options for promoting an element that can occur multiple times in the input. Then I had a post about how you can pseudo do it in BizTalk 2000/2002.

This post is about two things:

  1. Another way of doing it that doesn’t work
  2. Writing the custom pipeline component that was my second suggestion in my first post.

First issue

As discussed in my first post, if you change the XPath of the promoted property in the schema (either by clicking the ellipsis as I wrote about or manually editing the XSD) to add the “[1]”, you get this compilation error:

Node "ElementWhereNumber1IsPromoted" - The promoted property field or one of its parents has Max Occurs greater than 1. Only nodes that are guaranteed to be unique can be promoted as property fields.

But, since I couldn’t get the “[1]” to work in BizTalk 2000/2002 when editing my second post I used the position function instead. So i started wondering if that might work in BizTalk 2006 R2. It doesn’t, though. It compiles just fine, and I really had my hopes up, but at runtime you get this error:

There was a failure executing the receive pipeline: "Microsoft.BizTalk.DefaultPipelines.XMLReceive, Microsoft.BizTalk.DefaultPipelines, Version=, Culture=neutral, PublicKeyToken=31bf3856ad364e35" Source: "XML disassembler" Receive Port: "ReceivePort3" URI: "C:\Projects\Blog Entries\PrommotingReoccurringElementsEditXSD\Instances\In\*.xml" Reason: Unexpected XPath format:

So i had to drop that idea again.

Second issue

I decided to write the custom pipeline component that I discussed in my first post.

The main code is easy – just three properties:

  1. Name of the property
  2. Namespace of the property
  3. XPath

You can download it at http://eebiztalkpipelinecom.codeplex.com/ where I have just made my initial release.



The Context Accessor functoid, part III

by eliasen 15. april 2009 22:17

Hi all

I have had posts about the context accessor functoid here and here.

Just a couple of notes about the context accessor functoids (plural - because there are two functoids at codeplex):

  • One of the functoids will only work when called from a map that is executed inside an orchestration.
  • The other functoid will only work when called from a map in a receive port AND only if the pipeline component that ships with the functoid has been used in the receive pipeline.

As you can see, creating a map based on either of these functoids makes your map impossible to use in either an orchestration or a receive port based on which functoid you chose. So you are creating a pretty hard coupling between your map and where it should be used. This can be ok, but if other developers mess around with your solution in a year or so, they wont know that and things can start breaking up.

My self: I am a user of the functoids - I would use them instead of assigning values inside an orchestration using a message assignment shape.. but this discussion is pretty much academic and about religion :-)

Anyway, beware the limitations!



About the author

Jan Eliasen is 37 years old, divorced and has 2 sons, Andreas (July 2004) and Emil (July 2006).

Jan has a masters degree in computer science and is currently employed at Logica Denmark as an IT architect.

Jan is a 6 times Microsoft MVP in BizTalk Server (not currently an MVP) and proud co-author of the BizTalk 2010 Unleashed book.

BizTalk Server 2010 Unleashed

Buy from Amazon

Microsoft MVP

6 times: July 2004, July 2008, July 2009, July 2010, July 2011, and July 2012. Not currently an MVP.


Image to show

Month List

Page List