Character encoding in Send Pipeline?
Hi Everyone!
Hope you guys can help me out here...
Situation: My client wants to have a flat file in Western European-1252 (Windows-1252) encoding. I tested to change the design-time property TargetCharset in the pipeline to just that on my developer machine (2006 R1 on a VPC). Worked sooo nice. A walk in the park so to speak...
BUT, I deploy this to the customer test-machine (2006 R2) it doesn't work! I can't get it to produce anything else but UTF-8!
I have tried and failed with the following:
-
Preserve BOM both ways (true and false)
-
Setting the charset with XMLNorm.TargetCharset (That gave me ? instead of swedish characters, åäö)
-
Setting the schemas CodePage to 1252
-
Setting the TargetCharset property in the pipeline from the admin console
-
and some combos of the above...
Now I can't dwell on this anymore! Customer needs this up and runnig by monday afternoon...
If someone could help me to point out some tips I would be glad. Also I'd really like to know if anyone knows what to type in to the textbox on the TargetCharset property on the Send Pipeline properties in the admin console. I have search and search for a list of VALID strings to put in there! But have found nothing, frustrating...
-
All Replies
I know there is a bug in the XMLTransmit pipeline and that you have to set it in a custom pipeline. You might be running up against something like that.
Richard Hallgren
Hi Richard,
Yes, I found that report too. But the solution there was to use XMLNorm.TargetCharset to set the charset in the message. That did work partly. The encoding on the file is right, but swedish characters is represented with questionmarks.
Anyone with solutions on that? ;-)
The strange thing here is that my solution works perfectly on R1 machines (tried two different). Could this problem be due to that my solution is compiled in a R1-environment?
Ok, now I've tried three different R1-installations where my solution works flawless. I'm fairly confident in that my problem is due to the R2-environment. Now I have to try to find a R2 developer-environment and try to compile and deploy there to see if the problem is the compilation.
Still looking for a list of supported values for the property TargetCharset in the Send PipeLine configuration in the BizTalk Admin Console.... Anyone??
- Joakim,
If you're just using the per-instance pipeline configuration on the BTS Admin Console, then I can say that, while i haven't tested it with the Flat file disassembler, it does *not* work with the XML assembler, because you can't set the TargetCharset property to a value that will be recognized.
Does it work if you explicitly use a custom pipeline? Hi Tomas,
I'm using a custom pipeline which does nothing else then assembles a flat file. In the assemble stage component I can set the TargetCharset property and it works as long as I deploy to an R1 server. It does not work on the R2 server I was trying to deploy to.
But the workaround presented in the link Richard gave her above doesn't work either. I can't set the encoding to "windows-1252" directly on the message in my orchestration and keep the swedish characters. This really bothers me!
- Strange ... I'll throw some ideas out there that aren't very well thought through.
Is it the same file you drop in to BizTalk as in the R1 environment, or could it have something to do with the encoding of the incoming file in the R2 evironment (it's in in other environment, right?)? You actually say that you get the encoding of the file right in the end but the Swedish sign are messed up (so your main problem is solved). I guess something differs between the two environments in the way you test it - could that be it? As I said, just an idea.
Richard Hallgren
http://www.richardhallgren.com Hi Rihard,
Thanks for the ideas!
Well, I've tried several different approaches to sending in the first message. That does not seem to affect the result. I think that's due to that I recieve the message through a Web Service that exposes my orchestration.
And you're right when you say that my main problem i solved, but that is a small help when that solution exposes a new problem: swedish characters missing or replaced with '?'.
The testing procedure is the same in both environments, I send in a test message through the WS with a tool called WebServiceStudio 2.0. But if I analyse what differs between the environments I get this short list:
-
BizTalk version, R1 in development, R2 in test/production
-
BizTalk and SQL on same machine in development (separated in test/production)
-
My developer machine runs directly under "administrator" account
Not much to go on here as I see it?
I'm going to hunt down a R2-iso and set it up on a VPC and compile/deploy my project on that machine to see if that solves the problem.
My customer couldn't wait for more debugging of this problem and rolled their environment back to R1 today and as expected everything worked without any hiccup! But I really have to find this out for my own peace...
-
- Didn't mean to take away your problem (I blame it on poor English). What I meant is that I don't think it has something to with the encoding on the send port, or with it being on a R2 installation. I rather think it's something earlier in the process and that it has to do with the configuration of that environment to do (I could very well be wrong here!). Can you somehow trace the message as it travels through the process? Can you somewhere see the message having correct characters (maybe you could stop an orchestration or send port and view the message in the suspended messages queue?)?
Richard Hallgren
http://www.richardhallgren.com Ah, no offense taken! I was just afraid that you somehow missed that it gave me another problem...
I did do a quick check and the message travels through as UTF-8 all the way as far as I can tell. I stopped the orchestration at the send-shape and at other strategic shapes where I could intercept the messages created and transformed from the original message.
Right now I'm too tired to try and check every step when I set the outgoing message to windows-1252 in the orchestration. I'll try that later tomorrow...
- Joakim,
Ahh I see. Well, I can say that I've just tried specifying a targetcharset on the flat file assembler and get the file in the right encoding on R2, so that actually works.
From what you describe, though, I wonder if your problem isn't ocurring *before* the send pipeline, though. The problem with windows-1252 encoding is that it's not exactly trivial to distinguish it from other 8-bit encodings until you get to the extended characters. Since you aren't seeing those, it is possible that the file is indeed in the right encoding, but the extended characters were already screwed up at that point in time.
Have you verified the original message you were processing was interpreted correctly by biztalk? Hi,
I have exactly the same problem and is also using BizTalk 2006 R2
Whenever I try to change the encoding from utf-8 I get question marks instead of special characters like øæå.
Joakim, did you find a solution?
I actually had a similar problem with the target charset for the "BizTalk Framework assembler" but here Microsoft had a hotfix that solved the problem.
It is a dynamic send port, but I have tried setting the target charset in (BTS.SendPipelineConfig), on the physical port, on (XMLNORM.TargetCharset) and all combination, but the only way to get the correct result is to send it as utf-8, and I need it as windows-1252. If I set the charset to windows-1252 the file do come out as windows-1252, but the problem is the special characters have been truncated.
Any ideas would be appreciated.
Regards Paw
Hi Paw!
Sadly I haven't really solved the problem. Now it is a while since I was involved in the coding. But as I recall we found out that the biggest problem was the R2 installation. We switched back to "normal" BizTalk 2006 and it worked...
I have yet to try out a complete R2 development environment for this problem, have you tried that?
I'm really sorry that I couldn't help you out more... But, please, document your experience in this thread for later reference!
Best regards,
Joakim Schütt
Hi Joakim,
Thank you for the quick reply.
I have worked with this problem for almost a day now, and decided to go for a work-a-round. I'm pretty sure it's a bug in BizTalk 2006 R2, as it was with the target charset in the "BizTalk framework assembler", however there doesn't seem to be a hotfix for it yet. Hopefully there will come.
My work-a-round was to add a custom "Change Encoding" pipeline component after the "Flat file assembler" in the pipeline, that change the encoding from utf-8 to windows-1252. This will however add some extra load, so I hope for a fix sometime.
Best regards
Paw
