BizTalk EDI Processing: Some Characters are getting replaced by "??"

Question

Hi All,

I'm Processing EDI File through a EDI Custom receive pipeline. (in BizTalk 2009)

This EDI file will have information on the customer name and it might contain special characters like   "í ó ú à è ì ò ù â ê î ô".

The character set in the receive pipeline is set to UTF-8. But when BizTalk processes it, it will replace those characters with "??"

e.g. if the name is àBC when BizTalk processes it will be "??BC"

I thought UTF-8 would fix this issue. but it seems not.

Any help is appreciated.

Thanks,
Guy

Answer 1

EDI file can be almost everything
X12, Edifact, XML, CSV, Json, or??

What character encoding does your sending partner use
If X12, Edifact or XML you can set the encoding in the file

/Peter

Answer 2

what is the code page set at the schema level? If it is not set as UTF8 then can you try setting there and test.

Answer 3

Also, make sure you're using a good editor to view the files.  I use Notepad++ which will happily display all characters.

àBC -> ??BC really suggests it's just a display problem as the double byte à are not ascii characters, but they are there.

Answer 4

When processing an incoming EDIFACT message, BizTalk Server determines the character set to use for that message from the UNB1 data element. See the following link for more details about the EDI character set: https://docs.microsoft.com/en-us/biztalk/core/edi-character-sets

Answer 5

Thanks for the feedback, I checked with Notepad++ but it's not a display problem.

Answer 6

Which type of interchange do you have?
Without this information it is not possible to help you further

/Peter

---

When asking a question please be as thoroughly as possible this will make it easier to assist you http://www.catb.org/esr/faqs/smart-questions.html