File encoding problem

Question

User-1949027199 posted

Hi,

I have problem that VS is change the encoding type when i save the file. On my website i see text like,  frÃ¥n, rörande ...

Why is VS changing the encoding type from utf signed to utf unsigned every time? Same problem in VS 17 and VS 19.

Answer 1

User475983607 posted

File encoding is determined by your system settings.  See the following for working with file encoding in Visual Studio.

https://docs.microsoft.com/en-us/visualstudio/ide/how-to-save-and-open-files-with-encoding?view=vs-2019

If the above does not solve the problem then share an example that the forum community can reproduce. 

Answer 2

User-1949027199 posted

Yes, thanks. I use "save as" and select my file-encoding. But i dont want to remember to do this everytime i save . Where can i change this? When recive a new file from git and save my changes, i dont want to change the encoing type to utf-unsigned, i want it to be the same i recived the file in.

Answer 3

User753101303 posted

Hi,

What if using "Save as" (and the drop down button to use "Save with encoding") ? You should also have a "File","Advanced Save Options" (maybe available in "Tools", "Options" depending on your VS version) that should allow to set the default format once for all.

If it seems ok, it could be also maybe some misconfiguration in the web.config file (for example altering globalization responseEncoding).

Edit: finally I believe I saw once someone that forced a wrong encoding at the browser level.

Answer 4

User475983607 posted

Yes, thanks. I use "save as" and select my file-encoding. But i dont want to remember to do this everytime i save . Where can i change this? When recive a new file from git and save my changes, i dont want to change the encoing type to utf-unsigned, i want it to be the same i recived the file in.

What is the file encoding?

Are you sure Visual Studio is changing the file encoding?  Usually, this type of error is due to a difference in the machine (or application) default settings that saved the file and the machine opening the file.  

The Visual Studio default encoding can be changed from the Tools -> Options menu.  Then Environment -> Documents and check Save documents as Unicode when data cannot be saved in codepage.  That only works if you need you Unicode and the code page is not identified.

Again, is there anyway you can provide steps to reproduce this issue along with the steps to reproduce and the file encoding(s).

Answer 5

User-1949027199 posted

What i do is, i create a new webbapplication in VS 17, select MVC. In the default index file i recive i write, "För" and save. Then i run the website and i se, "för"

Answer 6

User475983607 posted

I tested this an was unable to reproduce the issue.   I added För to a View and ran the project is IIS Express and the browser displayed För.  Does this issue only happen after committing the file to GIT?  Are you deploying the file to a remote web server or local IIS?  Are you opening the file in a different OS than the OS that created the file?

Anyway, my default file encoding is set to Unicode (UTF-8 with signature) - Codepage 65001. 

My default Windows text encoding is set to.

PS C:\WINDOWS\system32> [System.Text.Encoding]::Default


IsSingleByte      : True
BodyName          : iso-8859-1
EncodingName      : Western European (Windows)
HeaderName        : Windows-1252
WebName           : Windows-1252
WindowsCodePage   : 1252
IsBrowserDisplay  : True
IsBrowserSave     : True
IsMailNewsDisplay : True
IsMailNewsSave    : True
EncoderFallback   : System.Text.InternalEncoderBestFitFallback
DecoderFallback   : System.Text.InternalDecoderBestFitFallback
IsReadOnly        : True
CodePage          : 1252

Answer 7

User-1949027199 posted

Does this issue only happen after committing the file to GIT? No, in this test, i created a new webapplication.
Are you deploying the file to a remote web server or local IIS?  I run this new webapplication in visual studio 17.
Are you opening the file in a different OS than the OS that created the file?  No

I recive this settings:

IsSingleByte : True
BodyName : iso-8859-1
EncodingName : Western European (Windows)
HeaderName : Windows-1252
WebName : Windows-1252
WindowsCodePage : 1252
IsBrowserDisplay : True
IsBrowserSave : True
IsMailNewsDisplay : True
IsMailNewsSave : True
EncoderFallback : System.Text.InternalEncoderBestFitFallback
DecoderFallback : System.Text.InternalDecoderBestFitFallback
IsReadOnly : True
CodePage : 1252

I try to check, "Save documents as Unicode when data cannot be saved in codepage"  and resave the file, but i still se, "för"  not "för"  :(

Answer 8

User-1174608757 posted

Hi mrcoder, 

According to your description, have you tried to set the encoding type directly? I suggest that you could add configuration in web.config file as below and see whether it could show well:

<configuration>
  <system.web>
    <globalization requestEncoding="utf-8" responseEncoding="utf-8" />
  </system.web>
</configuration>

Best Regards

Wei

Answer 9

User-1949027199 posted

Thanx, but i dont think this is the solution. When i go to, Save as..., then Save with encoding, Unicode (UTF-8 without signature) - Codepage 65001   is selected.
If i change this to,  Unicode (UTF-8 with signature) - Codepage 65001    i can se, För. But i must remember to change this everytime i save the file. Why?

Answer 10

User753101303 posted

 "with signature" is my default here. Do you have the same problem on the same file once you have done this ? I wonder if  VS just doesn't default to the encoding that is used for the current file you are trying to save.

Knowiing which VS version you are using could help in case someone would know if it could be globally overriden (but it seems to make sense to not change the encoding automatically at least for existing files).

You could try also to create a new cshtml file to see then which encoding is used by default on your side. I have "Unicode (UTF-8 with signature) - Codepage 65001" as well here.

Edit: I gave this a try and it seems to work this way. I saved the file using Notepad to change the encoding and if I change it back with VS it seems to be kept from now (but likely just for this file).

For now I believe the real issue is why those files were created or saved at some point without the utf8 signature.

Answer 11

User-1949027199 posted

If i, Save as, select Unicode (UTF-8 with signature) - Codepage 65001 and save. Then För is correct. If i close the file in the project and open it again, and go to, Save as, the Unicode (UTF-8 without signature) - Codepage 65001 is selected.  I think, Unicode (UTF-8 with signature) - Codepage 65001, is selected as long i have the file open in VS. If i close the file and open it again, then Unicode (UTF-8 without signature) - Codepage 65001 is selected as default.

I have the same problem in both VS 17 (15.9.11) and VS 19 (16.0.2).

If i create a new cshtml, i have Unicode (UTF-8 without signature) - Codepage 65001 selected.

Answer 12

User753101303 posted

Same code page here but VS 17 uses "uft-8 with signature" and it is always left unchanged as far as I know...

I gave a quick look but don't see for now where it could be changed. I would try:
- to reset all options (you should be able to save/restore them if needed) to see if it makes a difference.
- or could it be maybe caused by a 3rd party extension you installed on both ???

Answer 13

User475983607 posted

Have you installed any Visual Studio plugins that changes VS default behavior?

Answer 14

User-1949027199 posted

I reset the settings, created a new file, and the new file has Unicode (UTF-8 with signature) - Codepage 65001  selected. But when i open the old file, Unicode (UTF-8 without signature) - Codepage 65001 is selected, i change to Unicode (UTF-8 with signature) - Codepage 65001 and save, close the file, open the file and, Save as, Unicode (UTF-8 without signature) - Codepage 65001 is selected :(

I have Codemaid and Resharper. I dont know if they can have made someting?

Answer 15

User753101303 posted

Try https://docs.microsoft.com/en-us/visualstudio/ide/reference/safemode-devenv-exe?view=vs-2019 to launch VS without them and see what happens then. Good luck ;-)

Answer 16

User-1949027199 posted

Thanx, but there was the same problem :(   Unicode (UTF-8 without signature) - Codepage 65001  is selected :/  I try to save in Unicode (UTF-8 with signature) - Codepage 65001, but nex time, Unicode (UTF-8 without signature) - Codepage 65001  is selected  :(

Answer 17

User-893317190 posted

Hi mrcoder ,

Maybe you could try to install vs extension to force utf8 with bom.

In search bar , input utf-8 , you could try ForceUTF8 (with BOM).

Also refer to https://stackoverflow.com/questions/696627/how-to-set-standard-encoding-in-visual-studio

and https://forums.asp.net/t/1134534.aspx

Best regards,

Ackerly Xu

Answer 18

User-1949027199 posted

Hi,

ForceUTF8 (with BOM) didnt work :(

I try another way, i open the file in Notepad, save as, Encoding: ANSI (was selected by default) and Save. Open this file in Visual Studio, Save with encoding, Western European (Windows) - Codepage 1252   is selected by default. And i can se, "För" correct. So the problem must be in Visual Studio someware?

Answer 19

User753101303 posted

You really tried to launch VS from the command line so that no addins at all are loaded ? As far as I can see VS always handled this fine here.

Please try. For now it seems there is an addons that is changing that or that some VS settings were altered at some point. If you can try a fresh VS install on anotrher machine and you should see it works fine.

Answer 20

User1967090451 posted

Hi,

In our project we solved same problem. Issue was in .editorconfig file (displayed only in TFS Source control) which has native support in VS2019. If you have this in solution than settings from tools->options->documents->Save documents as Unicode when data cannot be saved in codepage is ignored. In section:

[*]
indent_style = space
indent_size = 2
trim_trailing_whitespace = true
charset = utf-8

we've changed charset = utf-8bom and encoding with signature (after save file) is working now.

Hopefully it will help.

JS