"Unmappable character for encoding ASCII" when building a UTF-8 project

issue-resolved

(Will Herrmann) #1

I have a project in Eclipse that uses UTF-8 character encoding (set via the Eclipse preferences). However, when I try to run the build task using Buildship, I get the following error:

MyFileDialog:66: error: unmappable character for encoding ASCII
	if (choice.equals("Save???")){
                               ^

The relevant line looks like the following:

if (choice.equals("Save…")){

Note that it uses the horizontal ellipsis unicode character. I have no trouble compiling in Eclipse (which fully recognizes that it is a UTF-8 project). So why would Buildship complain about it being unmappable to ASCII?


(Ian Stewart-Binks) #2

Hey Will,

Thanks for raising the issue. I’m keen on getting to the bottom of the character encoding problems :smile:

I’m unable to rerproduce this problem. I imagine that it’s the compileJava or compileGroovy subtasks of build that are failing (please correct me if I’m wrong).

I’ve gone about trying to reproduce this issue using these steps:

  1. Launch Eclipse
  2. Create new Gradle project
  3. Add an arbitrary line with the string "Save…" in it (either in groovy file or java file)
  4. Using the Gradle tasks view, I double click on ‘build’

The build successfully passes for me, however.

May I ask which version of Eclipse you are using?
The bug you reported in BugZilla states that you were using a mac - is this still the case with this issue? I’m not certain if that would have much of an affect, but it’s worth looking into. All of my tests have been with Linux/Windows.


(Etienne Studer) #3

I assume this is the compiler error that you have listed?
Can you list the complete error output?
Does the same error happen when running the Gradle build from the command line?
Are you sure the file is properly stored in UTF-8 format?

The error sounds to me like the compiler is expecting a file in ASCII format and then comes across a non-ASCII character while reading the file.


(Will Herrmann) #4

Thanks for the responses! I created a completely new Gradle project and to my surprise it did not complain about my UTF-8 characters. In both cases, Eclipse claims that the project is encoded in UTF-8 (which I see by going to Project -> Properties, and then the Resource section). I was hoping to upload a screenshot of this, but it says that new users can’t upload images.

I do not see any meaningful differences between the settings.gradle, build.gradle, or .project file contained in a fresh project and my old project. If there is something I should be looking for, please let me know.

I am running Mac OS 10.9 and Eclipse Luna (4.4.2). I am not familiar enough with Gradle to run it from the command line, so I don’t know if that will work.

Here is the full error output (with path and file names changed):

Gradle Tasks: build
Working Directory: /Users/Will/Eclipse Workspace/MyProject
Gradle Distribution: Gradle wrapper from target build
Gradle User Home: Use default defined by Gradle
Java Home: Use default defined by Gradle
JVM Arguments: Use default defined by Gradle
Program Arguments: None

:compileJava/Users/Will/Eclipse Workspace/MyProject/src/main/java/MyFileDialog.java:39: error: unmappable character for encoding ASCII
		if (choice.equals("Save???")){
                               ^

[Several more of these errors appear]

6 errors
 FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':compileJava'.
> Compilation failed; see the compiler error output for details.

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output.

BUILD FAILED

Total time: 2.121 secs

(Etienne Studer) #5

I would take a close look at the file called MyFileDialog. What is its encoding? The compiler seems to expect ASCII format but finds UTF-8 characters.

What happens if you run the build from the cmd line? No need to debug in Buildship until it runs successfully from the cmd line.

What happens if you replace the UTF-8 specific characters with their Unicode value (\uxxxx)? What happens if you configure the compiler to use UTF-8 encoding?

SO might give some inspiration, too:


(Will Herrmann) #6

How would an individual file have a different encoding than the project? There isn’t anywhere to declare the file encoding in the Java file.

I’m not sure how to run Gradle form the command line. I tried changing to the directory in the Terminal, but whenever I try gradle [anything], I am told that the gradle command is not found.

If I replace the UTF-8 specific characters with their unicode value, it works. However, I would like to be able to use the actual characters, as it is far easier to read (and this has worked in Eclipse before using Buildship and Gradle).

I don’t understand the question about what happens if I configure the compiler to use UTF-8 encoding. Isn’t that what I have been doing setting the Eclipse project properties?


(Etienne Studer) #7

It seems that by default, your Java compiler that is executed from the Gradle Compile task is expecting the source files to be in ASCII format. Now, since one of your source files contains UTF-8 characters, the compiler complains. Thus, you have to configure the Gradle Compile task such that it expects the source files in UTF-8 format.

The UTF-8 settings in the Eclipse preferences are not related/synchronized to the compiler settings of the Gradle Compile task when running a Gradle build.

If your project is using the Gradle wrapper, running Gradle from the command line is done via ./gradlew <task names>.


(Will Herrmann) #8

Thanks for the instructions on using the Gradle wrapper in the command line. I was able to run the build command there by using ./gradlew build and got the same output.

Now, since one of your source files contains UTF-8 characters, the compiler complains. Thus, you have to configure the Gradle Compile task such that it expects the source files in UTF-8 format.

How do I configure the Gradle Compile task to expect source files in UTF-8 format? And why would that be necessary for an existing Gradle project and not for a new one?

The UTF-8 settings in the Eclipse preferences are not related/synchronized to the compiler settings of the Gradle Compile task when running a Gradle build.

I figured that one of the major benefits of having an Eclipse Gradle plugin, especially one that is officially supported by the Eclipse foundation, would be that this sort of synchronization might happen. Would this be something that would be appropriate to file as a feature request using the issue tracker?


(Etienne Studer) #9

You should find the information in the docs of Gradle and Javac:
https://docs.gradle.org/current/dsl/org.gradle.api.tasks.compile.JavaCompile.html

If not successful, @donat might be able to help you out.

What we could do is derive the Eclipse settings from the Gradle build - but not the other way around since the build is the ultimate source of truth when it comes to building the project. Everything should be derived from the build.


(Will Herrmann) #10

I found that I can successfully build if I add the following:

compileJava { 
  options.encoding = "UTF-8" 
}

Still not sure why this is necessary since it isn’t used in a fresh Gradle project, but it did solve the issue.


(Etienne Studer) #11

Great. Adding this option when using UTF-8 encoded source files is the right thing to do.

I would always add this option if you have UTF-8 encoded source files.


(Kirill Marchuk) #12

would be nice to see it reflected in some official documentation (including text of error) because googling for it brings a lot of noise.

Thx