Gradle task of type copy changes lineendings when using expand

Bugs
1

When I use the copy task on a file that contains Windows linendings (CRLF) then the target file has Windows lineendings as expected.
If I use “expand” in the copy task then if a file that contains Windows lineendings (CRLF) is copied the target file has UNIX lineendings CR instead of keeping CRLF.

Here is a gradle.build to demonstrate (unfortunatly I am not allowed to upload the files.

import groovy.io.FileType



def src = file('TEMPLATE')
def tgt_with_substitution = file('RESULT/with_substitution')
def tgt_without_substitution = file('RESULT/without_substitution')


task copy_without_substitution (type: Copy) {
    from src
    into tgt_without_substitution
    rename { fileName ->
        "without_substitution_${(fileName - 'template_')}"
    }
}


task copy_with_substitution (type: Copy) {
    from src
    into tgt_with_substitution
    expand(gradle_fan: 'Kai')
    rename { fileName ->
        "with_substitution_${(fileName - 'template_')}"
    }
}

boolean has_windows_lineendings(File file) {
    String fileContents = file.text
    return ( fileContents =~ /\r\n/ )
}

task examine_results {
    dependsOn copy_with_substitution
    dependsOn copy_without_substitution
}

examine_results.doLast( {

    tgt_without_substitution.eachFile(FileType.FILES){ f -> 
    printf "%44s %s\n", f.name, ( has_windows_lineendings(f) ? "CRLF (Windows)" : "CR   (Unix)" )
    }
    tgt_with_substitution.eachFile(FileType.FILES){ f -> 
    printf "%44s %s\n", f.name, ( has_windows_lineendings(f) ? "CRLF (Windows)" : "CR   (Unix)" )
    }

    
    println '''
The difference occurs when this script is executed in Windows.
This is a bit of surprise, the behavior I would expect is that,
line endings of the template are not changed when substituting
i.e. 
with_substitution_windows_lineendings.txt       CR (Windows)
instead of
with_substitution_windows_lineendings.txt       CR (Unix)
'''
})
2

Hi @kai_von_thadden what’s the contents of TEMPLATE?

3

Hi Sterling,

I ran this on Windows and reproduced the issue with Gradle 2.4, 2.5 and 2.6
The Directory TEMPLATE contains two files:

  • template_unix_lineendings.txt
  • template_windows_lineendings.txt

Content of template_unix_lineendings.txt. As you might think the file has LF (Unix) lineendings.

	This text is UTF-8 without BOM with UNIX lineendings (LF)

Hello ${gradle_fan},

Gradle unfortunately does not preserve line endings on substitution.

Best regards
Kai

Content of template_unix_lineendings.txt is similar but as you might think the file has CR+LF (Windows) lineendings.

	This text is UTF-8 without BOM with WINDOWS lineendings (CR+LF)

Hello ${gradle_fan},

Gradle unfortunately does not preserve line endings on substitution.

Best regards
Kai

And sorry for causing additional confusion. I didn’t note that I wrote CR for Unix lineending instead of LF.

Best regards
Kai

4

Thanks, this is from line ending normalization that the underlying Groovy class we’re using (SimpleTemplateEngine) does: https://issues.apache.org/jira/browse/GROOVY-3487

A workaround is to end your lines with \r, which will preserve the windows line endings.

You might be able to also use an Ant filter https://ant.apache.org/manual/Tasks/fixcrlf.html

5

Dear Sterling,

thank you for pointing to GROOVY-3487. This makes pretty clear why this happens.

The Ant filter will help me to implement a well working workaround in my build.gradle file.
I will add a dependent task for all files with destination platform Windows.
As the target of the real build process are some ZIP files for Windows and some ZIP files for UNIX,
the task chain will unfortunately get longer, as I now need to stage in a temporary directory.

Would be nice if this implicit Groovy optimization could be controlled in a declarative manner in the Copy task
specification at some point.

Thank you very much for your help.

Best regards
Kai

6

To skip the staging step, you could write your own FilterReader that does the expand + CRLF fixing (when necessary).