Need help publishing artifact


(scott.raffanelli) #1

Assume I have a file that I generate in my build called: label.txt

I just simply want to publish this to Artifactory. I will even generate the POM for label.txt if need be. But I just can’t seem to figure out how to get it published.

Can someone give me a simple example of this? I am trying to hack my way through this basic scenario with no success. Even tried stuff like this:

configurations {
  myConfig
}
  // someFile is a reference to the label.txt file that I created
task buildMyNewConfig {
  configurations.myConfig.artifacts.add (someFile)
}
task artifactoryPublish.configuration = configurations.myConfig
task artifactoryPublish.doFirst {
 buildMyNewConfig
}

But there has to be a simpler way to publish an arbitrary artifact right?

(By the way, I am brand new to gradle so go easy on me. :slight_smile:


(René Groeschke) #2

you can use a task of the type ‘Upload’. The Upload task can be used to upload any configuration. all you need to do is to

  1. Create a customConfiguration (as you already did). 2. Pass the file to the configuration using the artifacts closure. 3. Create and configure your upload task.

I’ve created a tiny sample build file that shows how you can do that:

group = "org.acme"
version = "1.0-snapshot"
  configurations {
  myConfig
}
  artifacts{
 myConfig file("label.txt")
}
  task uploadLabel(type: Upload){
 configuration = configurations.myConfig
 repositories {
          maven {
         url 'http://localhost:8081/artifactory/acme'
        credentials {
          username 'user'
           password 'password'
         }
         }
  }
}

If you have any questions about the example, don’t hesitate to ask.

regards, René


(Peter Niederwieser) #3
artifacts {
  archives file("label.txt")
}

See 44.3.2. File artifacts in the Gradle User Guide.


What about if multiple files need to publish
(Peter Niederwieser) #4

Note that an ‘Upload’ task is automatically created for each configuration (assuming the ‘base’ plugin has been applied).

If Scott is using the Gradle Artifactory plugin (which provides an ‘artifactoryPublish’ task), it won’t be necessary to create/configure an ‘Upload’ task.


(scott.raffanelli) #5

Hi guys,

Thanks for the help thus far. I am just missing something obvious I think. :frowning:

I am indeed using the artifactory plugin. Here is the cleaned up build script. I still don’t know how to make it do the upload though.

import groovyx.net.http.*
import org.apache.http.entity.FileEntity
import groovy.xml.MarkupBuilder
import java.security.MessageDigest
  apply plugin: 'artifactory'
apply plugin: 'groovy'
  buildscript {
   repositories {
      maven {
         url '${artifactory_contextUrl}/plugins-release'
      }
     }
   dependencies {
      classpath(group: 'org.jfrog.buildinfo', name: 'build-info-extractor-gradle', version: '2.0.9')
      classpath(group: 'org.codehaus.groovy.modules.http-builder', name: 'http-builder', version: '0.5.1')
   }
}
   def someFile = file('label.txt')
  artifacts {
   archives someFile
}
  artifactory {
    contextUrl = "${artifactory_contextUrl}"
 //The base Artifactory URL if not overridden by the publisher/resolver
    publish {
        repository {
            repoKey = 'p4fa-gradle-repo'
            username = "${artifactory_user}"
            password = "${artifactory_password}"
            maven = false
        }
   }
   resolve {
      repository {
         repoKey = 'libs-release'
         username = "${artifactory_user}"
         password = "${artifactory_password}"
         maven = true
      }
   }
}

(René Groeschke) #6

Hello Scott, what problem do you currently have? do you not know which task to execute or does the upload fail? The artifactory plugin adds a task called “artifactoryPublish”. Have a look at http://wiki.jfrog.org/confluence/display/RTF/Gradle+Artifactory+Plugin to see how this task can be configured. My best guess at the moment is, that you have to set the correct configuration in the artifactoryPublish task.

regards, René


(scott.raffanelli) #7

Hi René,

Yeah, I am not sure what task to call. I have called both uploadArchives and artifactoryPublish. They both “succeed”, but no artifact gets put into my artifactory instance. For example, here is what I see:

Using uploadArchives, basically nothing happens.

[sraffane@adc2110217 p4fa.git]$ gradle uploadArchives
:compileJava UP-TO-DATE
:compileGroovy UP-TO-DATE
:processResources UP-TO-DATE
:classes UP-TO-DATE
:jar UP-TO-DATE
:uploadArchives
  BUILD SUCCESSFUL
  Total time: 13.087 secs

When I call artifactoryPublish, something does sort of happen, but it seems to publish artifacts I didn’t create.

[sraffane@adc2110217 p4fa.git]$ gradle artifactoryPublish
:artifactoryPublish
Deploying artifact: <my internal URL>/artifactory/p4fa-gradle-repo/p4fa.git/unspecified/pom-unspecified.xml
Deploying artifact: <my internal URL>/artifactory/p4fa-gradle-repo/p4fa.git/unspecified/p4fa.git-unspecified.jar
Deploying build info to: <my internal URL>/artifactory/api/build
  BUILD SUCCESSFUL
  Total time: 10.672 secs
[sraffane@adc2110217 p4fa.git]$

(René Groeschke) #8

As a default, the artifactory plugin uses the archives configuration for uploading. As you have applied the java plugin, this archive configuration contains the jar that was created by the jar task and the according pom file. If you want to upload a custom file, you have three options here I think:

  1. Add your custom file to the archives configuration using the snippet Peter provided above. 2. Create a custom configuration for your custom file and add the custom file to this configuration. Then

2.a) replace the configured archives configuration for the artifactoryPublish task by the custom configuration:

artifactoryPublish {
    publishConfigs ('myCustomConfig')
}

2.b) add the custom configuration as an additional configuration to the artifactoryPublish task

artifactoryPublish {
    publishConfigs ('archives', 'myConfig')
}

hope that helps. running the artifactoryPublish task with the -i option gives you more information what’s going on when the task is executed.

regards, René


(scott.raffanelli) #9

Thanks guys. I went back to just using the REST interface to put and get things. Can you tell me where gradle puts things that I retrieve via “GET”?

I poked around, but couldn’t find where there were being cached locally.

Thanks!


(René Groeschke) #10

Why do you went back to direct REST? How is this communication implemented? It depends on the code you use for http communication where your GETs are put. Can you provide some example code?


(scott.raffanelli) #11
def response = http.put (path: pathInRepo,
                            downloadUri: pathInRepo,
                            contentType: 'application/zip',
                      body: file//,
//leave
                          headers: ['X-Checksum-Deploy':'true','X-Checksum-Sha1':checksum]
                            )
    println 'Attempting to do a GET from: <my artifactory repo>/' + pathInRepo
    def getResponse = http.get (path: '<my artifactory repo>/' + pathInRepo
                             )

So basically, I am “putting” a zip file into artifactory. And then I immediately retrieve it. I am only retrieving it here, because I am trying to figure out where it will get put. I can’t find it though.

The PUT looked like this:

5:40:15.432 [DEBUG] [org.apache.http.headers] >> PUT /artifactory/p4fa-ps5rup3-repo/orc/patches4fa/atgpf/APM_RUP2_Bundle3_13782166/ps5rup3/generic/APM_RUP2_Bundle3_13782166.zip HTTP/1.1


(scott.raffanelli) #12

It’s a super long story on what I am trying to do, but I’ll try to give you the background. Today, we have a bunch of zip files. Around 120 or so. The content, filenames, number of artifacts, etc changes all the time. Right now they just sit on an NFS storage area. They are patches delivered by scores of different product teams in my company. We deliver these zip files to our customers. What I want to do is move them from this storage area and into artifactory. I want to take advantage of the ability to stage them into different phases of a release pipeline.

So the gradle build I am playing around with has no compilation or test step. I simply want to crawl this storage area, pick up all the zip files, and get them into Artifactory.

The on the consuming side, I want to be able to retrieve all the zip files and do certain actions on them. This is why I need to know where they get put when they are retrieved.

I would love to take advantage of the buildInfo that artifactory tracks, so that I can know which build copied which zips into artifactory. I don’t seem to get this when using the REST interface. And I am unable to really get any other mechanism going since there is no maven or ivy build here. There are arbitrary artifacts of ever changing size and name. Is there a way to dynamically add them to a configuration and then publish a configuration?

I also was thinking that I could create a “global” pom that tracks all the uploads I did. Then the consumer could just declare a dependency on the signle “global” pom. Since the consumer has no idea on the size, number of files, or names of the files, I figured this was the only way to go.


(René Groeschke) #13

To understand how to handle the data you get via http.get you should have a look at the HttpBuilder javadoc. see http://groovy.codehaus.org/modules/http-builder/apidocs/groovyx/net/http/HTTPBuilder.html#get(java.util.Map) for details


(scott.raffanelli) #14

Any ideas on the best way to handle a situation like mine?


(René Groeschke) #15

Hey Scott, You can write 2 lines of groovy code to dynamically add the zip files you want to upload to your artifactory server:

apply plugin:'base'
  group = "org.acme"
version = "1.0-snapshot"
  configurations {
  batchConfig
}
  artifacts{
  file("path/to/your/nfsstorage").eachFile{ zipFile ->
    batchConfig zipFile
  }
}
  uploadBatchConfig{
  repositories {
       ivy {
           url 'http://yourCompanyArtifactory:8081/acme'
          credentials {
            username 'user'
             password 'password'
           }
      }
   }
}

This publishes an ivy descriptor along to your artifacts. Using ivy makes it easy to reference configurations with lots of zip files in your consuming scripts.

regards, René


(scott.raffanelli) #16

This was very helpful, thank you!!! Is there a way I can control the layout? When I use [module] it ends up using the directory name of the project. I don’t want this. Here is the core of the loop that picks up these artifacts:

artifacts {
   installers_map.each { installer ->
      platforms_map.each { platform ->
         println 'Looking here: ' + goldenArea + '/' + version + '/' + installer + '/' + platform
         if (file (goldenArea + '/' + version + '/' + installer + '/' + platform).isDirectory()) {
            file (goldenArea + '/' + version + '/' + installer + '/' + platform + '/').eachFile { file ->
               println 'File added to config: ' + file
               batchConfig file: file, name: file.name, classifier: platform, module: installer
                 //
             batchConfig file: file, name: 'name', classifier: 'classifier', module: 'module', version: 'version', group: 'orc.patches4fa.' + installer
            }
         }
      }
   }
}

When I am adding them to the config, what am I allowed to change in terms of the values used to store the artifact?

I’d like it to be something like

/orc/patches4fa/<installer>/<platform>/<artifact>

(scott.raffanelli) #17

And one more question… Is there a way to control the layout on the consumption side?


(scott.raffanelli) #18

Hi guys, any help would be appreciated. I am struggling to understand how I can control the properties related to artifacts in the repository. I have a collection of artifacts that I want organized a certain way.

For example, I want to control the module name. But I can’t figure out how. Gradle seems to always just inject the name of my build directory (in this case p4fa.git).

In general I want to control everything about the layout. Something like this:

artifacts {
   installers_map.each { installer ->
      platforms_map.each { platform ->
         println 'Looking here: ' + goldenArea + '/' + version + '/' + installer + '/' + platform
         if (file (goldenArea + '/' + version + '/' + installer + '/' + platform).isDirectory()) {
             file (goldenArea + '/' + version + '/' + installer + '/' + platform + '/').eachFile { file ->
               println 'File added to config: ' + file
               batchConfig file: file, group: 'orc.patches4fa', module: installer, version: version, classifier: platform, baseRev: version
            }
         }
      }
   }
}

(Peter Niederwieser) #19

For example, I want to control the module name. But I can’t figure out how. Gradle seems to always just inject the name of my build directory (in this case p4fa.git).

Roughly speaking, the module name defaults to the Gradle project name, which defaults to the project directory name.

In general I want to control everything about the layout.

I suppose you want to do a bare-bone upload with full control over the artifacts’ coordinates (group, module, etc.). Once again, here is the link: 44.3.2. File artifacts

The full set of available properties can be found in the Javadoc for ConfigurablePublishArtifact. Apparently, the group can’t be set at this level, but you can set ‘project.group’.