Tutorial: How to push / upload artifacts into the Maven Central Repository

I have several GitHub repositories and that's great. It's so easy to share the projects. But for some projects it's not enough. I created a POM, which I'm using as parent POM for all my projects. So if you want to checkout a project which depends on the POM then it's necessary to checkout the pom-project first and do a $ mvn install. That's quite annoying. The second example is my spBee library. I think most users aren't interested to checkout that project and to contribute to it - they just want to use it. Therefore it's necessary in such cases to put the artifacts on the Maven Central Repository. Then the user can add the library to the Maven dependencies and everything works. So I faced the question: how to push my artifacts to the Maven Central Repository?

A short overview over all steps

1. install PGP - that's necessary to sign the artifacts - and upload your public key 
2. prepare the POM file to satisfy all requirements
3. create an Sonatype account
4. create a New Project ticket
5. wait for a comment.
6. perform a stating release and comment the ticket that you have successfully done the stating release and the artifacts are ready to be released.
7. wait until the sync to the Maven Central Repository will be activated and now you can do the releases yourself.
1. do another staging release
2. release your artifacts on this site https://oss.sonatype.org/
3. or drop your staging release
8. enjoy your artifacts on the Maven Central Repository

It seems to be quite complicated. But actually it's not that bad. Hopefully it's very easy with the help of this post!

1. Install PGP and upload your public key
 On this site everything is explained. In short:

• download and install GPG
• $ gpg --version
• $ gpg --gen-key
• enter the required information
• $ gpg --list-keys
• in the output you will see the keyid of the public certificate
• pub   1024D/C6EED57A 2010-01-13
• the C6EED57A string is the keyid
• distribute the public key so that the signed files can be verified
• $ gpg --keyserver hkp://pool.sks-keyservers.net --send-keys C6EED57A

2. Prepare the POM file to satisfy all requirements
On this site and that site everything is explained. In short:

• the groupId
• if the groupId is at.rseiler.spbee then you need to own rseiler.at.
• if you don't have an own domain, but use GitHub then the groupId must be: github.com/rseiler => com.github.rseiler
• the javadoc.jar must be generated
• the sources.jar must be generated
• all files must be signed with PGP
• following meta data must be provided
• project name, description and URL
• license information
• developer information 
• scm (the repository URL)
• the nexus-staging-maven-plugin must be setup
• the distributionManagement (snapshotRepository and
  repository) must be setup

Take a look at these both small POM files, which satisfies all requirements: spBee POM pom-project POM
The  release profile will create the javadoc.jar sources.jar and signs the artifacts. So if you do a release you need to activate the profile with: $ mvn clean deploy -P release

To upload the artifacts you need to setup the settings.xml (.m2/settings.xml).

<settings>
  <servers>
    <server>
      <id>ossrh</id>
      <username>your-jira-id</username>
      <password>your-jira-pwd</password>
    </server>
  </servers>
</settings>

3. Create an Sonatype account
Go to this site and create an account.

4. Create a New Project ticket

• go to this site and create a ticket - see mine as an example
• enter the root groupId
• at.rseiler - even if your first artifact uses at.rseiler as groupId
• com.github.rseiler - if you don't have an own domain
• fill out the rest of the fields

6. Preform a stating release

• check if everything is setup correctly
• I recommend to set <autoReleaseAfterClose>true</autoReleaseAfterClose> to false so you can check the output first
• notice that there aren't allowed any JavaDoc errors. If there are errors than the javadoc.jar file won't be created and then the requirements are missed.
• $ mvn clean deploy -P release
• check on this site under Build Promation => Repositories => Content your uploaded artifacts
• if everything is fine then use: $ mvn nexus-staging:release to do a stating release
• otherwise $ mvn nexus-staging:drop to drop the stating release
• both commands can be executed on the website, too
• comment the ticket

7. Release yourself
After the sync is activated you can release your stating-releases yourself to the Maven Central Repository.

8. Enjoy your artifacts on the Maven Central Repository 
You have done it! Congratulation! :)


I hope that my blog post helped you and gave you a good overview over all required steps.

Publish Maven Site Documentation automatically to the GitHub Pages

Maven site is a great tool as well as GitHub. The coolest thing about those tools is that they play together very well. Only a little Maven configuration is needed. But let me explain shortly what both tools do.

Maven Site: generates automatically a nice documentation website for your Maven project. Many Apache projects uses this generated site as their main website. So probably you have already seen such generated websites. An nice example is the my (parent) POM project or the maven-site-plugin. Mainly the documentation consists of three parts:

• Project information like dependencies, license, source repository and so on. 
• Project reports like unit tests reports, static code analysis reports, JavaDocs and so on. 
• The documentation written by the developers. With an additional Maven Plugin it's possible to write the documentation with Markdown.

GitHub Pages: GitHub doesn't only provide a GIT repository but also can be used as webhost for static website. Which is ideal for a project website. The GitHub Pages concept is very cool. It's just a branch, called gh-pages, in you project repository. All the files that are pushed into this branch will be served from GitHub`s webserver. An nice example is the documentation for my (parent) POM project.
Before you start with the Maven configuration you should read Creating Project Pages manually for a better understanding.

Before we can start with the GitHub Pages integration into Maven we firstly need to correctly build the website. For multi module projects it's necessary to deploy (locally is sufficient) the website first. Otherwise the references between modules won't work. To do so you just need to configure the distribution management like this:

<distributionManagement>
<site>
<id>site-docs</id>
<url>file://${env.HOME}/sitedocs/pom-project</url>
</site>
</distributionManagement>

How to fully configure the generation of the Maven Site is too much for this blog post. Look at the POM for some good basic configuration or just us it as parent POM for your project. The minimum of configuration is:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-site-plugin</artifactId>
            <version>3.4</version>
            <executions>
                <!-- used for multiproject builds -->
                <execution>
                    <id>attach-descriptor</id>
                    <goals>
                        <goal>attach-descriptor</goal>
                    </goals>
                </execution>
            </executions>
            <dependencies>
                <!-- To use the Markdown format -->
                <dependency>
                    <groupId>org.apache.maven.doxia</groupId>
                    <artifactId>doxia-module-markdown</artifactId>
                    <version>1.6</version>
                </dependency>
            </dependencies>
        </plugin>
    </plugins>
</build>

<reporting>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-project-info-reports-plugin</artifactId>
            <version>2.7</version>
            <configuration>
                <dependencyLocationsEnabled>false</dependencyLocationsEnabled>
            </configuration>
            <reportSets>
                <reportSet>
                    <reports>
                        <report>index</report>
                        <!--<report>cim</report>-->
                        <report>dependencies</report>
                        <!--<report>dependency-convergence</report>-->
                        <report>dependency-info</report>
                        <report>dependency-management</report>
                        <!--<report>distribution-management</report>-->
                        <!--<report>issue-tracking</report>-->
                        <report>license</report>
                        <!--<report>mailing-list</report>-->
                        <report>modules</report>
                        <report>plugin-management</report>
                        <report>project-team</report>
                        <report>scm</report>
                        <report>summary</report>
                    </reports>
                </reportSet>
            </reportSets>
        </plugin>
    </plugins>
</reporting>

To build the Maven Site just use: mvn clean site site:deploy

The integration into the GitHub pages is very easy and looks like this:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-scm-publish-plugin</artifactId>
            <version>1.1</version>
            <inherited>true</inherited>
            <configuration>
                <checkoutDirectory>${project.basedir}/github.com</checkoutDirectory>
                <checkinComment>publishing site documentation</checkinComment>
                <content>${env.HOME}/sitedocs/pom-project</content>
                <pubScmUrl>scm:git:https://github.com/rseiler/pom-project.git</pubScmUrl>
                <scmBranch>gh-pages</scmBranch>
            </configuration>
        </plugin>
    </plugins>
</build>

Additional you need to configure your username and password in the .m2/settings.xml. It should look like this:

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
                      http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <servers>
    <server>
      <id>github.com</id>
      <username>github-username</username>
      <password>github-password</password>
    </server>
  </servers>
</settings>

If everything is done just type: 

mvn clean site site:deploy scm-publish:publish-scm -Dscmpublish.dryRun=true

The scmpublish.dryRun flag prevents the plugin to commit anything and just outputs what changes it would commit. Check it and if everything works than rerun the command without this flag: 

mvn clean site site:deploy scm-publish:publish-scm

That's it! It's Very simple :)

Class Transformation with ASM

Have you ever asked yourself how class transformation works? Great, then you are reading the right blog post ;-)

Probably not everybody knows what class transformation is. Therefore I will start to explain it before I will explain how it works. 

What is class transformation?

Actually it's there is a quite simple answer to this question: the Java bytecode will be modified in some kind. But let me explain it more detailed. If you compile a Java file a class file will be generated. The class file represents the Java source file as Java binary code. So it's much smaller and optimized for execution. The methods consists of the Java opcodes. These opcodes will pushed sequentially onto the Java stack and will be executed. Class transformation means that the Java byte code, which represents a Java class, will be modified. So opcodes can be inserted or removed. But not only the opcodes inside of a method can be modified. Everything can be changed - any program can be transformed in anything else! At least as it is still valid Java byte code. Otherwise the Java Bytecode Verifier will reject the class if the class will be loaded.

Where is class transformation used?

Why would you transform a class if you just could write the Java class like you need it to be? Actually if you can accomplish your work without class transformation than don't use class transformation. Just write the Java code accordingly. I think the most commonly usage of class transformation is to instrument Java code at runtime. Imagine if you have a big program which has performance problems and there are no performance tools like VisualVM or JProfiler. What would you do to find the methods which takes long to execute?
You would have to insert at each method the code to measure the execution duration of the method. If there are thousands of methods this would be a quite boring work. Especially since you need to remove the code for the production code and probably add it again to do the analyze the execution durations again. With class transformation you can do exactly this boring work. You don't write the duration measurement code in you Java files. But you read the existing class file and insert the needed opcodes to each class, to each method.
Actually all performance tools work like this. Java allows to modify already loaded classes, with some restrictions, too. So these tools gets the binary code of the classes, rewrites the classes on a binary level and Java loads the modified classes. Than the tools can generate analysis and pretty diagrams from the instrumented classes.

ASM

ASM is a great library which allows to transform classes. It consists of three main parts

•  ClassReader: reads a binary class
• ClassWriter: writes a binary class
• ClassVisitor: transforms the binary class by calling the visit-methods of your implementation

To be able to create class transformations you need to understand the Java opcodes and how a stack based language works. At least you need a rudimentary understanding.

There are tools which will show you the byte code of any class and generate the ASM code to create this class with ASM. Therefore you don't need to write all the opcodes by hand. Just write a Java class which should be the result of your transformation. Than look at the generated code and adapt it to your needs. In theory that sounds very easy. But at least I had to read the ASM documentation and the Java opcodes documentation, too. To make a quite simple transformation work. 

 Example

This example, which can be found completely on GitHub, does two things.

• Wraps all static Logger variables
• Logs all method calls

For more details read the code. It's heavily documented and it makes more sense to read than anything else. Have fun! :-)

The code on GitHub with syntax hilighting, yeah: https://github.com/rseiler/concept-class-transformation-with-asm/blob/master/src/main/java/at/rseiler/concept/Main.java!

package at.rseiler.concept;

import org.objectweb.asm.*;
import org.objectweb.asm.commons.AdviceAdapter;

import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.lang.reflect.Method;
import java.util.logging.Logger;

import static org.objectweb.asm.Opcodes.ASM4;
import static org.objectweb.asm.Opcodes.INVOKESTATIC;

/**
 * A demo how to do byte code transformation with ASM.
 * 
 * The program will load the HelloWorld class file and manipulate the byte code:
 * 
 * 1. Wraps static {@link Logger} into the {@link LoggerWrapper#logger(Logger)}
 * 2. Adds at the beginning of each method a call to {@link MethodLogger#log(String, Object...)}
 * 
 * 1.
 * private static final Logger logger1 = Logger.getLogger(HelloWorld.class.getName());
 * will be transformed into:
 * private static final Logger logger1 = LoggerWrapper.logger(Logger.getLogger(HelloWorld.class.getName()));
 * 
 * 2.
 * public String foo(String arg) {
 * return bar("foo", arg);
 * }
 * will be transformed into:
 * public String foo(String arg) {
 * MethodLogger.log("foo", arg);
 * return bar("foo", arg);
 * }
 * 
 * 
 * You shouldn't relay on the ASM version packed into the jdk for production code!
 * Because if a new Java version will be shipped than it could contain a new version of AMS (or remove ASM) which will break your code.
 * Therefor you must repackage ASM into your own namespace, to prevent version conflicts, and ship it with your library.
 * 
 * Because this is non production code and I am lazy I didn't do it.
 * 
 * IMPORTANT: If you try to run the program on a JMV other than the JDK8 it will probably fail.
 *
 * @author reinhard.seiler@gmail.com
 */
public class Main {

    public static void main(String[] args) throws Exception {
        // creates the ASM ClassReader which will read the class file
        ClassReader classReader = new ClassReader(new FileInputStream(new File("HelloWorld.class")));
        // creates the ASM ClassWriter which will create the transformed class
        ClassWriter classWriter = new ClassWriter(ClassWriter.COMPUTE_MAXS);
        // creates the ClassVisitor to do the byte code transformations
        ClassVisitor classVisitor = new MyClassVisitor(ASM4, classWriter);
        // reads the class file and apply the transformations which will be written into the ClassWriter
        classReader.accept(classVisitor, 0);

        // gets the bytes from the transformed class
        byte[] bytes = classWriter.toByteArray();
        // writes the transformed class to the file system - to analyse it (e.g. javap -verbose)
        new FileOutputStream(new File("HelloWorld$$Transformed.class")).write(bytes);

        // inject the transformed class into the current class loader
        ClassLoader classLoader = Main.class.getClassLoader();
        Method defineClass = ClassLoader.class.getDeclaredMethod("defineClass", String.class, byte[].class, int.class, int.class);
        defineClass.setAccessible(true);
        Class helloWorldClass = (Class) defineClass.invoke(classLoader, null, bytes, 0, bytes.length);

        // creates an instance of the transformed class
        Object helloWorld = helloWorldClass.newInstance();
        Method hello = helloWorldClass.getMethod("hello");
        // class the hello method
        hello.invoke(helloWorld);
    }

    private static class MyClassVisitor extends ClassVisitor {

        public MyClassVisitor(int i, ClassVisitor classVisitor) {
            super(i, classVisitor);
        }

        public MethodVisitor visitMethod(int access, String name, String desc, String signature, String[] exceptions) {
            if (cv == null) {
                return null;
            }

            MethodVisitor mv = super.visitMethod(access, name, desc, signature, exceptions);
            //  defines the static block in which the assignment of static variables happens.
            // E.g. private static final Logger logger = Logger.getLogger(HelloWorld.class.getName());
            // The assignment of the logger variable happens in .
            if ("".equals(name)) {
                return new StaticBlockMethodVisitor(mv);
            } else {
                // all other methods (static and none static)
                return new MethodLogger(mv, access, name, desc);
            }
        }

        class StaticBlockMethodVisitor extends MethodVisitor {
            StaticBlockMethodVisitor(MethodVisitor mv) {
                super(ASM4, mv);
            }

            @Override
            public void visitFieldInsn(int opcode, String owner, String name, String desc) {
                // checks for: putstatic // Field *:Ljava/util/logging/Logger;
                if ("Ljava/util/logging/Logger;".equals(desc)) {
                    // adds before the putstatic opcode the call to LoggerWrapper#logger(Logger) to wrap the logger instance
                    super.visitMethodInsn(INVOKESTATIC, "at/rseiler/concept/LoggerWrapper", "logger", "(Ljava/util/logging/Logger;)Ljava/util/logging/Logger;");
                }
                // do the default behaviour: add the putstatic opcode to the byte code
                super.visitFieldInsn(opcode, owner, name, desc);
            }
        }

        class MethodLogger extends AdviceAdapter {

            private final int access;
            private final String name;
            private final String desc;

            protected MethodLogger(MethodVisitor mv, int access, String name, String desc) {
                super(ASM4, mv, access, name, desc);
                this.access = access;
                this.name = name;
                this.desc = desc;
            }

            @Override
            protected void onMethodEnter() {
                // checks if the method is static.
                // The difference is that "this" is stored in ALOAD_0 and the arguments are stored in ALOAD_1, ALOAD_2, ...
                // But there is no "this" for a static method call. Therefor the arguments are stored in ALOAD_0, ALOAD_1 ,...
                // If we want to access the arguments we need to differentiate between static and non static method calls.
                boolean isStatic = (access & ACC_STATIC) > 0;

                int length = Type.getArgumentTypes(desc).length;

                // pushes the method name on the stack
                super.visitLdcInsn(name);
                // pushes the count of arguments on the stack
                // could be optimized if we would use iconst_0, iconst_1, ..., iconst_5 for 0 to 5.
                super.visitIntInsn(BIPUSH, length);
                // creates an object array with the count of arguments
                super.visitTypeInsn(ANEWARRAY, "java/lang/Object");

                // stores the arguments in the array
                for (int i = 0; i < length; i++) {
                    // duplicates the reference to the array. Because the AASTORE opcode consumes the stack element with the reference to the array.
                    super.visitInsn(DUP);
                    // could be optimized
                    super.visitIntInsn(BIPUSH, i);
                    // puts the value of the current argument on the stack
                    super.visitVarInsn(ALOAD, i + (isStatic ? 0 : 1));
                    // stores the value of the current argument in the array
                    super.visitInsn(AASTORE);
                }

                // calls the MethodLogger#log(String, Object...) method with the corresponding arguments - which we created just before
                super.visitMethodInsn(INVOKESTATIC, "at/rseiler/concept/MethodLogger", "log", "(Ljava/lang/String;[Ljava/lang/Object;)V");
            }
        }

    }

}

Explanation how CGLIB proxies work

 In my lastblog entry I explained how proxy based mock frameworks work and created my own demo mock framework based on CGLIB. Now I will explain how exactly the CGLIB proxy works. Because then you will understand why these things doesn't work with a proxy:

• intercept static method calls
• intercept private method calls
• intercept final method calls
• build a proxy for a final class

ASM

CGLIB is based on ASM. ASM is a library to create class files, on bytecode level, on the fly. So it's very low level and you need a good understanding of the bytecode to use it. But it abstracts the bytecode and is much more comfortable to generate bytecode with ASM than writing the bytecode itself. Because it takes care of the common structures like classes, constructors, method definitions and so on. One of the most helpful features is that it takes care about the constant pool. Because the constant pool holds many elements and you need to adapt it if you change anything. E.g. if a new method is added than you have to add the needed elements to the constant pool and increase the constant pool size. Or the labels in ASM are quite nice, too. Or that there are constants for the opcodes. After the complete class is generated, you have the bytecode of the class. This bytecode is passed into the ClassLoader which will load the class and then the class can be used.

How the generated CGLIB looks

Now you know how the generation of the proxy works. The question is how the generated proxy looks like. Actually it's quite simple. The proxy class extends the original class. All classes which should be intercepted by the proxy will be overridden. These methods will call the MethodInterceptor:: intercept method with the corresponding parameters. If the proxy callback is null than the super method will be called. Here is the code snippet for a simple echo(String) method:

public class at.rseiler.concept.mock.Foo$$EnhancerByCGLIB$$c0699eac

  extends at.rseiler.concept.mock.Foo

  implements org.mockito.cglib.proxy.Factory

{

  private static final org.mockito.cglib.proxy.MethodProxy CGLIB$echo$1$Proxy;

  private static final java.lang.reflect.Method CGLIB$echo$1$Method;

  // ...

  private org.mockito.cglib.proxy.MethodInterceptor CGLIB$CALLBACK_0;

  // ...

  public final String echo(String arg0) {

    if(CGLIB$CALLBACK_0 == null) {

      CGLIB$BIND_CALLBACKS(this);

    }

    if(CGLIB$CALLBACK_0 != null) {

      // MethodInterceptor::intercept(Object obj, Method method, Object[] args, MethodProxy proxy)

      Object obj = CGLIB$CALLBACK_0.intercept(this, GLIB$echo$0$Method, new Object[] {arg0}, CGLIB$echo$0$Proxy);

      if(!(obj instanceOf String)) {

        throw new ClassCastException();

      }

      return obj;

    }

    return super.echo(arg0);

  }

  private static final void CGLIB$BIND_CALLBACKS(java.lang.Object object) {

    /* compiled code */

  }

  // ...

}

Now you should understand the limits of the proxy (see above). The limits caused because the proxy is a subclass of the original class. Therefore it's not possible to extend a final class or final methods and so on. That's just basic Java limitations.
With code generation it's not possible to work around these limits. Sure it would be possible to generate a proxy class without the relationship to the original class. Than it would be possible to copy the whole class and insert the MethodInterceptor::intercept calls. This new class could remove all final keywords and make private methods public. The problem is that you can't call any method of this class without reflection. Because you don't have the type information of the "proxy" class at  compile time - clearly because the class is generated at runtime. Further you can't pass the "proxy" object to any constructor or any method because it's not a subclass.

The only way to break this limits is to do bytecode transformations of the original class before the ClassLoader loads the class. Before the class is loaded replace all private keywords with public keywords and to remove all final keywords. To do so you need to create your own ClassLoader which will performs the bytecode transformation. I don't know the code of PowerMock, but it must work like this.

At last the with javap --verbose decompiled bytecode of the echo(String) method from which I created the Java code above:

public final java.lang.String echo(java.lang.String);

flags: ACC_PUBLIC, ACC_FINAL

Code:

stack=7, locals=2, args_size=2

0: aload_0

1: getfield      #37                 // Field CGLIB$CALLBACK_0:Lorg/mockito/cglib/proxy/MethodInterceptor;

4: dup

5: ifnonnull     17

8: pop

9: aload_0

10: invokestatic  #41                 // Method CGLIB$BIND_CALLBACKS:(Ljava/lang/Object;)V

13: aload_0

14: getfield      #37                 // Field CGLIB$CALLBACK_0:Lorg/mockito/cglib/proxy/MethodInterceptor;

17: dup

18: ifnull        45

21: aload_0

22: getstatic     #64                 // Field CGLIB$echo$1$Method:Ljava/lang/reflect/Method;

25: iconst_1

26: anewarray     #66                 // class java/lang/Object

29: dup

30: iconst_0

31: aload_1

32: aastore

33: getstatic     #68                 // Field CGLIB$echo$1$Proxy:Lorg/mockito/cglib/proxy/MethodProxy;

36: invokeinterface #53,  5           // InterfaceMethod org/mockito/cglib/proxy/MethodInterceptor.intercept:(Ljava/lang/Object;Ljava/lang/reflect/Method;[Ljava/lang/Object;Lorg/mockito/cglib/proxy/MethodProxy;)Ljava/lang/Object;

41: checkcast     #55                 // class java/lang/String

44: areturn

45: aload_0

46: aload_1

47: invokespecial #62                 // Method at/rseiler/concept/mock/Foo.echo:(Ljava/lang/String;)Ljava/lang/String;

50: areturn