Record method calls in one session for replaying in future test sessions?

Question

I have a backend system which we use a third-party Java API to access from our own applications. I can access the system as a normal user along with other users, but I do no

Answer 1

I had the same needs some months ago for non-regression testing when planning a heavy technical refactoring of a large application and... I have found nothing available as a framework.

In fact, replaying may be particularly difficult and may only work in a specific context - no (or few) application with a standard complexity can be really considered as stateless. It is a common problem when testing persistence code with a relational database. To be relevant, the complete system initial state must be restored and each replay step must impact the global state the same way. It becomes a challenge when a system state is distributed into pieces like databases, files, memory... Let's guess what happens if a timestamp taken from a system's clock is used somewhere !

So a more pratical option is to only record... and then do a clever comparison for subsequent runs.

Depending of the number of runs you plan, a human-driven session on the application may be enough, or you have to investing in an automated scenario in a robot playing with your application user interface.

First to record: you can use dynamic proxy interface or aspect programming to intercept method call and to capture state before and after invocation. It may mean: dump concerned database tables, copy some files, serialize Java objects in text format like XML.

Then compare this reference capture with a new run. This comparison should be tuned to exclude any irrelevant elements from each piece of state, like row identifiers, timestamps, file names... to only compare data where your backend's added value shines.

Finally nothing really standard, and often a few specific scripts and codes may be enough to achieve the aim: detect as much errors as possible and try to prevent non-expected side-effects.

Answer 2

Actually You can build such framework or template, by using proxy pattern. Here I explain, how you can do it using dynamic proxy pattern. The idea is to,

1. Write a proxy manager to get recorder and replayer proxies of API on demand!
2. Write a wrapper class to store your collected information and also implement hashCode and equals method of that wrapper class for efficient lookup from Map like data structure.
3. And finally use recorder proxy to record and replayer proxy for replaying purpose.

How recorder works:

1. invokes the real API
2. collects the invocation information
3. persists data in expected persistence context

How replayer works:

1. Collect the method information (method name, parameters)
2. If collected information matches with previously recorded information then return the previously collected return value.
3. If returned value does not match, persist the collected information (As you wanted).

Now, lets look at the implementation. If your API is MyApi like bellow:

public interface MyApi {
    public String getMySpouse(String myName);
    public int getMyAge(String myName);
    ...
}

Now we will, record and replay the invocation of public String getMySpouse(String myName). To do that we can use a class to store the invocation information like bellow:

    public class RecordedInformation {
       private String methodName;
       private Object[] args;
       private Object returnValue;

        public String getMethodName() {
            return methodName;
        }

        public void setMethodName(String methodName) {
            this.methodName = methodName;
        }

        public Object[] getArgs() {
            return args;
        }

        public void setArgs(Object[] args) {
            this.args = args;
        }

        public Object getReturnValue() {
            return returnType;
        }

        public void setReturnValue(Object returnValue) {
            this.returnValue = returnValue;
        }

        @Override
        public int hashCode() {
            return super.hashCode();  //change your implementation as you like!
        }

        @Override
        public boolean equals(Object obj) {
            return super.equals(obj);    //change your implementation as you like!
        }
    }

Now Here comes the main part, The RecordReplyManager. This RecordReplyManager gives you proxy object of your API , depending on your need of recording or replaying.

    public class RecordReplyManager implements java.lang.reflect.InvocationHandler {

        private Object objOfApi;
        private boolean isForRecording;

        public static Object newInstance(Object obj, boolean isForRecording) {

            return java.lang.reflect.Proxy.newProxyInstance(
                    obj.getClass().getClassLoader(),
                    obj.getClass().getInterfaces(),
                    new RecordReplyManager(obj, isForRecording));
        }

        private RecordReplyManager(Object obj, boolean isForRecording) {
            this.objOfApi = obj;
            this.isForRecording = isForRecording;
        }


        @Override
        public Object invoke(Object proxy, Method method, Object[] args) throws Throwable {
            Object result;
            if (isForRecording) {
                try {
                    System.out.println("recording...");
                    System.out.println("method name: " + method.getName());
                    System.out.print("method arguments:");
                    for (Object arg : args) {
                        System.out.print(" " + arg);
                    }
                    System.out.println();
                    result = method.invoke(objOfApi, args);
                    System.out.println("result: " + result);
                    RecordedInformation recordedInformation = new RecordedInformation();
                    recordedInformation.setMethodName(method.getName());
                    recordedInformation.setArgs(args);
                    recordedInformation.setReturnValue(result);
                    //persist your information

                } catch (InvocationTargetException e) {
                    throw e.getTargetException();
                } catch (Exception e) {
                    throw new RuntimeException("unexpected invocation exception: " +
                            e.getMessage());
                } finally {
                    // do nothing
                }
                return result;
            } else {
                try {
                    System.out.println("replying...");
                    System.out.println("method name: " + method.getName());
                    System.out.print("method arguments:");
                    for (Object arg : args) {
                        System.out.print(" " + arg);
                    }

                    RecordedInformation recordedInformation = new RecordedInformation();
                    recordedInformation.setMethodName(method.getName());
                    recordedInformation.setArgs(args);

                    //if your invocation information (this RecordedInformation) is found in the previously collected map, then return the returnValue from that RecordedInformation.
                    //if corresponding RecordedInformation does not exists then invoke the real method (like in recording step) and wrap the collected information into RecordedInformation and persist it as you like!

                } catch (InvocationTargetException e) {
                    throw e.getTargetException();
                } catch (Exception e) {
                    throw new RuntimeException("unexpected invocation exception: " +
                            e.getMessage());
                } finally {
                    // do nothing
                }
                return result;
            }
        }
    }

If you want to record the method invocation, all you need is getting an API proxy like bellow:

    MyApi realApi = new RealApi(); // using new or whatever way get your service implementation (API implementation)
    MyApi myApiWithRecorder = (MyApi) RecordReplyManager.newInstance(realApi, true); // true for recording
    myApiWithRecorder.getMySpouse("richard"); // to record getMySpouse
    myApiWithRecorder.getMyAge("parker"); // to record getMyAge
    ...

And to replay all you need:

    MyApi realApi = new RealApi(); // using new or whatever way get your service implementation (API implementation)
    MyApi myApiWithReplayer = (MyApi) RecordReplyManager.newInstance(realApi, false); // false for replaying
    myApiWithReplayer.getMySpouse("richard"); // to replay getMySpouse
    myApiWithRecorder.getMyAge("parker"); // to replay getMyAge
    ...

And You are Done!

Edit: The basic steps of recorder and replayers can be done in above mentioned way. Now its upto you, that how you want to use or perform those steps. You can do what ever you want and whatever you like in the recorder and replayer code blocks and just choose your implementation!

Answer 3

This can be done with AOP, aspect oriented programming. It allows to intercept method calls by byte code manipulation. Do a bit of search for examples.

In one case this can do recording, in the other replaying.

Pointers: wikipedia, AspectJ, Spring AOP.

Unfortunately one moves a bit outside the java syntax, and a simple example can better be sought elsewhere. With explanation.

Maybe combined with unit tests / some mocking test framework for offline testing with recorded data.