Execute a user program in parallel

This topic explains parallel execution of a user program using RaSC. Before starting the following steps, complete the tutorial Run a user program as a RaSC service to start a RaSC service.

Configure a service definition XML

Open the service definition XML to change the service property from jp.go.nict.wisdom.wrapper.StdIOCommandService to jp.go.nict.wisdom.wrapper.StdIOCommandParallelArrayService as follows. Make sure poolSize and initPoolSize are also set.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE beans PUBLIC "-//SPRING//DTD BEAN//EN" "http://www.springframework.org/dtd/spring-beans.dtd">
<beans>
  <bean id="target"
    class="jp.go.nict.langrid.servicecontainer.handler.TargetServiceFactory">
    <property name="service">
      <bean class="jp.go.nict.wisdom.wrapper.StdIOCommandParallelArrayService">
        <property name="cmdLine" value="___PATH_TO_PROGRAM___" />
        <property name="delimiterIn" value="\n" />
        <property name="delimiterOut" value="EOS\n" />
        <property name="delLastNewline" value="true" />
        <property name="poolSize" value="8" />
        <property name="initPoolSize" value="8" />
      </bean>
    </property>
  </bean>
</beans>

jp.go.nict.wisdom.wrapper.StandardInputArrayParallelService has implemented a strategy for parallel execution. Calling String[] analyzeArray(String[] input) of the class will start the process instances of a user program according to poolSize. The input elements are assigned to each process instance. In the case of the above setting file, the number of process instances will be increased to 8 as poolSize is 8.

By default, a process instance will be started only after a request is received. But initPoolSize indicates the number of process instances of the user program immediately after the RaSC service is started. For this reason, the response to the first request can be processed without waiting for a process instance to start.

When you change service definition XML, you must restart the service. Refer to Run a user program as a RaSC service for commands to start or stop a RaSC service.

Change a client to enable parallel execution

SampleClient.java contained in the core package calls analyze(), which processes one input. Change this program so that it gets data from a file and calls analyzeArray() instead.

import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStreamReader;
import java.net.InetSocketAddress;
import java.util.ArrayList;
import java.util.List;

import jp.go.nict.langrid.client.msgpackrpc.MsgPackClientFactory;
import jp.go.nict.wisdom.wrapper.api.TextAnalysisService;

public class SampleClient {
  public static void main(String[] args) throws Exception {
    MsgPackClientFactory factory = new MsgPackClientFactory();

    TextAnalysisService client = factory.create(TextAnalysisService.class,
              new InetSocketAddress(args[0], Integer.parseInt(args[1])));

    BufferedReader br = null;
    try{
      br = new BufferedReader(new InputStreamReader(new FileInputStream(args[2])));
      List<String> list = new ArrayList<String>();
      String str;
      while((str = br.readLine()) != null){
        list.add(str);
      }

      String[] ret = client.analyzeArray(list.toArray(new String[0]));
      for(String s : ret){
        System.out.println(s);
      }
    }finally{
      br.close();
    }
    factory.close();
  }
}

The initialization part is same as that of the example in Run a user program as a RaSC service. This program reads all input sentences in the specified file, and converts it to an array. Then it passes the array to analyzeArray. The order of sentences in the input is preserved in output.

For command to compile and run the program, refer to Run a user program as a RaSC service.

Cases where parallelization is effective

The following example shows the execution time when using the dependency and case structure analyzer KNP, which processes sentences with high computation load. In the example, the different ports are assigned to two RaSC services, which run KNP in parallel or in non-parallel. The following shows parallelization can drastically improve performance.

Note

In this example, the user programs and RaSC ran on the system with Intel Xeon X5675.

$ ./server.sh KNPService 19998 start # Start an RaSC service running KNP in parallel
$ time java -cp ./lib/*: SampleClient localhost 19998 ~/sentence_500.txt # Parallel
real    0m29.402s
user    0m0.566s
sys     0m0.045s
$ ./server.sh KNPService 19999 start # Start an RaSC service running KNP in non-parallel
$ time java -cp ./lib/*: SampleClient localhost 19999 ~/sentence_500.txt # Non-parallel
real   2m32.473s
user   0m0.901s
sys    0m0.167s
$ time cat INPUT_TXT | juman | knp > OUTPUT_TXT   # Directly run the user program (Non-parallel)
real    2m28.456s
user    2m17.557s
sys     0m1.011s

Generally, parallelization by RaSC is effective when processing one unit of input takes a long time. If a user program processes each input unit in very short period, parallelization cannot improve performance due to the overhead in parallel processing and network communication. As the morphological analyzer MeCab, for instance, completes processing per sentence in a very short time, the parallelization does not improve performance, or may even slow it down.

Refer to User programs tested on RaSC or How to connect multiple user programs through a pipe for information on using KNP as an RaSC service.