Posts by Chang Park

Stack based VM vs. Register based VM

Virtual Machine

A virtual machine (VM) is a high level abstraction on top of the native operating system, that emulates a physical machine. A virtual machine enables the same platform to run on multiple operating systems and hardware architectures. The Interpreters for Java and Python can be taken as examples, where the code is compiled into their VM specific bytecode. Here, we are talking about process virtual machines and not system virtual machines.

There are basically two main ways to implement a virtual machine

  • Stack based: ex. Java VM
  • Register based: ex. Dalvik VM

Stack based

The memory structure where the operands are stored is a stack data structure. Operations are carried out by popping data from the stack, processing them and pushing in back the results in LIFO (Last in First Out) fashion. In a stack based virtual machine, the operation of adding two numbers would usually be carried out in the following manner (where 20, 7, and ‘result’ are the operands):

stackadd

  1. POP 20
  2. POP 7
  3. ADD 20,7, result
  4. PUSH result

Because of the PUSH and POP operations, four lines of instructions is needed to carry out an addition operation. An advantage of the stack based model is that the operands are addressed implicitly by the stack pointer (SP in above image). In stack based VM’s, all the arithmetic and logic operations are carried out via Pushing and Popping the operands and results in the stack.

Pro:

  • Virtual machine does not need to know the operand addresses explicitly, as calling the stack pointer will give (Pop) the next operand.

 

 

Register based

In the register based implementation of a virtual machine, the data structure where the operands are stored is based on the registers of the CPU.

registeradd

  1. ADD R1, R2, R3;           # Add contents of R1 and R2, store result in R3

There is no POP or PUSH operations, so the instruction for adding is just one line. But unlike the stack, we need to explicitly mention the addresses of the operands as R1, R2, and R3.

Pro:

  • Overhead of pushing to and popping from a stack is non-existent, and instructions in a register based VM execute faster within the instruction dispatch loop.
  • it allows for some optimizations that cannot be done in the stack based approach. One such instance is when there are common sub expressions in the code, the register model can calculate it once and store the result in a register for future use when the sub expression comes up again, which reduces the cost of recalculating the expression.

Con:

  • Average register instruction is larger than an average stack instruction, as we need to specify the operand addresses explicitly.

 

Source: https://markfaction.wordpress.com/2012/07/15/stack-based-vs-register-based-virtual-machine-architecture-and-the-dalvik-vm/

Java Native Interface

jni1

Overview

JNI allows you to use native code when an application cannot be written entirely in the Java language. The following are typical situations where you might decide to use native code:

  • You want to implement time-critical code in a lower-level, faster programming language.
  • You have legacy code or code libraries that you want to access from Java programs.
  • You need platform-dependent features not supported in the standard Java class library.

Six steps to call C/C++ from Java code

The process of calling C or C ++ from Java programs consists of six steps. We’ll go over each step in depth in the sections that follow, but let’s start with a quick look at each one.

  1. Write the Java code. We’ll start by writing Java classes to perform three tasks: declare the native method we’ll be calling; load the shared library containing the native code; and call the native method.
  2. Compile the Java code. We must successfully compile the Java class or classes to bytecode before we can use them.
  3. Create the C/C++ header file. The C/C++ header file will declare the native function signature that we want to call. This header will then be used with the C/C++ function implementation (see Step 4) to create the shared library (see Step 5).
  4. Write the C/C++ code. This step consists of implementing the function in a C or C++ source code file. The C/C++ source file must include the header file we created in Step 3.
  5. Create the shared library file. We’ll create a shared library file from the C source code file we created in Step 4.
  6. Run the Java program. We’ll run the code and see if it works. We’ll also go over some tips for dealing with the more commonly occurring errors.

Step 1: Write the Java code

We’ll start by writing the Java source code file, which will declare the native method (or methods), load the shared library containing the native code, and actually call the native method.

Here’s our example Java source code file, called Sample1.java:

public class Sample1
{
   public native int intMethod(int n);
   public native boolean booleanMethod(boolean bool);
   public native String stringMethod(String text);
   public native int intArrayMethod(int[] intArray);

   public static void main(String[] args)
   {
      System.loadLibrary("Sample1");
      Sample1 sample = new Sample1();
      int square = sample.intMethod(5);
      boolean bool = sample.booleanMethod(true);
      String text = sample.stringMethod("JAVA");
      int sum = sample.intArrayMethod(
                  new int[]{1,1,2,3,5,8,13});
      System.out.println("intMethod: "+square);
      System.out.println("booleanMethod: "+bool);
      System.out.println("stringMethod: "+text);
      System.out.println("intArrayMethod: "+sum);
   }
}

What’s happening in this code?

First of all, note the use of the native keyword, which can be used only with methods. The native keyword tells the Java compiler that a method is implemented in native code outside of the Java class in which it is being declared. Native methods can only be declared in Java classes, not implemented, so a native method cannot have a body.

Now, let’s look at the code line by line:

  • In lines 3 through 6 we declare four native methods.
  • On line 10 we load the shared library file containing the implementation for these native methods. (We’ll create the shared library file when we come to Step 5.)
  • Finally, in lines 12 through 15 we call the native methods. Note that this operation is no different from the operation of calling non-native Java methods.

Note: Shared library files on UNIX-based platforms are usually prefixed with “lib”. In this case, line 10 would be System.loadLibrary("libSample1"); . Be sure to take notice of the shared library file name that you generate in Step 5: Create the shared library file.

Step 2: Compile the Java code

Next, we need to compile the Java code down to bytecode. One way to do this is to use the Java compiler, javac, which comes with the SDK. The command we use to compile our Java code to bytecode is:


$ javac Sample1.java

Step 3: Create the C/C++ header file

The third step is to create a C/C++ header file that defines native function signatures. One way to do this is to use the native method C stub generator tool, javah.exe, which comes with the SDK. This tool is designed to create a header file that defines C-style functions for each native method it finds in a Java source code file. The command to use here is:


$ javah Sample1

Results of running javah.exe on Sample1.java

Sample1.h, below, is the C/C++ header file generated by running the javah tool on our Java code:

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class Sample1 */

#ifndef _Included_Sample1
#define _Included_Sample1
#ifdef __cplusplus
extern "C"{
#endif

JNIEXPORT jint JNICALL Java_Sample1_intMethod
   (JNIEnv *, jobject, jint);

JNIEXPORT jboolean JNICALL Java_Sample1_booleanMethod
   (JNIEnv *, jobject, jboolean);


JNIEXPORT jstring JNICALL Java_Sample1_stringMethod
   (JNIEnv *, jobject, jstring);


JNIEXPORT jint JNICALL Java_Sample1_intArrayMethod
   (JNIEnv *, jobject, jint);
#ifdef __cplusplus
}
#endif
#endif

 

About the C/C++ header file

As you’ve probably noticed, the C/C++ function signatures in Sample1.h are quite different from the Javanative method declarations in Sample1.java. JNIEXPORT and JNICALL are compiler-dependent specifiers for export functions. The return types are C/C++ types that map to Java types.

The parameter lists of all these functions have a pointer to a JNIEnv and a jobject, in addition to normal parameters in the Java declaration. The pointer to JNIEnv is in fact a pointer to a table of function pointers. As we’ll see in Step 4, these functions provide the various faculties to manipulate Java data in C and C++.

The jobject parameter refers to the current object. Thus, if the C or C++ code needs to refer back to the Java side, this jobject acts as a reference, or pointer, back to the calling Java object. The function name itself is made by the “Java_” prefix, followed by the fully qualified class name, followed by an underscore and the method name.

Step 4: Write the C/C++ code

When it comes to writing the C/C++ function implementation, the important thing to keep in mind is that our signatures must be exactly like the function declarations from Sample1.h. We’ll look at the complete code for both a C implementation and a C++ implementation, then discuss the differences between the two.

The C function implementation

Here is Sample1.c, an implementation written in C:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
 #include "Sample1.h"
 #include <string.h>
 
 JNIEXPORT jint JNICALL Java_Sample1_intMethod
   (JNIEnv *env, jobject obj, jint num) {
    return num * num;
 }
 
 JNIEXPORT jboolean JNICALL Java_Sample1_booleanMethod
   (JNIEnv *env, jobject obj, jboolean boolean) {
   return !boolean;
 }
 
 JNIEXPORT jstring JNICALL Java_Sample1_stringMethod
   (JNIEnv *env, jobject obj, jstring string) {
     const char *str = (*env)->GetStringUTFChars(env, string, 0);
     char cap[128];
     strcpy(cap, str);
     (*env)->ReleaseStringUTFChars(env, string, str);
     return (*env)->NewStringUTF(env, strupr(cap));
 }
 
 JNIEXPORT jint JNICALL Java_Sample1_intArrayMethod
   (JNIEnv *env, jobject obj, jintArray array) {
     int i, sum = 0;
     jsize len = (*env)->GetArrayLength(env, array);
     jint *body = (*env)->GetIntArrayElements(env, array, 0);
     for (i=0; i<len; i++)
     {   sum += body[i];
     }
     (*env)->ReleaseIntArrayElements(env, array, body, 0);
     return sum;
 }
 
 void main(){}

The C++ function implementation

And here’s Sample1.cpp, the C++ implementation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
 #include "Sample1.h"
 #include <string.h>
 
 JNIEXPORT jint JNICALL Java_Sample1_intMethod
  (JNIEnv *env, jobject obj, jint num) {
   return num * num;
 }
 
 JNIEXPORT jboolean JNICALL Java_Sample1_booleanMethod
   (JNIEnv *env, jobject obj, jboolean boolean) {
   return !boolean;
 }
 
 JNIEXPORT jstring JNICALL Java_Sample1_stringMethod
   (JNIEnv *env, jobject obj, jstring string) {
     const char *str = env->GetStringUTFChars(string, 0);
     char cap[128];
     strcpy(cap, str);
     env->ReleaseStringUTFChars(string, str);
     return env->NewStringUTF(strupr(cap));
 }
 
 JNIEXPORT jint JNICALL Java_Sample1_intArrayMethod
   (JNIEnv *env, jobject obj, jintArray array) {
     int i, sum = 0;
     jsize len = env->GetArrayLength(array);
     jint *body = env->GetIntArrayElements(array, 0);
     for (i=0; i<len; i++)
     {   sum += body[i];
     }
     env->ReleaseIntArrayElements(array, body, 0);
     return sum;
 }
 
 void main(){}

C and C++ function implementations compared

The C and C++ code is nearly identical; the only difference is the method used to access JNI functions. In C, JNI function calls are prefixed with “(*env)->in order to de-reference the function pointer. In C++, theJNIEnv class has inline member functions that handle the function pointer lookup. This slight difference is illustrated below, where the two lines of code access the same function but the syntax is specialized for each language.

C syntax:       jsize len = (*env)->GetArrayLength(env,array);

C++ syntax:  jsize len = env->GetArrayLength(array);

Step 5: Create the shared library file

Next, we create a shared library file that contains the native code. Most C and C++ compilers can create shared library files in addition to machine code executables. The command you use to create the shared library file depends on the compiler you’re using. Below are the commands that will work on Windows and Solaris systems.

Windows:  cl -Ic:\jdk\include -Ic:\jdk\include\win32 -LD Sample1.c -FeSample1.dll

Solaris:       cc -G -I/usr/local/jdk/include -I/user/local/jdk/include/solaris Sample1.c -o Sample1.so

Step 6: Run the Java program

The last step is to run the Java program and make sure that the code works correctly. Because all Java code must be executed in a Java virtual machine, we need to use a Java runtime environment. One way to do this is to use the Java interpreter, java, which comes with the SDK. The command to use is:

1
java Sample1

When we run the Sample1.class program, we should get the following result:

$java Sample1
intMethod: 25
booleanMethod: false
stringMethod: JAVA
intArrayMethod: 33

Troubleshooting

You can run into many problems when using JNI to access native code from Java programs. The three most common errors you’ll encounter are:

  • A dynamic link cannot be found. This results in the error message:java.lang.UnsatisfiedLinkError. This usually means that either the shared library cannot be found, or a specific native method inside the shared library cannot be found.
  • The shared library file cannot be found. When you load the library file using the file name with theSystem.loadLibrary(String libname) method, make sure that the file name is spelled correctly and that you do not specify the extension. Also, make sure that the library file is accessible to the JVM by ensuring that the library file’s location is in the classpath.
  • A method with the specified signature cannot be found. Make sure that your C/C++ function implementation has a signature that is identical to the function signature in the header file.

Conclusion

Calling C or C++ native code from Java, while not trivial, is a well-integrated function in the Java platform. Although JNI supports both C and C++, the C++ interface is somewhat cleaner and is generally preferred over the C interface.As you have seen, calling C or C++ native code requires that you give your functions special names and create a shared library file. When taking advantage of existing code libraries, it is generally not advisable to change the code. To avoid this, it is common to create proxy code, or a proxy class in the case of C++, that has the specially named functions required by JNI. These functions, then, can call the underlying library functions, whose signatures and implementations remain unchanged.

tee command

tee

Tee is a very useful command when using with pipe. If you use, ls -l > file.txt, you will get same file.txt which contains all lists of files, directories, and information for those, but you will not see the result on the screen. However, tee command makes files with output, and it also passes the output to next command.

less command on the picture is used to display information to the terminal one page at a time. less is faster than other command because it doesn’t need to read entire file before starting.

Let’s say that you are about to put some debugging results in text file for huge program or multiple programs. Are you going to just sit there doing nothing until debugging and writing process done or watch which errors occur while the process is running?

  • This command makes file and also does word counting of the output.

$ ls -l | tee result.txt | wc

 

  • This command makes file and also prints out the output, because there is no command after.

$ ls -l | tee result.txt

File permission

If you type ls -l, you can see file permission like below.

 
drwxr-xr-x 3 chang users 512  Jan 21 22:49 101 
-rw-r--r-- 1 chang users 2312 Jan 15 21:54 index.html 
drwxr-xr-x 2 chang users 512  Jan 15 16:54 javascript 
-rw-r--r-- 1 chang users 2859 Jan 12 16:54 work.html 

permission-structure

  • First letter indicates if it is a Directory.
  • Next each group of three characters indicate file permission
    • First three are permissions for owner, second three are permissions for the group, and last three are permissions for everybody. Every “-” means  no permission.
      • r: read
      • w: write
      • x: execute

 

 

[ ssh ] Public key vs. Private key

How they work

keys

If you generate key with the command sshkeygen -t rsa, you will get two types of key.

Public key is for encrypting and private key is for decrypting. Which one do you think more important? Yes, of course private key. You can provide your public key to others to communicate. So, other people will use the public key to encrypt the file to send.

However, you are the only one who have a private key. That means you are the only one person who can decrypt the files. This is how the public key and private key work. 

  • Example
    • For Git, you need to copy your public key and paste it on the git site before use.
    • So if you pull repositories from Git, the server encrypts the repositories with your public key before sending.

Dead code

dead

Definition

Dead code can never be executed (unreachable code), and it only affects dead variables (written, but never read again), that is, irrelevant to the program. It needs to be deleted for several reasons.

  • It shrinks program size
  • It allows the running program to avoid executing irrelevant operations, which reduces its running time.
  • It can also enable further optimizations by simplifying program structure. (avoid confusion)

 

Method Inlining

Definition

Inlining is an optimization performed by the Java Just-In-Time compiler.

public int addPlusOne(int a, int b){
   return a + b + 1;
}

public void testAddPlusOne(){
   int v1 = addPlusOne(2, 5);
   int v2 = addPlusOne(7,13);
}

The compiler may decide to replace your function call with the body of the function, so the result would effectively look like this

public void testAddPlusOne(){
   int v1 = 2 + 5 + 1;
   int v2 = 7 + 13 + 1;
}
  • The compiler does this to save the overhead of actually making a function call, which would involve pushing each parameter on to the stack.
  • This can be done for non-virtual functions because you won’t know which code it contains if virtual function is overridden by others until runtime. Since java methods are virtual by default. You can make methods static or final to make non-virtual.

Pipe & Redirection

Bash-script-21

Difference between Pipe & Redirection

Pipe is used to pass output to another Program or Utility.

  • thing1 | thing2
    • thing1’s output s will be passed to things2 Program or Utility.

Redirect is used to pass output to either a File or Stream.

  • thing1 > thing2
    • thing1’s outputs will be placed in a file called thing2.

 

Pipe

With Pipe, the standard output of one command is fed into the standard input of another.

$ ls | grep "java"

It will find all the file names from ls that contain “java” string.

 

Redirection

Standard output directs its contents to the display. To redirect standard output to a file, the “>” character is used.

$ ls > file_list.txt

In this example, all the file names from ls command will be written in an file named file_list.txt. Since the output of ls was redirected to the file, no results on the display. Each time the command above is repeated, file_list.txt is overwritten (from the beginning) with the output of the command ls. If you want the new results to be appended to the file instead, use “>>“.

$ ls >> file_list.txt

 

Standard Input gets its contents from the keyboard, but like standard output, it can be redirected. To redirect standard input from a file instead of the keyboard, the “<” character is used.

$ sort < file_list.txt

Sort command print the contents of file_list.txt. We could redirect standard output to another file.

$ sort < file_list.txt > sorted_file_list.txt

As you can see, a command can have both its input and output redirected. The redirection operators (the “<” and “>”) must appear after the other options and arguments in the command.

 

Filters

  • sort: Sorts standard input then outputs the sorted result on standard output.
  • uniq: Given a sorted stream of data from standard input, it removes duplicate lines of data (i.e., it makes sure that every line is unique).
  • grep: Examines each line of data it receives from standard input and outputs every line that contains a specified pattern of characters.
  • fmt: Reads text from standard input, then outputs formatted text on standard output.
  • pr: Takes text input from standard input and splits the data into pages with page breaks, headers and footers in preparation for printing.
  • head: Outputs the first few lines of its input. Useful for getting the header of a file.
  • tail: Outputs the last few lines of its input. Useful for things like getting the most recent entries from a log file.
  • tr: Translates characters. Can be used to perform tasks such as upper/lowercase conversions or changing line termination characters from one type to another (for example, converting DOS text files into Unix style text files).
  • sed: Stream editor. Can perform more sophisticated text translations than tr.
  • awk: An entire programming language designed for constructing filters. Extremely powerful.

 

Soot

soot-logo

What is Soot

A framework for analyzing and transforming Java and Android Applications.

  • Soot can process and produce code from the following sources.
    • Java bytecode
    • Android bytecode
    • Jimple
    • Jasmin
  • This is a powerful framework that many research group use.
  • For example, Soot can transform application file(apk) to Jimple code which developers can see actual code.
  • To run Soot, you will also need the projects Jasmin and Heros.