ANDROID INSTRUMENTATION

ABSTRACT

Android codebase has been changing extensively over the last few years to support a wide range of mobile devices and attractive features. Although it is an open source project, due to its dynamic nature and continuous growth, there is an ever-increasing need for a tool that facilitates developers to instrument the code at the platform level and aid them in analysis of the source code by providing handles into the platform. The objective of this project is to develop a tool that provides the developers with the ability to instrument the Android platform using Aspect-Oriented Programming.

The Instrumentation tool enables its users to inject custom Java-based code into Android Open Source Project (AOSP) which can be very useful in many situations such as providing the developers with better insights of Android source code, understanding method and class usage, monitoring the behavior of applications and checking any security violations by Android apps. Furthermore, the tool can also be extended and customized to meet the individual needs.

I certify that the Abstract is a correct representation of the content of this thesis.

Chair, Thesis Committee       Date

iv

PREFACE AND/OR ACKNOWLEDGEMENTS

This research project was supported and supervised by Dr. Arno Puder, whom I would like to gratefully and sincerely thank for his guidance, encouragement, and patience during my study at San Francisco State University. Also, I would like to thank my peer reviewers Syed Omer Khureshi, Sai Krishna Undurthi and Anshul Vyas from San Francisco State University.

I would like also to thank Dr. William Hsu for reviewing my report in his capacity as the second member of the committee.

v

Table of Contents

1. Introduction

1.1 Motivation

1.2 Objective

1.3 Contribution

1.4 Organization of Document

2. Use Cases

3. Background & Related work

3.1 Android Open Source Project(AOSP)

3.2 Android Architecture

3.2 Android Build System

3.2.1 Configuration

3.2.2 Function Definitions

3.2.3 Cleaning

3.2.4 Module Build Templates

3.2.5 Output

3.2.6 Build Recipes

3.2.7 Compiling with Jack and Jill

3.3 Aspect Oriented Programming

3.3.1 AspectJ Library

3.4 Byte Code Engineering Library (BCEL)

vi

4. Implementation Process

4.1 Build Flow Modification

4.2 Weaving Custom Code

4.2.1 Using Bytecode Manipulation

4.2.2 Using Aspect Oriented Programming

5. Demonstration

5.1 Launcher Process

5.2 Activity Manager

5.3 Zygote

5.4 Application Process

5.4.1 Android Platform Variations

5.4.1.1 Gingerbread (2.3.7)…………………………………………..

5.4.1.2 Gingerbread (2.3.7) with Android support library

5.4.1.3 Nougat (7.1.1)……………………………………………….

5.4.1.4 Nougat (7.1.1) with support library……………………………………

5.4.1.5 Comparison of the results of analysis on Android platform variations………………..

6. Summary and Conclusion

6.1 Project Summary

6.2 Limitations

6.3 Conclusion

7. References

8. Appendix

Steps to instrument and build Android

vii

List of Figures

 

 

 

Figure 1: Working of the Instrumentation tool

Figure 2: Android architecture

Figure 3: Working of Zygote process

Figure 4: Android build architecture

Figure 5: Jack toolchain structure

Figure 6: Jack library format

Figure 7: Android Build flow with Jack and Jill toolchain.

Figure 8: Android build process

Figure 9: Android build process with AspectJ instrumentation

Figure 10: Application startup process

Figure 11: Call-stack graph of the APIs invoked by launcher process

Figure 12: Call-stack graph of the APIs invoked by Activity Manager

Figure 13: Call-stack graph of the APIs called by Zygote Process

Figure 14: Call-stack graph of the APIs invoked during application startup on Gingerbread

Figure 15: Comparison of the results of analysis on different versions of Android

 

 

 

 

 

 

 

 

 

viii

List of Tables

 

 

Table 1: Directory contents of Android Open Source Project(AOSP)

Table 2: Set of values that can be provided for TARGET_PRODUCT in the Android build

Table 3: List of functions added by build/envsetup.sh script

Table 4: Configuration information set up by lunch for generic-eng combo

Table 5: List of module build templates and their corresponding .mk files in AOSP

Table 6: Comparison of custom code weaving approaches: Bytecode manipulation and

Aspect-oriented programming

Table 7: List of processes observed at the application startup

Table 8: Metrics used in the analysis of a call-stack graph

Table 9: Graph analysis of the call-stack graph of application process on Gingerbread (2.3.7)

Table 10: Graph analysis of call-stack graph of the application process on Gingerbread (2.3.7) with Android support library

Table 11: Graph analysis of call-stack graph of the application process on Nougat (7.1.1)

Table 12: Graph analysis of the application startup sequence on Nougat (7.1.1) with support library

Table 13: Percentage comparison of the results of analysis on different versions of Android

 

 

 

 

 

 

 

 

 

 

 

ix

List of Code snippets

 

Code snippet 1: Makefile of AOSP

Code snippet 2: List of choices offered by lunch command on Gingerbread

Code snippet 3: Part of the code in envseup.sh which adds lunch combinations…………..

Code snippet 4: vendorscript.sh script in Gingerbread 2.3.7………………..

Code snippet 5: Example of AspectJ around advice

Code snippet 6: Changes made in build/core/definitions.mk file to support

custom weaving……………………………………………………

Code snippet 7: Program to insert log statements at the beginning of every method in a class using BCEL library

Code snippet 8: Program to insert log statements at the beginning of every method in a class using AspectJ library

Code snippet 9: Content of post-compile.sh script

Code snippet 10: AspectJ program to add enter/exit logs in every method of AOSP

x

CHAPTER 1

Introduction

This chapter outlines the objective and the motivation behind this project and explains the organization of this document.

1.1  Motivation

The idea for this project first formed when we learned about an interesting situation involving a law suit targeting a major Android app on the PlayStore. The allegation was that the app was accessing private information in situations where it didn’t require such access and was misusing the information collected. This called for the analysis of the app to verify if such claims were valid. This was not an easy task, more importantly, many other situations arise where we are left wondering what sort of information an app might be privy to. Since the access to source is not available, it was very difficult, almost impossible, to accurately ascertain if the app accesses the private information. Furthermore, there are other major hurdles such as,

1. No Access to handles in the platform for APIs accessing private information

Android framework provides multiple ways to access the same piece of information, making it difficult to determine if the information was accessed. There was a need to have a handle in the platform to detect when such APIs are invoked to make the illegal accesses to private information more visible for analysis.

2. App level visibility is insufficient

Although the current Android permission model allows a user to control what information/resources an app can access, it does not enable the users to control how frequent the access is or what the app does with the private information. This level of control requires platform level visibility.

3. Lack of access to source code

Access to Android application source code is not always available. In fact, often, companies do not make source code open. This leaves us with the only option to perform analysis on the bytecode.

4. Lack of existing tool

Although there are tools available in the market that perform analysis at the application level, they do not provide flexibility and visibility needed for a conclusive and definitive answer to the question of whether the app is misusing its permissions. No tool is available for analyzing the APIs at the platform level.

To summarize, there was a need for a tool that could resolve the mentioned hurdles above and since we could not find it, we started to develop our own tool which could instrument the Android platform with which it would be possible to insert handles into the platform.

1.2 Objective

The aim of the project is to provide an end-to-end tool that enables users to inject any custom Java code based on user specified constraints into the Android platform using Aspect Oriented Programming. Figure 1 explains the objective of the project.

Figure 1: Working of the Instrumentation tool

The end user can write the custom Java code that they want to execute every time a condition is met in the base code which is the Android Open Source Project (AOSP). The user can then use the Instrumentation tool to inject the custom code into the base code. The Instrumentation tool weaves the custom code into the base code through the build process and generates a custom emulator i.e. a final executable image of the AOSP. The custom emulator can be used by the end users to execute the APKs and see the output of their custom code every time the condition specified by them is met.

1.3 Contribution

1. Custom code injection

The Instrumentation tool provides an ability to inject a custom code into Android Open Source Project that can be very useful in many ways such as understanding APIs and class usage in AOSP, profiling applications and monitoring behavior of applications. There is no need to generate an emulator image every time. Preconfigured emulator images can be reused to analyze multiple apps.

2. Support for multiple versions of Android

The Instrumentation tool supports the different versions of Android. It has been specifically developed for all the versions from Gingerbread (2.3.3) to the latest version Nougat (7.1.1).

3. Consolidated documentation to understand AOSP and Android Build system.

Though there are several documents available that provide information about Android Open Source Project and Android build system, the information is fragmented. The document consolidates all the relevant information in one place.

4. Environment setup guide

The appendix attached at the end of the document provides step by step instructions on how to download, build and generate a custom emulator with the Instrumentation tool.

5. Framework for other projects

The Instrumentation tool provides a necessary framework for other projects like Android best practices verification tool by Sai Undurthi, security inspection tool by Anshul Vyas and visualization tool by Syed Omer Khureshi.

6. A detailed analysis of application startup process

 

The demonstration of the Instrumentation tool provides an insight into the sequence of actions that take place when an app is launched on the Android OS. An analysis of the application startup process is carried out on different versions of Android with different configurations, which helps in understanding the phenomenal growth of Android over the last few years and impact of Android support library on the application startup process.

 

1.4 Organization of Document

Chapter 2 outlines the use cases of the tool. Chapter 3 discusses the relevant background knowledge, including Android Open Source Project (AOSP), Android build architecture and Aspect-oriented programming. Chapter 4 gives a detailed walkthrough of our implementation process to support custom weaving into the AOSP. Chapter 5 provides a demonstration of the tool using a sample app. Finally, Chapter 6 summarizes the project. The appendix provided at the end explains how to get the Android sources from the Android website and how to compile them with custom aspects to generate a functional emulator image.

CHAPTER 2

Use Cases

 

In this chapter, we present four use-cases of the Instrumentation tool which highlight its applications.

1. Understanding Android APIs and features

Sydney’s team has a published app on Android PlayStore which is designed to detect earthquakes by using mobile sensors. The app uses a background service to record the sensor data constantly, process it and upload to a server. Starting from Android 6.0 (API level 23), Android has introduced a new feature Standby and App Doze to reduce battery consumption. It does so by deferring background services and network activity for apps when the device is unused for long periods of time. To understand how the Doze and App Standby affects the functionality of the app, she utilizes the functionality of the Instrumentation tool by writing custom aspects to add logs in the relevant Java classes and builds an instrumented emulator image. She then installs her app on the emulator and tests it with a set of test cases. After analyzing the logs generated by the tool, she is able to check how the functionality of the app is affected. She can then modify her app functionality accordingly.

2. Profiling Android application

Jen is a part time Android app developer who developed a map application on the PlayStore. She has a crash reporting tool ACRA setup in her app. When she checks the recent crash report, she sees a few crashes due to Application Not Responding(ANR) error. To find out the root cause of the issue, she proceeds to use the Instrumentation tool. With the help of the logs generated by the tool, she realizes that her app is spending more than normal time in the Android callbacks, triggering the ANR error. This makes it easy for her to fix the bug and speed up the execution time of bottlenecks in her software code.

3. Ensuring if an application is following best practices.

Kevin is a team lead responsible for Android game development. He has been receiving feedback from the app users about excessive battery drainage when using the app. Having heard about the Instrumentation tool, he writes a set of custom aspects to check if the app follows the best practices suggested by Android. With this analysis, Kevin discovers that the app is violating some of the best practices. He incorporates the use of the tool in his team’s standard testing protocol. The issues such as not unregistering sensor listener and broadcast receivers on onPause(), not stopping background services are detected by the tool early in the testing phase. His team is able to fix the code to ensure the best practices are followed. After a couple of months, the app has got increased number of installations and an incredible rating on the PlayStore.

 

4. Detecting privacy breach in an app

 

Raymond is an Android enthusiast with an extremely deep understanding of the Android framework. He is contacted by a Law firm that is working on a client’s complaint about a potential privacy breach by a leading social networking app and request Raymond, in his capacity as an Android expert, to ascertain the validity of this claim by providing him access to the app’s APK file. Since he does not have access to the source code of the app, it becomes very difficult to understand the working and to examine the behavior of the app. Being aware of the Instrumentation tool, Raymond writes custom aspects on potential APIs that can access private information like contacts, location, etc. and runs the app on an instrumented VM instance. By analyzing the generated logs, he is able to understand the flow of data in the app. He provides his findings to the Law firm.

 

As can be observed from the presented use cases, there is a need to instrument the Android platform when the source code of Android apps is not accessible, or the application level access is insufficient. In such scenarios, the Instrumentation tool can be used with minimal efforts.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

CHAPTER 3

Background & Related work

 

This chapter introduces the relevant background knowledge for our project by providing a brief explanation of the key concepts related to the project. Section 3.1 gives an overview of the Android Open Source Project (AOSP), whereas section 3.2 explains the Android architecture. Section 3.2 starts with the introduction of Android build system and then digs into Android’s internals such as Android build architecture, build configuration parameters, module template, reusable functions and the newly introduced build toolchain ‘Jack and Jill’. Section 3.3 discusses Aspect-Oriented Programming paradigm. Section 3.4 covers a high-level overview of Apache’s Byte Code Engineering Library (BCEL) used for bytecode manipulation.

3.1 Android Open Source Project(AOSP)

 

Android is an open source software project which works on a range of devices. The purpose of Android is to develop an open source platform available for developers and to create a product that enhances the mobile experience for users [5]. AOSP is available for download, customization, experimentation and porting.

Android source can be downloaded and built to get custom Android OS running on a device or an emulator.  Step by step instructions on how to download and build Android can be found in Appendix. AOSP is a fairly large project consisting of more than 14,000 directories and 100,000 files in Gingerbread (2.3.7) and more than 1340000 files and 132,000 directories in Nougat (7.1.1).

Table 1 provides an overview of the important directories and their contents in the AOSP project.

Directory	Content
Abi	Minimal C++ Run-Time Type Information support
bionic	Android’s custom C library
bootable	OTA, recovery mechanism and reference bootloader
build	Build system
cts	Compatibility Test Suite
Dalvik	Dalvik VM
development	Development Tools
device	Device specific files and components
docs	Documentation of the source code
external	External projects imported into the AOSP
frameworks	Core components such as system services
hardware	HAL and hardware support libraries
libcore	Apache Harmony Java source
ndk	Native Development Kit
pdk	Platform Development Kit
prebuilt	Prebuilt binaries, including toolchains
sdk	Software Development Kit
system	“Embedded Linux” platform that houses Android
tools	Various IDE tools

Table 1: Directory contents of Android Open Source Project(AOSP)

 

‘prebuilt’ and ‘external’ are the two major directories in the AOSP tree. They account for close to 70% of its size. Both the directories mostly have content from other open source projects such as kernel images, GNU toolchains, common libraries such as OpenSSL and WebKit. ‘libcore’ is also a part of another open source project, Apache Harmony [1]. The key components of Android reside in frameworks/. Inside frameworks, System services can be found in frameworks/base/services and frameworks/base/media. frameworks/base /core contains core components including Runtime and Zygote, the system key elements which are explained in the following section. Native daemons can be found in frameworks/base/cmds.

3.2 Android Architecture

Figure 2: Android architecture

This section describes the key components of Android exhibited in Figure 2. in the order in which they are loaded during system startup.

Bootloader

When the Android system is started, a bootloader is the first program executed by CPU. The bootloader initializes the RAM, loads the kernel and RAM disk, and jumps to the kernel.

Kernel

The main goal of Android Kernel is to load the necessary things for CPU.  It initializes several subsystems and invokes the ‘init’ functions of all built-in drivers. It also mounts the root filesystem and fires up the init process of Android.

 Init

The init process executes instructions stored in init.rc file. The init.rc file contains instructions to create mount points, mount filesystems, start native daemons and set up environment variables such as system-path.

Android Runtime

Init process sends an app_process command to Android Runtime. On receiving the command, the Runtime kicks off the first Dalvik VM, which invokes Zygote’s main() method to launch the Zygote process.

Zygote

Zygote is a daemon process responsible for starting an app as its child process. During system startup, the Zygote preloads all necessary Java classes and resources, starts System Server process and opens a socket /dev/socket/zygote to listen to incoming requests for launching new applications [9].

When the Zygote receives a request to launch a new app through the socket, it takes advantage of the fork() system call to create a new process. With the fork() call, it creates a replica of itself. Therefore, a new Dalvik VM is preloaded with all the necessary classes and resources that any app might need. This makes the process of creating a VM and loading of resources more efficient. With the Copy On Write (COW) technique implemented by the Linux system, the memory pages are shared between the parent and the child processes. When one of the processes attempts to modify the shared memory, the kernel intercepts the call and makes a copy of the shared pages. In the case of Android, the pages are not writable. This means that all the process forked from the Zygote use the same copy of the system classes and resources. In addition to being efficient, this approach also saves physical memory space on a device; regardless of how many applications have started, the increase in memory usage will be a lot smaller [1]. The above process is summarized in Figure 3 provided below.

Figure 3: Working of Zygote process

System Server

System Server is started by the Zygote during system startup and stays as a background process.  It hosts the majority of the system services that run on Android within a single process. Some system services are written in Java whereas rest of them are written in C/C++. It also includes some native code access through JNI to allow some of the Java-based services to interface to Android’s lower layers.

Activity Manager

Activity Manager is one of the services hosted by the System server. It handles the starting of new components, such as Activities and Services, fetching of Content Providers and intent broadcasting. It is also involved in the maintenance of out of memory adjustments used by the in-kernel low- memory handler, permissions, task management, etc.

Launcher

Launcher app is started by the Activity Manager by sending an intent of type Intent.CATEGORY_HOME. The launcher is responsible for displaying the home screen which is familiar to the Android users.

3.2 Android Build System

 

Android build system resembles make-based build system, with a few notable differences. The Android build system does not count on recursive makefiles, unlike the make-based build system. Instead, it makes use of a special makefile, Android.mk. The Android.mk file defines how a local module is built. A module is any part of the AOSP such as a binary, an app package or a library that needs to be built. The Android build system invokes a script that traverses all the subdirectories till it locates an Android.mk file. After finding an Android.mk, it stops. It does not explore the subdirectories beneath that file’s location, unless explicitly specified in the Android.mk. [14].

Another difference between the make and the Android build system is in the way the Android build system is configured. Android depends on a set of variables that are are defined statically in a buildspec.mk file or either set dynamically by envsetup.sh and lunch scripts. Also, the level of configurability allowed by the Android’s build system is limited. Although the properties of the target can be specified, there is no way to enable or disable most of the features. For instance, it is not possible to disable power management support or Location Service.

Also, the Android build system does not store an intermediate output within the same location as the AOSP source files. Instead, the build system generates the intermediate output as well as final output in a new directory out/. Therefore, removing the out/ directory also removes everything that was generated during build. In other words, ‘make clean’ is the same thing as rm -rf out/.

The root directory of AOSP includes a single Make File. That file is mostly empty; its main use is to include the entry point for the Android’s build system as can be seen in Code snippet 1.

Code snippet 1: Makefile of AOSP

The build/core/main.mk file is the entry point to the build system. The build system pulls everything into a single makefile. Therefore, each .mk file in the end becomes a part of a single huge makefile. Therefore, this single makefile contains the rules for building all the modules in the system. Figure 4 presents the components of the build system. The components are explained in detail in the subsequent sections.

Figure 4: Android build architecture

3.2.1 Configuration

The build configuration is specified in config.mk file. The build system pulls in the build configuration by including config.mk. The config.mk file defines the following environment variables. [14].

1. TARGET_PRODUCT:

 

Android flavor to be built. The set of values that can be provided to the TARGET_PRODUCT variable, includes the following:

Value	Description
generic	the most basic build of the AOSP parts
full	With most apps and the major locales enabled
full_crespo	Same as full but for Crespo (Samsung Nexus S)
full_grouper	Same as full but for Grouper (Asus Nexus 7)
Sim	Android simulator
sdk	The SDK; includes a vast number of locales

 

Table 2: Set of values that can be provided for TARGET_PRODUCT in the Android build

2. TARGET_BUILD_VARIANT:

 

Dictates which modules to install. Each module sets a LOCAL_MODULE_TAGS variable in its Android.mk from the list: user, debug, eng, tests, optional, or samples. With the selection of the variant, module subsets to be included can be specified.

3. TARGET_BUILD_TYPE:

Decides on whether to use release or debug build type.

4. ARGET_TOOLS_PREFIX:

By default, the build system uses one of the cross-development toolchains in the prebuilt/ directory. To use a different toolchain, the value of the TARGET_TOOLS_PREFIX variable should point to the location of the toolchain.

5. OUT_DIR:

By default, the build system generates the build output into the out/ directory. This variable is used to provide a different output directory.

6. BUILD_ENV_SEQUENCE_NUMBER:

If the default template build/buildspec.mk.default is used to create buildspec.mk file, this value is set correctly. However, if the buildspec.mk created with an older AOSP release and is used in the newer AOSP release having important build system changes, this variable acts as a safety net. It causes the build system to update the users that buildspec.mk file is not compatible with the build system.

envsetup.sh

envsetup.sh sets up the build environment for Android. Primarily, it defines a chain of shell commands useful to many AOSP jobs. Invoking build/envsetup.sh from a shell adds the functions described in Table 3 to the environment.

Variable	Description
croot	Changes directory to the top of the tree
m	Makes from the top of the tree
mm	Builds all the modules in the current directory
mmm	Builds all the modules in the supplied directories
cgrep	Greps on all local C/C++ files
jgrep	Greps on all local Java files
resgrep	Greps on all local res/*.xml files
godir	Go to the directory containing a file

 

Table 3: List of functions added by build/envsetup.sh script

‘m’ and ‘mm’ are quite useful commands. ‘m’ command allows users to build from top level regardless of the current path, whereas ‘mm’ builds the modules located in the current directory. For example, if a modification is made to the Launcher and the current path is packages/apps/Launcher2, then the module can be rebuilt by with mm instead of going back to the topmost level, and typing make. Since mm does not rebuild the entire tree, it does not rebuild AOSP images even though a dependent module has modified. mm can be still helpful to tryout local changes and to check if they crash the build [14].

lunch

 

lunch command is defined by envsetup.sh. When lunch is executed without any arguments, it displays a list of alternatives. For example, on Gingerbread, the following list appears.

Code snippet 2: List of choices offered by lunch command on Gingerbread

Basically, the menu asks the users to select a combination of the TARGET_PRODUCT and the TARGET_BUILD_VARIANT. The menu gives the default combination, but the other options can be chosen as parameters on the command line. These choices are not generated dynamically based on content of AOSP. They are individually added using the add_lunch_combo() function defined in the envsetup.sh [14]. For instance, in Gingerbread 2.3.7, envsetup.sh includes generic-eng and simulator as illustrated in Code snippet 3.

Code snippet 3: Part of the code in envseup.sh which adds lunch combinations

lunch also offers a way to add vendor-specific scripts. Here is how it is done in 2.3.7 Gingerbread:

Code snippet 4: vendorscript.sh script in Gingerbread 2.3.7

Table 4 describes the required configuration information set up by lunch for default ‘generic-eng’ combo.

Variable	Value
PATH	$ANDROID_JAVA_TOOLCHAIN:$PATH:$ANDROID_BUILD_PATHS
ANDROID_EABI_TOOLCHAIN	aosp-root/prebuilt/linux-x86/toolchain/arm- eabi-4.4.3/bin
ANDROID_QTOOLS	aosp-root/development/emulator/qtools
ANDROID_BUILD_PATHS	aosp-root/out/host/linux-x86:$ANDROID_TOOLCHAIN:$AN DROID_QTOOLS:$ANDROID_TOOLCHAIN:$ANDROID_EABI_TOOLCHAIN
ANDROID_BUILD_TOP	aosp_root
ANDROID_JAVA_TOOLCHAIN	$JAVA_HOME/bin
ANDROID_PRODUCT_OUT	Aosp- root/out/target/product/generic
BUILD_ENV_SEQUENCE_NUMBER	10
OPROFILE_EVENTS_DIR	aosp-root/prebuilt/linux-x86/oprofile
TARGET_BUILD_TYPE	release
TARGET_PRODUCT	Generic
TARGET_BUILD_VARIANT	eng
TARGET_BUILD_APPS	empty
TARGET_SIMULATOR	false
PROMPT_COMMAND	”33]0;[${TARGET_PRODUCT}-${TARGET_BUILD_VARIANT}] $ {USER}@${HOSTNAME}: ${PWD}07”

Table 4: Configuration information set up by lunch for generic-eng combo

Using ccache

ccache stands for Compiler Cache. It is a mechanism used to accelerate the build process. It caches the object files generated by a compiler based on the preprocessor’s output. Therefore, if two separate builds generate an identical preprocessor’s output, with the use of ccache, the second build will not use a compiler to build the file; the cached object file will be copied to the destination folder.

To enable the use of ccache in the build, the USE_CCACHE variable should be set to 1 before starting the build, as shown here.

$ export USE_CCACHE=1

When run for the first time, there will not be any gain in the acceleration, since the cache will be empty at that point. But whenever the build is run afterwards, the cache will assist in accelerating the build process. The downside of ccache is that it is for C/C++ files only [14].

3.2.2 Function Definitions

Since the build system is enormous, to make reuse of code, the build system defines many functions in the definitions.mk file. Various functions specified in the definitions.mk perform common operations, such as transformation (transform-c-to-o, transform-java-to-classes.jar), copying (copy-file-to-target), file lookup (all-makefiles-under, all-c-files-under) and utility (my-dir). The build system components as well as in modules’ Android.mk files make use of these functions [1].

3.2.3 Cleaning

‘make clean’ is same as removing the out/ directory. The main.mk defines the clean target. Another cleanup target installclean is defined in cleanbuild.mk, and is automatically invoked whenever the values of the variables TARGET_PRODUCT, TARGET_BUILD_VARIANT or PRODUCT_LOCALES are changed.

3.2.4 Module Build Templates

 

In AOSP,module build recipes are independent of the build system’s internals. The purpose of the build templates is to facilitate the module authors to get their modules built. Each template is adapted for a particular type of module. The template’s behavior and output can be controlled by a set of documented variables. The templates are located in build/core, where also the rest of the build system is located. To get access to the templates, Android.mk makes the use of the include directive. For instance,

include $(BUILD_PACKAGE)

As shown above, Android.mk files do not add the .mk templates by name. Instead, they include a variable which is set to the corresponding .mk file. Because of the build templates, the Android.mk files are very lightweight [1].

Table 5 lists the module build templates supplied by the Android build system.

Variable	Template	What it builds
BUILD_HOST_SHARED_LIBRARY	host_shared_library.mk	Host shared libraries
BUILD_HOST_STATIC_LIBRARY	host_static_library.mk	Host static libraries
BUILD_RAW_STATIC_LIBRARY	raw_static_library.mk	Target static libraries that run on bare metal
BUILD_PREBUILT	prebuilt.mk	Copies prebuilt target files
BUILD_HOST_PREBUILT	host_prebuilt.mk	Copies prebuilt host files
BUILD_MULTI_PREBUILT	multi_prebuilt.mk	Copies prebuilt like Java libraries or executables
BUILD_PACKAGE	package.mk	Built-in AOSP apps
BUILD_KEY_CHAR_MAP	key_char_map.mk	Device character maps

Table 5: List of module build templates and their corresponding .mk files in AOSP

3.2.5 Output

out/ contains two directories, signifying its operating modes: host/ and target/.

The build process generates an output in three stages:

1. The module sources generate intermediates. The format and location of the intermediates depend on the type of module sources. For example, .o files for C/C++ code, .jar files for Java code.
2. By using the intermediates, the build system creates binaries and packages. For example, linking .o files into an actual binary.
3. Finally, the binaries and packages are assembled to create the final output. For instance, binaries are copied into /system directory which are used on the actual device.

3.2.6 Build Recipes

$ make droid

The default build target is droid which is defined in main.mk. So, running ‘make’ is equivalent to the above command.

$ make showcommands

When the AOSP is built, it does not print out the commands it is running. Instead, it shows only a summary of each step. To see the commands the build is executing, the showcommands target can be added to the command line.

$ make –j4

With a -jN argument, parallel tasks can be started to accelerate the build process. The common value of N is usually between 1 and 2 times the number of hardware threads on the system utilized for the build.

$ make sdk

It is possible to build a custom SDK using the AOSP. To do so, it is required to select a series of commands, given below:

$ . build/envsetup.sh

$ lunch sdk-eng

$ make sdk

$ make update-api

The build systems have safeguards to detect if the AOSP’s core API are modified. If the modifications are detected, the build fails by default. To continue the build, the update-api target needs to be added.

$ make Launcher2

Building individual modules is possible. The above command, for example, shows how to build Launcher2 module.

$ make clean-Launcher2

The command allows the users to clean modules individually.

$ make Launcher2 snod

To include an updated module in the system image, the snod target can be added to the command line. It forces the build system to regenerate the system image. [6].

3.2.7 Compiling with Jack and Jill

Jack is a new Android toolchain introduced to reduce the Java code compilation time. It compiles Java source files directly into Android Dex bytecodes, replacing the previous Android toolchain consisting of multiple tools, such as javac, ProGuard, jarjar and dx [13]. The Jack toolchain offers following advantages:

• Completely open source
• Speeds compilation
• Handles shrinking, obfuscation, repackaging, and multidex.

Figure 5: Jack toolchain structure

As can be seen from Figure 5, Jack takes Java source and Jack libraries as an input, performs the operations such as shrinking, obfuscation, repackaging and multidexing, and finally, produces Dex files as output. Since it converts the Java source to Dex in a single step, the compilation time is reduced.

Jack library format

 

As illustrated in Figure 6, Jack has its own library format, which contains pre-compiled Dex code for the library and therefore allows faster compilation.

 

Figure 6: Jack library format

 

Jill

 

As presented in Figure 7, to include the existing third-party libraries compiled in Java bytecode format, Jill is used. It processes the class files from the libraries and transforms them into Jayce format which can be used as an input for the Jack compiler.

Figure 7: Android Build flow with Jack and Jill toolchain.

Using Jack in Android build

Jack is the default build toolchain in Android 6 (Marshmallow) and 7 (Nougat). When the Jack is used for the first time, a local Jack compilation server is launched. This server speeds up the build since it eliminates the need to start a new host JVM/JRE, to load Jack code, to initialize Jack and to warm up the JIT at each compilation. It also provides excellent compilation times during the incremental mode. The server is also a solution to control the number of parallel Jack compilations to avoid overloading issues since it limits the number of parallel compilations. The Jack server shuts down itself after an idle time without compilation [13].

However, there are a few downsides of the Jack and Jill toolchain. Since the toolchain converts Java source directly into Dex, the existing tools and plugins dependent on the Java class file format cannot be used anymore. Also, as Jack does not currently support annotation processing, the libraries like Dagger and AutoValue can no longer be used. The toolchain also increases the cost of adding Java 8 support. Therefore, despite its advantages, the Jack toolchain is now deprecated by the Android community.

3.3 Aspect-Oriented Programming

 

Aspect-Oriented Programming (AOP) is a programming model that addresses cross-cutting concerns. AOP provides a pluggable way to add the cross-cutting concerns in the business logic [2].

Some of the key terms and concepts frequently used in AOP are described below.

1. Cross-cutting concerns: a concern which affects the entire application and is applicable throughout the application. For instance, security, logging, and data transfer are the concerns which are required in every component of an application.
2. Advice: Additional code that you want to apply to your existing model, i.e. the code corresponding to the cross-cutting concerns.
3. Join point: The point of execution at which cross-cutting concern needs to be applied in the application.
4. Pointcut: A predicate that matches join points.
5. Aspect: The pointcut and the advice together form an aspect [6,10].

3.3.1 AspectJ Library

AspectJ is an implementation of Aspect-oriented programming for Java. With just a few new constructs, AspectJ supports an implementation of a number of crosscutting concerns. AspectJ can be used for following tasks.

• Checking permissions
• Interrupting action that takes too long
• Monitoring
• Preparing any data/environment before call and processing results after call
• Opening/closing resources [8]

AspectJ weaving

There are three different ways to inject the AOP code in an application. They all require an extra step to be applied to the code. This extra step is called weaving.

1. Compile-time weaving

If both the source code and the aspect code are available, the source code and the aspects can be directly compiled with the help of AspectJ compiler. This is known as compile-time weaving.

2. Post-compile weaving / Binary weaving

If source code is not available, the aspects can be injected into already compiled classes or jars using post-compile weaving.

3. Load-time weaving

 

Load-time weaving acts the same way as post-compile weaving/binary weaving but waits to inject aspects into the code until the class loader loads the class file. This requires one or more weaving class loaders [8].

Types of advice

Advice can be added either before, after or around a join point.

1. Before advice: Advice that is invoked before a join point.
2. Afterreturning advice: Advice that is invoked after a join point completes normally. For instance, if a method returns without throwing an exception.
3. After throwing advice: Advice that is executed if a method exits unexpectedly by throwing an exception.
4. After (finally) advice: Advice that is invoked irrespective of how a join point exits (normal or exceptional return).
5. Around advice: Advice that surrounds a join point. It is the most powerful kind of advice. For instance, it can introduce a custom behavior before and after the method invocation. [7].

The example given in Code snippet 5 uses Around advice. It adds log statements before and after the execution of method print()in the class com.itool.HelloWorld.

HelloWorld class

TestAspect class

Code snippet 5: Example of AspectJ around advice

3.4 Byte Code Engineering Library (BCEL)

The Byte Code Engineering Library (BCEL) is one of the most widely used frameworks for Java classworking. BCEL is aimed to give users a convenient way to analyze, create, and manipulate Java class files. It enables its users to examine the bytecode of Java classes. BCEL library repsents the classes by objects that contain all the symbolic information of a given class: methods, fields and bytecode instructions. Such objects can be created from an existing file, be transformed by a program and written to a file again. It can also be used to create classes from scratch at the run-time [11].

The BCEL APIs abstract the bytecode representation of Java classes.  They can be used for static analysis and dynamic creation or transformation of Java class files. They assist developers to implement the desired functionality on a high-level abstraction without a need to handle the internal details of the Java bytecode format.

The APIs mainly consist of three parts:

1. Static Information: A package that describes static constraints of class files. The classes can be used to read and write class files from or to a file. This is useful especially for analyzing Java classes without having access to the source files.
2. Dynamic Information: A package to dynamically generate or modify JavaClass or Method objects. It can be used to insert, analyze or remove a piece of code from class files.
3. Utilities: Various code examples and utilities like a class file viewer, a tool to convert class files into HTML, full verifier implementation to validate a binary class according to the JVM specification, a disassembler that creates a well framed JVM-level view of a binary class.

CHAPTER 4

Implementation Process

This chapter provides a detailed walkthrough of the implementation process of the project. Section 4.1 explains the modifications done to the Android build flow to enable custom code weaving into the base code.  Section 4.2 gives an overview of the techniques that can be used for weaving a custom code, namely bytecode manipulation and Aspect-oriented programming. Pros and cons of the two techniques are discussed further in detail.

4.1 Build Flow Modifications

The Instrumentation tool enables the developers to weave a custom code (in the form of aspects) into Android Open Source Code project. To understand the working of the Instrumentation tool, let’s first look at the build flow of Android. The Android source code is mostly written in Java. As demonstrated in Figure 8, the source code is first compiled to Java bytecode. Dex tool then converts the Java bytecode to Dalvik bytecode.

Figure 8: Android build process

To add a weaving step into the build process, bytecode weaving technique is used which injects the aspects into the compiled Java bytecode binaries.  The build process after adding the weaving step can be represented by Figure 9. The process of weaving the custom aspects into the Android binaries is transparent to the end users. The instrumented emulator built this way can be used for the intended purpose.

Figure 9: Android build process with AspectJ instrumentation

As demonstrated in Figure 9, the build flow needs to be modified to add the weaving step. To achieve so, some changes are made in the file build/core/definitions.mk. The definitions.mk file defines many useful functions for file lookup, transformation, and utility. The ‘transform-java-to-classes.jar’ function defined in the definitions.mk file is responsible for compiling Java source files and packaging the compiled files into a jar. The jar file is then converted to Dex bytecode by the Dex tool. The ‘transform-java-to-classes.jar’ function is modified to add the weaving step. After the source files are compiled, custom aspects are injected. The modified classes are packaged into a jar and then the build process continues from there. A program to inject the aspects is defined in script post-compile-weaving.sh. Code snippet 6 shows the changes made to the definitions.mk file. The changes are highlighted with red color.

Code snippet 6: Changes made in build/core/definitions.mk file to support custom weaving

4.2 Weaving Custom Code

To inject a Java-based code into already compiled classes, two alternatives are available in the market:

1. Using Bytecode manipulation
2. Using Aspect-Oriented Programming

4.2.1 Using Bytecode Manipulation

Java compiler compiles Java source code to bytecode. The custom code can be injected at the bytecode level. Many well-maintained Java bytecode manipulation libraries with an intuitive APIs are available such as BCEL, Javaassist, ASM [12]. As described in section 3.4, BCEL library can be used to inject the custom code into the compiled library. As shown in Code snippet 7, BCEL library is used to add print statements at the beginning of every method in CustomWeaving class.

Code snippet 7: Program to insert log statements at the beginning of every method in a class using BCEL library

The main advantages of BCEL are its stability, extensive JVM instruction-level support and commercial-friendly Apache licensing. These features have made it a very popular choice for classworking applications. The disadvantage of BCEL is that it is not easy to use. The users of BCEL need to have a very detailed knowledge of JVM instructions. Therefore, we use an alternative option Aspect-Oriented Programming in this project.

 4.2.2 Using Aspect-Oriented Programming

As described in section 3.3, Aspect-Oriented Programming (AOP) provides a powerful way to link the aspects with the pure business logic code. AspectJ is currently the most concrete implementation of AOP language for Java. AspectJ uses bytecode manipulation technique underneath but provides an abstract way to specify the code to be injected (advice) and where it should be injected (pointcut). The example shown in Code snippet 8 adds a log at the beginning of all the methods in the class com.example.CustomWeaving

Code snippet 8: Program to insert log statements at the beginning of every method in a class using AsepectJ

A comparison of the two code weaving approaches is provided below in Table 6.

Bytecode Manipulation	Aspect Oriented Programming (AspectJ)
Provides a set of APIs to read from an existing class file, transform and write to a file again.	Provides an abstract way to inject custom code with the help of aspects. It uses bytecode manipulation technique underneath.
The users need to understand the details of JVM instructions such as the bytecode format of a class, opcodes and the operand stack since the level of abstraction is little.	It provides powerful Java-based annotations to represent the code to be injected and the place where the code needs to be inserted. Therefore, the knowledge of JVM instructions is not required.
The custom code is injected programmatically using the APIs provided.	AspectJ compiler (ajc) weaves the custom aspects into the source code.
It is not possible to insert the same piece of code into multiple methods at the same time.	With the help of Pattern matching pointcut expressions, it is feasible to insert the same piece of code into multiple methods at the same time.
Since the code injection is done by manipulating the class file directly, there is no way to verify the modified bytecode. The errors introduced due to the code manipulation are exposed at run-time only.	The ajc compiler compiles the aspect code. Therefore, the errors in the aspects can be caught at the earlier stage.
There is no run-time dependency on the third-party libraries after the code injection.	The modified code depends on the AspectJ runtime library. Therefore, the library must be present at run-time.
This technique is the most powerful since it is at low-level. For example, it is possible to create a new class from scratch, remove a method from a class, etc.	Thought AspectJ provides an expressive language, its capability is limited as compared to the Bytecode manipulation technique.

Table 6: Comparison of custom code weaving approaches: Bytecode manipulation and

Aspect-oriented programming

Because of its simplicity and the support of expressive pointcuts, we used the AspectJ library in this project. We implemented the post-compile weaving technique to instrument the Android source. Using the post-compile weaving, the custom aspects are applied to the compiled Java source classes. Since the woven code depends on the AspectJ library, the library is linked to the final executable image of the emulator. The above process is implemented in post-compile-weaving.sh script. The content of post-compile-weaving.sh script is displayed in Code snippet 9. The script is called in the modified build flow as described in section 4.1.

Code snippet 9: Content of post-compile.sh script

CHAPTER 5

Demonstration

This chapter provides a demonstration of the Instrumentation tool to understand the sequence of actions that take place when an app is launched on the Android OS. Multiple versions of Android OS have been instrumented to aid us in our analysis. There are four primary processes that participate in the application startup and are paid special attention in our analysis. These are the Zygote process, the launcher process, the Activity Manager process, and the application process. The study gives us an insight into the changes that have taken place in the mentioned processes above through the different versions of Android OS. The analysis is carried out with the help of logs that are generated by various versions of instrumented emulators corresponding to the different version of Android OS. This entire process is explained in more detail in the next few paragraphs.

The instrumentation is carried out on different versions of Android OS ranging from Gingerbread to Nougat. Aspects are used to aid the injection of logging statements.

Code snippet 10: AspectJ program to add enter/exit logs in every method of AOSP

The above aspects instrument the complete OS (except the APIs that could cause recursive method calls) and enable us to generate logs and analyze them later.

After instrumentation, we installed a sample HelloWorld app, run on the instrumented emulators for various versions of Android and collected the logs are generated by the tool through the emulators ranging through the entire app launch process. A few lines of the generated log file are shown here. Each log statement has the following fields:

<process-ID> <method signature> <Enter/Exit message> < (only with Enter message) [comma separated method arguments]>

We repeated the process on two major Android platforms namely Gingerbread (2.3.7) and Nougat (7.1.1); Gingerbread being the first publicly available version and Nougat being the latest stable version of Android. Then, we compared the results from these platforms. The analysis carried out on the platforms helps us in understanding the phenomenal growth of Android over the last few years.

Figure 10: Application startup process

Figure 10 shows the application startup process and the processes that participate in the application startup. The subsequent sections describe the analysis of the processes in detail. Through the analysis, we find that, except for the application process, the working of other processes has not changed much between the two Android platforms. Therefore, an explanation aided through visual graphs is provided for these four processes. For the application process, a detailed comparison is provided that highlights various metrics related to the APIs invoked during the startup process.

A relevant part of the output is generated by executing ‘adb shell ps’ command is provided below. The ps command shows a list of currently running processes.

———————————————————————————————————

USER      PID     PPID   NAME

———————————————————————————————————

root            1              0   /init

root         32               1            zygote

media        33                1      /system/bin/mediaserver

bluetooth    34                1      /system/bin/dbus-daemon

keystore     36                1       /system/bin/keystore

root         38                1      /system/bin/qemud

shell        40                1      /system/bin/sh

system       68            32     system_server

system          125            32      com.android.systemui

app_28          135            32      com.android.launcher

app_6            145             32     com.android.inputmethod.latin

radio              161            32     com.android.phone

app_29          481            32     com.example.pooja.test_launch

root                490            41     /system/bin/sh

————————————————————————————————————

Table 7: List of processes observed at the application startup

5.1 Launcher Process

Launcher process is started by the Activity Manager during system startup. It is responsible to register onClick() listeners to enable users to start apps from the home screen. When an app icon is clicked, it invokes startActivity() method of the Activity Manager along with the  parameters:

{act = android.intent.action. MAIN, cat = [android.intent.category. LAUNCHER], flg=0x10200000, cmp=com.example.pooja.test_launch/.MainActivity }.

The graph displayed below provides the sequence of APIs called by the launcher process. The numbers affixed on the nodes of the graph indicate the sequence in which the APIs are called. For example, in this graph, Method 4 is calling Method 5 first, Method 6 afterward and then Method 7 at last.

Figure 11: Call-stack graph of the APIs invoked by launcher process

5.2 Activity Manager

Activity Manager is one of the services hosted by the System Server (Process ID 1744). It is created by the Zygote using fork() call during the system boot and runs as a background service. Among others, the Activity Manager is primarily responsible, during the application startup, for validation tasks such as checking if the application has sufficient permissions, checking security violations and gathering stats. It also sends a request to the Zygote to create a new process for the app to be launched over a socket connection.

Figure 12 illustrates the sequence of APIs invoked by the Activity Manager. The traversal order of the graph is indicated by the numbers on the nodes of the graph.

Figure 12: Call-stack graph of the APIs invoked by Activity Manager

5.3 Zygote

As described in section 3.2, Zygote preloads all the classes that might be required by Android apps during system startup. After loading all the required classes, Zygote process listens on a socket and goes in an infinite loop, as can be seen in the code snippet shown below.

Code snippet 11: Code snippet of Zygote process which accepts a new socket request

The Zygote is responsible for launching applications by replicating itself. Therefore, it is the parent process of all the applications on Android. This can be confirmed by looking at the process IDs of the processes. As is evident from Table 7, process Id of the Zygote daemon is 32, which is also the parent ID of the application ‘com.example.pooja.test_launch’ with process Id 481.

When a new app is launched, ActivityManager sends arguments to Zygote process over the socket connection. After receiving a new request, Zygote calls runOnce() method in the ZygoteConnection class. As might be seen from Code snippet 12, from the runOnce() method, a fork() system call is made. The fork() call creates a new process which becomes a child process of the caller. After a new child process is created, both processes execute the next instruction following the fork() call. The fork() returns a zero to the newly created child process and process ID  of the child process (process ID 481) to the parent. The handleChildProc() method is called from the child process, whereas handleParentProc() method is called from the parent process to handle post-fork setup.

Code snippet 12: Code snippet of Zygote process to create a child process using fork()

The child process then throws MethodAndArgsCaller exception which gets caught in ZygoteInit.main(). It responds by invoking the exception’s run() method. This arrangement clears up all the stack frames required in setting up the process. In order to load Application-specific classes and libraries into memory, call to bindApplication() method is made. The method sends AppBindData message to the message queue, which leads to an invocation of makeApplication() method. The makeApplication() method loads the application-specific classes into memory.

A call-stack of the primary APIs invoked by the Zygote process during the application launch is illustrated in Figure 13.

Figure 13: Call-stack graph of the APIs called by Zygote Process

 5.4 Application Process

Application process (Process Id 481) is a child process of the Zygote. The details of the evaluation results of the application process are provided in the following section.

5.4.1 Android Platform Variations

A HelloWorld application is executed on different versions and configurations of Android. The two different versions are Gingerbread (2.3.7) and Nougat (7.1.1) while the two different configurations are with and without the Android Support Library. The aim of this approach was to compare the code complexity across the various versions and configurations. For each version, the logs are generated from the creation of an application process until the onCreate() method invocation of the launching activity. The onCreate() is the first callback of the Activity life cycle and it is called when the application launch is complete.  The sequence of APIs called between the application process creation and the invocation of the onCreate() method are logged and represented in the form of a graph. The call-stack of API invocations was studied based on the metrics summarized in Table 8.

Metric	Description
Number of classes loaded into memory	A total number of classes loaded in memory, the value of this parameter is calculated using Debug.getLoadedClassCount() API.
Number of classes used in Application startup	A total number of classes used in launching an application.
Number of method calls	A total number of methods invoked during the application launch
Maximum out-degree of the graph	Maximum number of edges coming out of a node in the graph
Maximum depth of graph	Number of edges from root to the deepest nodes of the graph
Average children per node	Total number of child nodes Total number of internal nodes

Table 8: Metrics used in the analysis of a call-stack graph

The observations made from the graphs analysis on the four variations are explained separately in the subsequent sections.

5.4.1.1 Gingerbread (2.3.7)

Gingerbread was the first publicly available Android platform. When an application is launched on the instrumented emulator, total 302 APIs were called during application startup. Figure 14 shows a representative part of the graph depicting primary API invocations, as the entire graph would be too large to fit here. However, the evaluation analyzes the complete stack.

Figure 14: Call-stack graph of the APIs invoked during application startup on Gingerbread

The complete graph is evaluated and the values of the evaluation parameters are calculated programmatically. Table 9 presents the values of the metrics calculated during the assessment.

Metric	Value
Number of classes loaded into memory	254
Number of classes used in Application startup	48
Number of method calls	302
Maximum out-degree of graph	14
Maximum depth of graph	18
Average children per Node	2.43

 

Table 9: Graph analysis of the call-stack graph of application process on Gingerbread (2.3.7)

5.4.1.2 Gingerbread (2.3.7) with Android support library

When developing apps that support multiple API versions, Android support library acts as a compatibility layer and provides a uniform way to provide newer features on previous versions of Android. In addition, the support library also provides additional classes and features not available in the standard Framework API for easier development and support across multiple devices.

The support library is linked as a part of the application’s APK file at the compilation time. To aid our analysis, we used an instrumented version of the support library.  With the use of the support library, additional APIs are called during the application. Table 10 provides the summary of the evaluation performed on the call stack graph generated on the instrumented platform. Since the support library API invocations are scattered across the various framework API, obtaining a smaller relevant subset of the graph does not help in understanding the differences between the two configurations of Gingerbread.

Metric	Value
Number of classes loaded into memory	2177
Number of classes used in Application startup	76
Number of method calls	510
Maximum out-degree of graph	31
Maximum depth of graph	28
Average children per Node	2.36

Table 10: Graph analysis of call-stack graph of the application process on Gingerbread (2.3.7) with Android support library

5.4.1.3 Nougat (7.1.1)

Comparing the Nougat version to the Gingerbread (without support library), we see that there is a considerable increase in complexity in terms of the number of classes loaded and methods invoked. This can be observed by looking at the graph analysis described in Table 11. Although the most part of the startup process remains the same, a lot of additional method calls are observed in the log files as compared to Gingerbread (2.3.7). This difference is made apparent in Table 13.

Parameter	Value
Number of classes loaded into memory	377
Number of classes used in Application startup	53
Number of method calls	329
Maximum out-degree of graph	14
Maximum depth of graph	21
Average children per Node	2.45

 

Table 11: Graph analysis of call-stack graph of the application process on Nougat (7.1.1)

5.4.1.4 Nougat (7.1.1) with support library

The increased code complexity of the Nougat version with the support library is reflected in the results of the evaluation described in Table 12. From the results, it can be observed that this configuration has the highest complexity.

Parameter	Value
Number of classes loaded into memory	2189
Number of classes used in Application startup	82
Number of method calls	567
Maximum out-degree of graph	32
Maximum depth of graph	39
Average children per Node	3.08

Table 12: Graph analysis of the application startup sequence on Nougat (7.1.1) with support library

5.4.1.5 Comparison of the results of analysis on Android platform variations

Comparison of the results of the analysis carried out on different versions and configurations are compared and presented with reference to the analysis on the Gingerbread (without support library) version in Table 13. Figure 15 depicts the graphical representation of the comparison.

Metrics Gingerbread Gingerbread with support library Nougat Nougat with support library Number of classes loaded in memory 254 757.08 % 48.42% 761.81% Number of classes used in the application startup 48 58.33% 10.41% 70.83% Number of method calls 302 68.87% 8.94% 87.74% Maximum out-degree of the graph 14 121.42% 7.14% 128.57% Maximum depth of the graph 18 14.29% 5.56% 116.67% Average children per node 2.43 -2.88% 0.82% 26.75%

 

Table 13: Percentage comparison of the results of analysis on different versions of Android

Figure 15: Comparison of the results of analysis on different versions of Android

 

 

 

 

 

 

 

CHAPTER 6

Summary and Conclusion

 

6.1 Project Summary

The project was divided into two phases with the first phase carried out during our CSC 899 and second phase carried out in CSC 895. We started with understanding the structure of AOSP and the build process and then rebuilt the AOSP on our local machines. Our next step was to speed up this process of modifying and rebuilding the AOSP using the Amazon EC2 instance that provided us with faster computing resources. Additionally, we explored the possibility of using different ways to inject custom code and generate logging statements. We also explored different ways to modify the AOSP such that the changes done to the Android framework API were reflected across the entire operating system. Since the recent releases of the Android codebase have expanded extensively, we started with the Gingerbread release (Android 2.3.7).

Next step was to understand Aspect-Oriented Programming better. We researched various alternatives to support AOP and selected the AspectJ library which enabled modification of the existing code by allowing runtime injection of custom code, at the same time being Java compatible. We executed post-compile time Aspect weaving, rebuilt the AOSP and generated custom log files. Once the aspect weaving was successful on Gingerbread, we extended the work to support the latest versions of Android (Nougat).

6.2 Limitations

1. Versions prior to Android 5.0 (API 21) have a limit on the total number of references that can be invoked by the code within a single Dalvik Executable (DEX) bytecode file. For each Dex file, a maximum number of allowed references is 65536. While weaving custom code, AspectJ library creates additional methods. These additionally injected methods may cause the number of references to exceed the limit, leading to a build failure. From Android 5.0, Android supports multidexing which resolves this problem.

2. Android 6.0 onwards, a new build toolchain called Jack is used to speed up the source compilation time. This toolchain replaces the previous toolchain, which consisted of multiple tools, such as javac, ProGuard, jarjar and dx. With the new toolchain, Java source code is directly compiled into Dex format reducing compilation time. Since AspectJ only works on compiled class files, the Jack toolchain has to be disabled in order to use the Instrumentation tool. This may lead to increase in the compilation time. However, since the Jack toolchain is now deprecated by the Android community, this is no longer an issue.

6.3 Conclusion

We developed a tool which enables an instrumentation for the Android platform using Aspect-Oriented Programming. With this tool, a custom code can be injected into the platform. The ability to inject the custom code can be very useful in many ways such as providing developers with better insights of Android source code, monitoring the behavior of applications and checking for security violations by Android apps.  The tool will be helpful in Android instrumentation for developers as well as end-users. It can also be extended and customized to meet the individual needs.

 

References

1. Karim Yaghmour, Porting, Extending and Customizing Embedded Android, O’Reilly media Inc., March 2013.

2. Joseph Gradecki, Nicholas Lesiecki, Mastering AspectJ- Aspect Oriented Programming in Java, Willey Technology Publishing, 2003

3. Ramnivas Laddad, AspectJ in Action, Manning publications, 2010.

4. Yliès Falcone, Sebastian Currea, Weave droid: aspect-oriented programming on Android devices: fully embedded or in the cloud, ACM publication, August 2012

5. Android Developer community, The Android Source Code [online]

Available: https://source.android.com/source/index.html

6. Fernando Cejas, Aspect Oriented Programming in Android [online]

Available: https://fernandocejas.com/2014/08/03/aspect-oriented-programming-in-              android/

7. Espen, AspectJ cheatsheet, March 2010, [online]

Available: https://blog.espenberntsen.net/2010/03/20/aspectj-cheat-sheet/

8. Gregor Kiczales, Erik Hilsdale, Jim Hugunin, Mik Kersten, Jeffrey Palm and William G. Griswold, An Overview of AspectJ, [online]

Available: http://www.ccs.neu.edu/home/lieber/com3362/w02/lectures/working-  

      lectures/ECOOP2001-Overview.pdf

9. Ankit Singh Android Inside, November 2011, [online]

Available: http://riteattitude.blogspot.com/2011/11/android-zygote-and-processes.html

10. Graham O’Regan, Introduction to Aspect-Oriented Programming, January 2014, [online] Available: http://archive.oreilly.com/pub/a/onjava/2004/01/14/aop.html

11. Apache Software Foundation, Commons BCEL, July 2016, [online]

Available: https://commons.apache.org/proper/commons-bcel/

12. Jaku Holy, Practical Introduction into Code Injection with AspectJ, Javassist, and Java Proxy, September 2011, [online] Available: https://www.javacodegeeks.com/2011/09/practical-introduction-into-code.html

13. Android Developer Community, Compiling with Jack, [online]

Available: https://source.android.com/source/jack.html

14. Tim Bird, Android Build System, June 2010, [online]

Available http://elinux.org/Android_Build_System

Appendix

Steps to instrument and build Android

 

This section provides step by step instructions to download and build Android source with AspectJ instrumentation.

 

1. Establish build environment and Download Android source

 

1. Establish your environment

Follow instructions on Android source page to establish a build environment: https://source.android.com/source/initializing.html

2. Install Repo

i.  Create a bin/ directory in your home directory and make sure that it is included in your                   path:

$ mkdir ~/bin

$ PATH=~/bin:$PATH

ii. Download the Repo tool and ensure that it is executable.

$curl https://storage.googleapis.com/git-repo-downloads/repo > ~/bin/repo

$ chmod a+x ~/bin/repo

3. Create an empty directory to hold your working files.

$ mkdir Working_Directory

$ cd Working_Directory

4. Configure git with your name and email address.

$ git config –global user.name “Your Name”

$ git config –global user.email “you@example.com”

5. Initialize repo with the following command.  (for a different version, provide a branch name with -b option. For a list of branches, see Source Code Tags and Builds.

For example,

$repo init –u https://android.googlesource.com/platform/manifest -b android-2.3.7_r1

6. Download Android source

To pull down the Android source tree to your working directory from the repositories

as specified in the default manifest, run

$ repo sync

This step may take a couple of hours depending upon network speed.

7. Download folder “instrumentation_tool” from git repository into Working_Directory

$git clone https://github.com/poojakanchan/instrumentation_tool.git

8. Apply patch to the build files. Make sure you take a backup before making the changes. Provide execution permissions to the scripts in the instrumentation_tool folder.

$cp  build/core/definitions.mk build/core/definitions_bak.mk

$ patch build/core/definitions.mk < instrumentation_tool/definitions.patch

$ chmod +x instrumentation_tool/*.sh

2. Build Android source with custom aspects

1. Write custom aspects and place them in the src/ folder. (Example Aspect class “TestAspect.java” is provided) The files under the src/ folder are automatically compiled and injected by the Instrumentation tool while running a build.

2. Initialize environment

$ export ANDROID_HOME=<path to Working_Directory>

$ source build/envsetup.sh

$ lunch

3. Build Android source

$make showcommands

To redirect logs to a log file, run

$make showcommands 2>&1 | tee log_file

To use multiple threads, run it with -j option as

$ make -j4

Note that, building the source may require up to 2-3 hours.

4. After the build is successful, launch an emulator using the command:

$ emulator

To see adb logs, use command:

$ adb logcat