Building Readium for x86 Android

Readium SDK is an open source ePub3 reader. It has support for iOS and Android. The company I work for is using the Readium SDK for ePub3 support in a mobile application. I think it does a pretty good job.

Unfortunately, for Android it uses the NDK and the library is only compiled for arm. This means it is difficult to use with the Android emulator.

I have managed to rebuild Readium for x86, and this post details the steps involved. These steps are not exact command by command instructions, so you will need at least to be comfortable on the command line and reading diff files.

Readium has several dependencies. Most of them are included in the SDK as source so nothing special needs to be done when compiling for a different Android platform. However, there are three libraries which Readium uses that need to be pre-built for the Android platform you are targeting. They are OpenSSL, Boost, and ICU.

The basic steps are:

1. Compile Readium for arm and copy out the arm library.
2. Compile OpenSSL, Boost, and ICU for x86 and copy those to the Readium project's dependencies.
3. Recompile Readium for x86 and create a jar composing the x86 and arm libraries.

This jar should run on an x86 emulator as well as on an arm device.

Let's get started.

Step-by-step guide

1. Make sure you have the Android NDK and have added the NDK to your path (running 'which ndk-build' from the command line should return something). NDK >= r9d is required.
2. Retrieve the readium-sdk source code and its dependencies. I assume you are cloning all of these from the same working directory later in the post. I also assume that the next step occurs from the same working directory.

$ git clone https://github.com/readium/readium-sdk.git $ git clone https://github.com/guardianproject/openssl-android.git $ git clone https://github.com/readium/Boost-for-Android.git $ mkdir ICU; cd ICU; svn export http://source.icu-project.org/repos/icu/icu/tags/release-4-6 icu-4.6

3. Add a directory for creating the jar after building Readium

$ mkdir -p epub3/lib

4. Build readium-sdk for arm

   $ cd readium-sdk/Platform/Android readium-sdk/Platform/Android$ ./ndk-compile build [Path to NDK]

5. You should now have the following:

   readium-sdk/Platform/Android$ find libs libs libs/armeabi-v7a libs/armeabi-v7a/libepub3.so

6. Copy the x86 library to the epub3 directory

   readium-sdk/Platform/Android$ cp -r libs/armeabi-v7a ../../../epub3/lib/

7. Configure and build Boost for x86 (readium sdk already has the armv7 library with it)
   1. Change directories to the location Boost-for-Android was cloned into.
   2. Alter the build-android.sh build script for boost. There is a case statement which selects the path to the tools to use. Add the case statement for the r9d NDK. Note that the variable $Platfrom is spelled wrong. Also, note that the toolkit which specifies build paramers is the same as r8e for r9d.
      Alter Boost's build script

      case "$NDK_RN" in ...     9d|"9d (64-bit)")         CXXPATH=$AndroidNDKRoot/toolchains/x86-4.6/prebuilt/$Platfrom/bin/i686-linux-android-g++         TOOLSET=gcc-androidR8e         ;;     *)         echo "Undefined or not supported Android NDK version!"         exit 1 esac

   3. Under the config directory, alter the user config for version 1.53. Remove the config for androidR8b and modify the config for androidR8e. The following is a diff illustrating the changes. You may have to apply this by hand.

      diff --git a/configs/user-config-boost-1_53_0.jam b/configs/user-config-boost-1_53_0.jam index 666d4c8..c61714a 100644 --- a/configs/user-config-boost-1_53_0.jam +++ b/configs/user-config-boost-1_53_0.jam @@ -41,74 +41,23 @@ import os ;  local AndroidNDKRoot = [ os.environ AndroidNDKRoot ] ;  # -------------------------------------------------------------------- -# Is same for 8b, 8c and 8d -using gcc : androidR8b -: -arm-linux-androideabi-g++ -: -<archiver>arm-linux-androideabi-ar -<compileflags>-fexceptions -<compileflags>-frtti -<compileflags>-fpic -<compileflags>-ffunction-sections -<compileflags>-funwind-tables -<compileflags>-D__ARM_ARCH_5__ -<compileflags>-D__ARM_ARCH_5T__ -<compileflags>-D__ARM_ARCH_5E__ -<compileflags>-D__ARM_ARCH_5TE__ -<compileflags>-Wno-psabi -<compileflags>-march=armv5te -<compileflags>-mtune=xscale -<compileflags>-msoft-float -<compileflags>-mthumb -<compileflags>-Os -<compileflags>-fomit-frame-pointer -<compileflags>-fno-strict-aliasing -<compileflags>-finline-limit=64 -<compileflags>-I$(AndroidNDKRoot)/platforms/android-9/arch-arm/usr/include -<compileflags>-Wa,--noexecstack -<compileflags>-DANDROID -<compileflags>-D__ANDROID__ -<compileflags>-DNDEBUG -<compileflags>-O2 -<compileflags>-g -<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/include -<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi/include -# @Moss - Above are the 'oficial' android flags -<architecture>arm -<compileflags>-fvisibility=hidden -<compileflags>-fvisibility-inlines-hidden -<compileflags>-fdata-sections -<cxxflags>-D__arm__ -<cxxflags>-D_REENTRANT -<cxxflags>-D_GLIBCXX__PTHREADS -; - -# --------------------------------------------------------------------  using gcc : androidR8e  : -arm-linux-androideabi-g++ +i686-linux-android-g++  : -<archiver>arm-linux-androideabi-ar +<archiver>i686-linux-android-ar  <compileflags>-fexceptions  <compileflags>-frtti  <compileflags>-fpic  <compileflags>-ffunction-sections  <compileflags>-funwind-tables -<compileflags>-D__ARM_ARCH_5__ -<compileflags>-D__ARM_ARCH_5T__ -<compileflags>-D__ARM_ARCH_5E__ -<compileflags>-D__ARM_ARCH_5TE__  <compileflags>-Wno-psabi -<compileflags>-march=armv5te -<compileflags>-mtune=xscale  <compileflags>-msoft-float -<compileflags>-mthumb  <compileflags>-Os  <compileflags>-fomit-frame-pointer  <compileflags>-fno-strict-aliasing  <compileflags>-finline-limit=64 -<compileflags>-I$(AndroidNDKRoot)/platforms/android-9/arch-arm/usr/include +<compileflags>-I$(AndroidNDKRoot)/platforms/android-9/arch-x86/usr/include  <compileflags>-Wa,--noexecstack  <compileflags>-DANDROID  <compileflags>-D__ANDROID__ @@ -116,13 +65,12 @@ arm-linux-androideabi-g++  <compileflags>-O2  <compileflags>-g  <compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/include -<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi/include +<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/include  # @Moss - Above are the 'oficial' android flags -<architecture>arm +<architecture>x86  <compileflags>-fvisibility=hidden  <compileflags>-fvisibility-inlines-hidden  <compileflags>-fdata-sections -<cxxflags>-D__arm__  <cxxflags>-D_REENTRANT  <cxxflags>-D_GLIBCXX__PTHREADS  ;

   4. Build Boost for Android x86

      Boost-for-Android$ ./build-android.sh --with-libraries=regex [Path to your NDK]

      Note: Downloading the Boost source code may fail. If it does (as it did for me), visit http://sourceforge.net/projects/boost/files/boost/1.53.0/ and download the bz2 file. Place it in the same directory as the build script.
   5. Under the readium-sdk project, move aside the arm boost library.

      readium-sdk/ePub3/ThirdParty/boost/lib$ mv libboost_regex.a libboost_regex_arm.a

   6. Copy the x86 boost library into its place

      Boost-for-Android$ cp build/lib/libboost_regex-gcc-mt-1_53.a ../readium-sdk/ePub3/ThirdParty/boost/lib/libboost_regex.a

8. Configure and build openssl for Android x86
1. Alter the build scripts to build a static library for x86
   
   1. Edit Android.mk in openssl root and remove the line adding the apps directory to the list of subdirectories to include
   2. Edit crypto/Android.mk. Under the line "LOCAL_LDLIBS += -lz" add "LOCAL_EXPORT_LDLIBS += -lz" and change any include $(BUILD_SHARED_LIBRARY) to include $(BUILD_STATIC_LIBRARY)
   3. Edit ssl/Android.mk. Change all instances of LOCAL_SHARED_LIBRARIES to be LOCAL_STATIC_LIBRARIES and also make the same include change as above making the library build as static vs shared.
2. Run ndk-build indicating it should build for x86

   openssl-android$ ndk-build APP_ABI=x86 NDK_TOOLCHAIN=x86-4.6

3. Copy the x86 openssl libraries into their place

   openssl-android$ cp obj/local/x86/lib*.a ../readium-sdk/ePub3/ThirdParty/openssl-android/lib/

9. Configure and build icu forAndroid x86. You may notice that readium has their own fork of ICU and that I am not using it. I was unable to figure out how to build their copy much less build it for x86 Android. It would be better to use the copy that their SDK is designed to use if I could compile it.
   1. First, configure and build for your own system (we'll build for Android afterwards). You can run these commands individually or past them into a shell script.

      ICU$ mkdir build_icu_osx ICU$ cd build_icu_osx ICU/build_icu_osx$ export BASE_ICU_DIR=the parent ICU dir where you cloned icu ICU/build_icu_osx$ export ICU_SOURCES=$BASE_ICU_DIR/icu-4.6 ICU/build_icu_osx$ export CPPFLAGS="-O3 -DU_USING_ICU_NAMESPACE=1 -fno-short-enums \ -DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0 \ -DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1 \ -DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0 \ -DUCONFIG_NO_REGULAR_EXPRESSIONS=1" ICU/build_icu_osx$ sh $ICU_SOURCES/source/runConfigureICU Linux --prefix=$PWD/icu_build --enable-extras=no \ --enable-strict=no -enable-static --enable-shared=no --enable-tests=no \ --enable-samples=no --enable-dyload=no ICU/build_icu_osx$ make -j4 ICU/build_icu_osx$ make install ICU/build_icu_osx$ cd ..

   2. Configure and build for Android. Note that the first two lines here may need to be altered for your system, platform, and NDK.

      ICU$ mkdir build_icu_android ICU$ cd build_icu_android ICU/build_icu_android$ export NDK_ROOT=~/Documents/android-ndk-r9d ICU/build_icu_android$ export NDK_TOOLCHAIN_BIN=$NDK_ROOT/toolchains/x86-4.6/prebuilt/darwin-x86_64/bin ICU/build_icu_android$ export SDK_ROOT=$NDK_ROOT/platforms/android-9/arch-x86 ICU/build_icu_android$ export ICU_PATH=/Users/mwu/src/ICU/icu-4.6 ICU/build_icu_android$ export ICU_CROSS=/Users/mwu/src/ICU/build_icu_osx ICU/build_icu_android$ export ICU_FLAGS="-I$ICU_PATH/source/common/ -I$ICU_PATH/source/tools/tzcode/" ICU/build_icu_android$ export CPPFLAGS="-fno-exceptions --sysroot=$SDK_ROOT -D__STDC_INT64__ $ICU_FLAGS -I$SDK_ROOT/usr/include/ -I$NDK_ROOT/platforms/android-9/arch-x86/usr/include -I$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.6/include -I$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/include" ICU/build_icu_android$ export LDFLAGS="-lsupc++ -lgnustl_static --sysroot=$SDK_ROOT -L$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/ -Wl,-rpath-link=$NDK_ROOT/platforms/android-9/arch-x86/usr/lib/" ICU/build_icu_android$ export CC=$NDK_TOOLCHAIN_BIN/i686-linux-android-gcc ICU/build_icu_android$ export CXX=$NDK_TOOLCHAIN_BIN/i686-linux-android-g++ ICU/build_icu_android$ export RANLIB=$NDK_TOOLCHAIN_BIN/i686-linux-android-ranlib ICU/build_icu_android$ export AR=$NDK_TOOLCHAIN_BIN/i686-linux-android-ar ICU/build_icu_android$ export LD=$NDK_TOOLCHAIN_BIN/i686-linux-android-ld ICU/build_icu_android$ sh $ICU_PATH/source/configure --host=i686-linux-android --enable-extras=no --enable-strict=no --enable-static --enable-shared=no --enable-tests=no --enable-samples=no --enable-dyload=no --with-cross-build=$ICU_CROSS --prefix $PWD/icu_build

   3. Remove compiling ICU tools from the Makefile. Open the Makefile that the configure call created and search for the word "tools." Remove it from the line setting the SUBDIRS variable.
   4. Build ICU

      ICU/build_icu_android$ make -j4 ICU/build_icu_android$ make install

   5. Copy the libraries to readium

      ICU/build_icu_android$ cd icu_build/lib ICU/build_icu_android/icu_build/lib$ cp libicudata.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/ ICU/build_icu_android/icu_build/lib$ cp libicui18n.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/ ICU/build_icu_android/icu_build/lib$ cp libicuio.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/ ICU/build_icu_android/icu_build/lib$ cp libicuuc.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/

10. Compile Readium for x86 with x86 static libraries
1. Go to readium-sdk/Platform/Android. Edit Application.mk. Change armeabi-v7a to x86
2. I had to fix naming in CPUCacheUtils to get it to compile. Edit ePub3/utilities/CPUCacheUtils_i386.s and remove the underscores prefixing the names epub_sys_cache_flush and epub_sys_cache_invalidate
3. Build Readium

   readium-sdk/Platform/Android$ ./ndk-compile.sh build [path-to-ndk]

11. You should now have the following:

    readium-sdk/Platform/Android$ find libs libs libs/armeabi-v7a libs/x86 libs/x86/libepub3.so

12. Copy the x86 library to the epub3 directory

    readium-sdk/Platform/Android$ cp -r libs/x86 ../../../epub3/lib/

13. Create the epub3.jar file

    epub3$ jar -cvf epub3.jar lib

14. Copy the created jar file into your Android project.
15. Copy java files from readium-sdk/Platform/Android/src to your Android project.


You should now be able to use the java interfaces to the NDK library in your project. Note that Readium has javascript for the viewer which would also be needed to complete the viewer.