Building Readium for x86 Android

Readium SDK is an open source ePub3 reader. It has support for iOS and Android. The company I work for is using the Readium SDK for ePub3 support in a mobile application. I think it does a pretty good job.

Unfortunately, for Android it uses the NDK and the library is only compiled for arm. This means it is difficult to use with the Android emulator.

I have managed to rebuild Readium for x86, and this post details the steps involved. These steps are not exact command by command instructions, so you will need at least to be comfortable on the command line and reading diff files.

Readium has several dependencies. Most of them are included in the SDK as source so nothing special needs to be done when compiling for a different Android platform. However, there are three libraries which Readium uses that need to be pre-built for the Android platform you are targeting. They are OpenSSL, Boost, and ICU.

The basic steps are:

  1. Compile Readium for arm and copy out the arm library.
  2. Compile OpenSSL, Boost, and ICU for x86 and copy those to the Readium project's dependencies.
  3. Recompile Readium for x86 and create a jar composing the x86 and arm libraries.
This jar should run on an x86 emulator as well as on an arm device.


Let's get started.

Step-by-step guide

  1. Make sure you have the Android NDK and have added the NDK to your path (running 'which ndk-build' from the command line should return something). NDK >= r9d is required.
  2. Retrieve the readium-sdk source code and its dependencies. I assume you are cloning all of these from the same working directory later in the post. I also assume that the next step occurs from the same working directory.

  3. $ git clone https://github.com/readium/readium-sdk.git
    $ git clone https://github.com/guardianproject/openssl-android.git
    $ git clone https://github.com/readium/Boost-for-Android.git
    $ mkdir ICU; cd ICU; svn export http://source.icu-project.org/repos/icu/icu/tags/release-4-6 icu-4.6
  4. Add a directory for creating the jar after building Readium
  5. $ mkdir -p epub3/lib
  6. Build readium-sdk for arm
    $ cd readium-sdk/Platform/Android
    readium-sdk/Platform/Android$ ./ndk-compile build [Path to NDK]
  7. You should now have the following:
    readium-sdk/Platform/Android$ find libs
    libs
    libs/armeabi-v7a
    libs/armeabi-v7a/libepub3.so
  8. Copy the x86 library to the epub3 directory
    readium-sdk/Platform/Android$ cp -r libs/armeabi-v7a ../../../epub3/lib/
  9. Configure and build Boost for x86 (readium sdk already has the armv7 library with it)
    1. Change directories to the location Boost-for-Android was cloned into.
    2. Alter the build-android.sh build script for boost. There is a case statement which selects the path to the tools to use. Add the case statement for the r9d NDK. Note that the variable $Platfrom is spelled wrong. Also, note that the toolkit which specifies build paramers is the same as r8e for r9d.
      Alter Boost's build script
      case "$NDK_RN" in
      ...
          9d|"9d (64-bit)")
              CXXPATH=$AndroidNDKRoot/toolchains/x86-4.6/prebuilt/$Platfrom/bin/i686-linux-android-g++
              TOOLSET=gcc-androidR8e
              ;;
          *)
              echo "Undefined or not supported Android NDK version!"
              exit 1
      esac
    3. Under the config directory, alter the user config for version 1.53. Remove the config for androidR8b and modify the config for androidR8e. The following is a diff illustrating the changes. You may have to apply this by hand.
      diff --git a/configs/user-config-boost-1_53_0.jam b/configs/user-config-boost-1_53_0.jam
      index 666d4c8..c61714a 100644
      --- a/configs/user-config-boost-1_53_0.jam
      +++ b/configs/user-config-boost-1_53_0.jam
      @@ -41,74 +41,23 @@ import os ;
       local AndroidNDKRoot = [ os.environ AndroidNDKRoot ] ;

       # --------------------------------------------------------------------
      -# Is same for 8b, 8c and 8d
      -using gcc : androidR8b
      -:
      -arm-linux-androideabi-g++
      -:
      -<archiver>arm-linux-androideabi-ar
      -<compileflags>-fexceptions
      -<compileflags>-frtti
      -<compileflags>-fpic
      -<compileflags>-ffunction-sections
      -<compileflags>-funwind-tables
      -<compileflags>-D__ARM_ARCH_5__
      -<compileflags>-D__ARM_ARCH_5T__
      -<compileflags>-D__ARM_ARCH_5E__
      -<compileflags>-D__ARM_ARCH_5TE__
      -<compileflags>-Wno-psabi
      -<compileflags>-march=armv5te
      -<compileflags>-mtune=xscale
      -<compileflags>-msoft-float
      -<compileflags>-mthumb
      -<compileflags>-Os
      -<compileflags>-fomit-frame-pointer
      -<compileflags>-fno-strict-aliasing
      -<compileflags>-finline-limit=64
      -<compileflags>-I$(AndroidNDKRoot)/platforms/android-9/arch-arm/usr/include
      -<compileflags>-Wa,--noexecstack
      -<compileflags>-DANDROID
      -<compileflags>-D__ANDROID__
      -<compileflags>-DNDEBUG
      -<compileflags>-O2
      -<compileflags>-g
      -<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/include
      -<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi/include
      -# @Moss - Above are the 'oficial' android flags
      -<architecture>arm
      -<compileflags>-fvisibility=hidden
      -<compileflags>-fvisibility-inlines-hidden
      -<compileflags>-fdata-sections
      -<cxxflags>-D__arm__
      -<cxxflags>-D_REENTRANT
      -<cxxflags>-D_GLIBCXX__PTHREADS
      -;
      -
      -# --------------------------------------------------------------------
       using gcc : androidR8e
       :
      -arm-linux-androideabi-g++
      +i686-linux-android-g++
       :
      -<archiver>arm-linux-androideabi-ar
      +<archiver>i686-linux-android-ar
       <compileflags>-fexceptions
       <compileflags>-frtti
       <compileflags>-fpic
       <compileflags>-ffunction-sections
       <compileflags>-funwind-tables
      -<compileflags>-D__ARM_ARCH_5__
      -<compileflags>-D__ARM_ARCH_5T__
      -<compileflags>-D__ARM_ARCH_5E__
      -<compileflags>-D__ARM_ARCH_5TE__
       <compileflags>-Wno-psabi
      -<compileflags>-march=armv5te
      -<compileflags>-mtune=xscale
       <compileflags>-msoft-float
      -<compileflags>-mthumb
       <compileflags>-Os
       <compileflags>-fomit-frame-pointer
       <compileflags>-fno-strict-aliasing
       <compileflags>-finline-limit=64
      -<compileflags>-I$(AndroidNDKRoot)/platforms/android-9/arch-arm/usr/include
      +<compileflags>-I$(AndroidNDKRoot)/platforms/android-9/arch-x86/usr/include
       <compileflags>-Wa,--noexecstack
       <compileflags>-DANDROID
       <compileflags>-D__ANDROID__
      @@ -116,13 +65,12 @@ arm-linux-androideabi-g++
       <compileflags>-O2
       <compileflags>-g
       <compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/include
      -<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/libs/armeabi/include
      +<compileflags>-I$(AndroidNDKRoot)/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/include
       # @Moss - Above are the 'oficial' android flags
      -<architecture>arm
      +<architecture>x86
       <compileflags>-fvisibility=hidden
       <compileflags>-fvisibility-inlines-hidden
       <compileflags>-fdata-sections
      -<cxxflags>-D__arm__
       <cxxflags>-D_REENTRANT
       <cxxflags>-D_GLIBCXX__PTHREADS
       ;
    4. Build Boost for Android x86
      Boost-for-Android$ ./build-android.sh --with-libraries=regex [Path to your NDK]
      Note: Downloading the Boost source code may fail. If it does (as it did for me), visit http://sourceforge.net/projects/boost/files/boost/1.53.0/ and download the bz2 file. Place it in the same directory as the build script.
    5. Under the readium-sdk project, move aside the arm boost library.
      readium-sdk/ePub3/ThirdParty/boost/lib$ mv libboost_regex.a libboost_regex_arm.a
    6. Copy the x86 boost library into its place
      Boost-for-Android$ cp build/lib/libboost_regex-gcc-mt-1_53.a ../readium-sdk/ePub3/ThirdParty/boost/lib/libboost_regex.a
  10. Configure and build openssl for Android x86
      1. Alter the build scripts to build a static library for x86
        1. Edit Android.mk in openssl root and remove the line adding the apps directory to the list of subdirectories to include
        2. Edit crypto/Android.mk. Under the line "LOCAL_LDLIBS += -lz" add "LOCAL_EXPORT_LDLIBS += -lz" and change any include $(BUILD_SHARED_LIBRARY) to include $(BUILD_STATIC_LIBRARY)
        3. Edit ssl/Android.mk. Change all instances of LOCAL_SHARED_LIBRARIES to be LOCAL_STATIC_LIBRARIES and also make the same include change as above making the library build as static vs shared.
      2. Run ndk-build indicating it should build for x86
        openssl-android$ ndk-build APP_ABI=x86 NDK_TOOLCHAIN=x86-4.6
      3. Copy the x86 openssl libraries into their place
        openssl-android$ cp obj/local/x86/lib*.a ../readium-sdk/ePub3/ThirdParty/openssl-android/lib/
  11. Configure and build icu forAndroid x86. You may notice that readium has their own fork of ICU and that I am not using it. I was unable to figure out how to build their copy much less build it for x86 Android. It would be better to use the copy that their SDK is designed to use if I could compile it.
    1. First, configure and build for your own system (we'll build for Android afterwards). You can run these commands individually or past them into a shell script.
      ICU$ mkdir build_icu_osx
      ICU$ cd build_icu_osx
      ICU/build_icu_osx$ export BASE_ICU_DIR=the parent ICU dir where you cloned icu
      ICU/build_icu_osx$ export ICU_SOURCES=$BASE_ICU_DIR/icu-4.6
      ICU/build_icu_osx$ export CPPFLAGS="-O3 -DU_USING_ICU_NAMESPACE=1 -fno-short-enums \
      -DU_HAVE_NL_LANGINFO_CODESET=0 -D__STDC_INT64__ -DU_TIMEZONE=0 \
      -DUCONFIG_NO_LEGACY_CONVERSION=1 -DUCONFIG_NO_BREAK_ITERATION=1 \
      -DUCONFIG_NO_COLLATION=1 -DUCONFIG_NO_FORMATTING=1 -DUCONFIG_NO_TRANSLITERATION=0 \
      -DUCONFIG_NO_REGULAR_EXPRESSIONS=1"
      ICU/build_icu_osx$ sh $ICU_SOURCES/source/runConfigureICU Linux --prefix=$PWD/icu_build --enable-extras=no \
      --enable-strict=no -enable-static --enable-shared=no --enable-tests=no \
      --enable-samples=no --enable-dyload=no
      ICU/build_icu_osx$ make -j4
      ICU/build_icu_osx$ make install
      ICU/build_icu_osx$ cd ..
    2. Configure and build for Android. Note that the first two lines here may need to be altered for your system, platform, and NDK.
      ICU$ mkdir build_icu_android
      ICU$ cd build_icu_android
      ICU/build_icu_android$ export NDK_ROOT=~/Documents/android-ndk-r9d
      ICU/build_icu_android$ export NDK_TOOLCHAIN_BIN=$NDK_ROOT/toolchains/x86-4.6/prebuilt/darwin-x86_64/bin
      ICU/build_icu_android$ export SDK_ROOT=$NDK_ROOT/platforms/android-9/arch-x86
      ICU/build_icu_android$ export ICU_PATH=/Users/mwu/src/ICU/icu-4.6
      ICU/build_icu_android$ export ICU_CROSS=/Users/mwu/src/ICU/build_icu_osx
      ICU/build_icu_android$ export ICU_FLAGS="-I$ICU_PATH/source/common/ -I$ICU_PATH/source/tools/tzcode/"
      ICU/build_icu_android$ export CPPFLAGS="-fno-exceptions --sysroot=$SDK_ROOT -D__STDC_INT64__ $ICU_FLAGS -I$SDK_ROOT/usr/include/ -I$NDK_ROOT/platforms/android-9/arch-x86/usr/include -I$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.6/include -I$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/include"
      ICU/build_icu_android$ export LDFLAGS="-lsupc++ -lgnustl_static --sysroot=$SDK_ROOT -L$NDK_ROOT/sources/cxx-stl/gnu-libstdc++/4.6/libs/x86/ -Wl,-rpath-link=$NDK_ROOT/platforms/android-9/arch-x86/usr/lib/"
      ICU/build_icu_android$ export CC=$NDK_TOOLCHAIN_BIN/i686-linux-android-gcc
      ICU/build_icu_android$ export CXX=$NDK_TOOLCHAIN_BIN/i686-linux-android-g++
      ICU/build_icu_android$ export RANLIB=$NDK_TOOLCHAIN_BIN/i686-linux-android-ranlib
      ICU/build_icu_android$ export AR=$NDK_TOOLCHAIN_BIN/i686-linux-android-ar
      ICU/build_icu_android$ export LD=$NDK_TOOLCHAIN_BIN/i686-linux-android-ld
      ICU/build_icu_android$ sh $ICU_PATH/source/configure --host=i686-linux-android --enable-extras=no --enable-strict=no --enable-static --enable-shared=no --enable-tests=no --enable-samples=no --enable-dyload=no --with-cross-build=$ICU_CROSS --prefix $PWD/icu_build
    3. Remove compiling ICU tools from the Makefile. Open the Makefile that the configure call created and search for the word "tools." Remove it from the line setting the SUBDIRS variable.
    4. Build ICU
      ICU/build_icu_android$ make -j4
      ICU/build_icu_android$ make install
    5. Copy the libraries to readium
      ICU/build_icu_android$ cd icu_build/lib
      ICU/build_icu_android/icu_build/lib$ cp libicudata.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/
      ICU/build_icu_android/icu_build/lib$ cp libicui18n.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/
      ICU/build_icu_android/icu_build/lib$ cp libicuio.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/
      ICU/build_icu_android/icu_build/lib$ cp libicuuc.a [readium-sdk]/ePub3/ThirdParty/icu4c/lib/
  12. Compile Readium for x86 with x86 static libraries
      1. Go to readium-sdk/Platform/Android. Edit Application.mk. Change armeabi-v7a to x86
      2. I had to fix naming in CPUCacheUtils to get it to compile. Edit ePub3/utilities/CPUCacheUtils_i386.s and remove the underscores prefixing the names epub_sys_cache_flush and epub_sys_cache_invalidate
      3. Build Readium
        readium-sdk/Platform/Android$ ./ndk-compile.sh build [path-to-ndk]
  13. You should now have the following:
    readium-sdk/Platform/Android$ find libs
    libs
    libs/armeabi-v7a
    libs/x86
    libs/x86/libepub3.so
  14. Copy the x86 library to the epub3 directory
    readium-sdk/Platform/Android$ cp -r libs/x86 ../../../epub3/lib/
  15. Create the epub3.jar file
    epub3$ jar -cvf epub3.jar lib
  16. Copy the created jar file into your Android project.
  17. Copy java files from readium-sdk/Platform/Android/src to your Android project.

    You should now be able to use the java interfaces to the NDK library in your project. Note that Readium has javascript for the viewer which would also be needed to complete the viewer.

Comments

  1. In step 9.a. there seems to something missing on the line:
    --enable-strict=no -enable-static --enable-shared=no --enable

    Can you confirm, and if so, how should the line look?

    ReplyDelete
  2. sh $ICU_SOURCES/source/runConfigureICU Linux --prefix=$PWD/icu_build --enable-extras=no --enable-strict=no --enable-static --enable-shared=no --enable --enable-samples=no --enable-dyload=no

    ReplyDelete
  3. I had intended to type more above. The line above is without the line breaks. The other option is to escape the line break with a '\' as I did in the edited post above.

    ReplyDelete
  4. Thanks for your reply.
    There still seems to be something wrong. I get this error:

    configure: error: unrecognized option: `--enable'

    If I remove the --enable the configuration fails.

    ReplyDelete
  5. Ok it took me a while, but I figured out what got clipped out in that command. I have updated the post above. The command should have been:

    sh $ICU_SOURCES/source/runConfigureICU Linux --prefix=$PWD/icu_build --enable-extras=no --enable-strict=no -enable-static --enable-shared=no --enable-tests=no --enable-samples=no --enable-dyload=no

    The clipped parameter was --enable-tests=no

    ReplyDelete
  6. Here is latest epub3.jar using latest sdk as of 1/4/15 following steps above http://cl.ly/141r030z3S33

    ReplyDelete
  7. Hi

    Thx for sharing.

    I'm stuck on the step 9-b. Running the configure give me :
    Invalid configuration `i686-linux-android': system `android' not recognized

    Thx for your support,

    Regards.
    Mory

    ReplyDelete
  8. Mory,

    It's been a while since I looked at this. I think the problem is likely that it cannot find the tools for that --host option. Are you compiling on OSX or a different platform? Make sure all of the environment variables exported prior to running configure are appropriate for your system. In particular, pay attention to the value of NDK_TOOLCHAIN_BIN. I had it set to $NDK_ROOT/toolchains/x86-4.6/prebuilt/darwin-x86_64/bin which is specific to OSX. You will need to make sure it is set for your platform and also that the path in the NDK exists with the gcc tools within it. Look at the naming of those tools and make sure the --host option matches.

    I hope I'm not too late on the answer and that this helps.

    ReplyDelete
  9. Mory,

    It's been a while since I looked at this. I think the problem is likely that it cannot find the tools for that --host option. Are you compiling on OSX or a different platform? Make sure all of the environment variables exported prior to running configure are appropriate for your system. In particular, pay attention to the value of NDK_TOOLCHAIN_BIN. I had it set to $NDK_ROOT/toolchains/x86-4.6/prebuilt/darwin-x86_64/bin which is specific to OSX. You will need to make sure it is set for your platform and also that the path in the NDK exists with the gcc tools within it. Look at the naming of those tools and make sure the --host option matches.

    I hope I'm not too late on the answer and that this helps.

    ReplyDelete
  10. It's a some tutorial or sample to use epub3.jar

    ReplyDelete
    Replies
    1. Readium has a sample Android project for their reader at https://github.com/readium/SDKLauncher-Android

      Delete
  11. In step 7.c, it means I should edit user-config-boost-1_53_0.jam following a illustrating in 7.c Or I should use command diff --git for did it.

    ReplyDelete
    Replies
    1. I updated the diff yesterday and it should work but if it doesn't, manually modify the file according to the changes I listed.

      It has been a little while since I posted this and some of the steps needed updating. I updated it through completing the openssl build, but there are problems getting the ICU build going.

      Delete
  12. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. It looks like you are missing an additional include path. This might be due to differences compiling on Windows which I have not done.

      Around line 164 of Android.mk look for LOCAL_C_INCLUDES. Add the following to the list:
      $(LOCAL_PATH)/Platform/Android/include

      Delete
    2. This comment has been removed by the author.

      Delete
  13. This comment has been removed by the author.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. ANDROID is a problem during the build.
    Eclipse console message: In file included from ./../../ePub3/xml/utilities/base.cpp:21:0:
    ./../../ePub3/xml/utilities/base.h:37:29: fatal error: libxml / xmlerror.h: No such file or directory #include ^
    Eclipse Problems: Description Resource Path Location Type
    fatal error: libxml / xmlerror.h: No such file or directory ePub3-Library line 37 C / C ++ Problem
    My os is windows8.
    Answers I'll wait...

    ReplyDelete
  16. I simply wanted to write down a quick word to say thanks to you for
    those wonderful tips and hints you are showing on this site.


    Android Training in Bangalore

    ReplyDelete

Post a Comment

Popular Posts