324 lines
13 KiB
Text
324 lines
13 KiB
Text
LibreOffice Android
|
|
*******************
|
|
|
|
Bootstrap
|
|
*********
|
|
|
|
Contains common code for all projects on Android to bootstrap LibreOffice. In
|
|
addition it is a home to LibreOfficeKit (LOK - see libreofficekit/README) JNI
|
|
classes.
|
|
|
|
stuff in source directory
|
|
*************************
|
|
|
|
LibreOffice Android application - the code is based on Fennec (Firefox for Android).
|
|
It uses OpenGL ES 2 for rendering of the document tiles which are gathered from
|
|
LibreOffice using LOK. The application contains the LibreOffice core in one shared
|
|
library: liblo-native-code.so, which is bundled together with the application.
|
|
|
|
Architecture and Threading
|
|
**************************
|
|
|
|
The application implements editing support using 4 threads:
|
|
1. The Android UI thread, we can't perform anything here that would take a considerable
|
|
amount of time.
|
|
2. An OpenGL thread which contains the OpenGL context and is responsible for drawing
|
|
all layers (including tiles) to the screen.
|
|
3. A thread (LOKitThread), that performs LibreOfficeKit calls, which may take more time
|
|
to complete. In addition it also receives events from the soffice thread (see below)
|
|
when the callback emits an event. Events are stored in a blocking queue (thread
|
|
processes events in FCFS order, goes to sleep when no more event is available and
|
|
awakens when there are events in queue again).
|
|
4. A native thread created by LibreOfficeKit (we call it the soffice thread), where
|
|
LibreOffice itself runs. It receives calls from LOKitThread, and may emit callback
|
|
events as necessary.
|
|
|
|
LOKitThread
|
|
***********
|
|
|
|
LOKitThread (org.libreoffice.LOKitThread) communicates with LO via JNI (this can
|
|
be done only for one thread) and processes events (defined in org.libreoffice.LOEvent)
|
|
triggered from UI.
|
|
|
|
Application Overview
|
|
********************
|
|
|
|
LibreOfficeMainActivity (org.libreoffice.LibreOfficeMainActivity) is the entry point
|
|
of the application - everything starts up and tears down from here (onCreate, onResume,
|
|
onPause, onStart, onStop, onDestroy).
|
|
|
|
Document view
|
|
-------------
|
|
|
|
From here on one of the most interesting pieces are the classes around document view,
|
|
which includes listening to touch events, recalculating the viewport, tiled handling
|
|
and rendering the layers to the document.
|
|
|
|
Viewport - the viewport is the currently visible part of the document. It is defined
|
|
by view rectangle and zoom.
|
|
|
|
Layers - document view is rendered using many layers. Such layers are: document
|
|
background, scroll handles, and also the document tiles.
|
|
|
|
Document view classes
|
|
---------------------
|
|
|
|
- LayerView (org.mozilla.gecko.gfx.LayerView) is the document view of the application.
|
|
It uses the SurfaceView (android.view.SurfaceView) as the main surface to draw on
|
|
using OpenGL ES 2.
|
|
|
|
- GLController (org.mozilla.gecko.gfx.GLController) - holder of the OpenGL context.
|
|
|
|
- RenderControllerThread (org.mozilla.gecko.gfx.RenderControllerThread) executes the
|
|
rendering requests through LayerRenderer.
|
|
|
|
- LayerRenderer (org.mozilla.gecko.gfx.LayerRenderer) renders all the layers.
|
|
|
|
- GeckoLayerClient (org.mozilla.gecko.gfx.GeckoLayerClient) is the middle man of the
|
|
application, which connects all the bits together. It is the document view layer
|
|
holder so the any management (including tiled rendering) usually go through this
|
|
class. It listens to draw requests and viewport changes from PanZoomController
|
|
(see "Touch events").
|
|
|
|
Touch events, scrolling and zooming
|
|
-----------------------------------
|
|
|
|
The main class that handles the touch event, scrolling and zooming is JavaPanZoomController
|
|
org.mozilla.gecko.gfx.JavaPanZoomController (implementation of PanZoomController interface).
|
|
When the user performs a touch action, the document view needs to change, which means the
|
|
viewport changes. JavaPanZoomController changes the viewport and signals the change through
|
|
PanZoomTarget (org.mozilla.gecko.gfx.PanZoomTarget).
|
|
|
|
TiledRendering
|
|
--------------
|
|
|
|
Tiled rendering is a technique that splits the document to bitmaps of same size (typically
|
|
256x256) which are fetched on demand.
|
|
|
|
In the application the ComposedTileLayer (org.mozilla.gecko.gfx.ComposedTileLayer) is the
|
|
layer responsible for tracking and managing the tiles. Tiles are in this case also layers
|
|
(sub layers?) implemented in SubTile (org.mozilla.gecko.gfx.SubTile), where each one is
|
|
responsible for one tile bitmap (actually OpenGL texture once it has been uploaded).
|
|
|
|
When the viewport changes, the request for tile rechecking is send to LOKitThread (see
|
|
LOKitThread#tileReevaluationRequest), where the tiles are rechecked, add and removed if
|
|
necessary.
|
|
|
|
CompositeTileLayer is actually an abstract class, which has two implementations. One is
|
|
DynamicTileLayer (org.mozilla.gecko.gfx.DynamicTileLayer), which is used for main tile
|
|
view of the document, and FixedZoomTileLayer (org.mozilla.gecko.gfx.FixedZoomTileLayer),
|
|
which just renders the tiles at a fixed zoom level. This is then used as a background
|
|
low resolution layer.
|
|
|
|
Tile invalidation
|
|
-----------------
|
|
|
|
Tile can change in LibreOffice when user changes the content (adds, removes text or changes
|
|
the properties). In this case, an invalidation rectangle is signaled from LibreOffice, which
|
|
includes a rectangle that needs to be invalidated. In this case LOKitThread gets this request
|
|
via callback, and rechecks all tiles if they need to be invalidated. For more details see
|
|
LOKitThread#tileInvalidation).
|
|
|
|
Editing
|
|
*******
|
|
|
|
For editing there are 2 coarse tasks that the LibreOffice app must do:
|
|
1. send input events to LibreOffice core (keyboard, touch and mouse)
|
|
2. listen to messages (provided via callback) from LibreOffice core and react accordingly
|
|
|
|
In most cases when an input event happens and is send to the LO core, then a message from
|
|
LO core follows. For example: when the user writes to the keyboard, key event is sent and
|
|
a invalidation request from LO core follows. When user touches an image, a mouse event is
|
|
sent, and a "new graphic selection" message from LO core follows.
|
|
|
|
All keyboard and touch events are send to LOKitThread as LOEvents. In LOKitThread they are
|
|
processed and send to LibreOffice core. The touch events originate in JavaPanZoomController,
|
|
the keyboard events in LOKitInputConnectionHandler (org.libreoffice.LOKitInputConnectionHandler),
|
|
however there are other parts too - depending on the need.
|
|
|
|
InvalidationHandler (org.libreoffice.InvalidationHandler) is the class that is responsible
|
|
to process messages from LibreOffice core and to track the state.
|
|
|
|
Overlay
|
|
*******
|
|
|
|
Overlay elements like cursor and selections aren't drawn by the LO core, instead the core
|
|
only provides data (cursor position, selection rectangles) and the app needs to draw them.
|
|
DocumentOverlay (org.libreoffice.overlay.DocumentOverlay) and DocumentOverlayView
|
|
(org.libreoffice.overlay.DocumentOverlayView) are the classes that provide the overlay over
|
|
the document, where selections and the cursor is drawn.
|
|
|
|
|
|
Icons
|
|
*****
|
|
|
|
App uses material design icons available at [1].
|
|
|
|
|
|
[1] - https://www.google.com/design/icons/
|
|
|
|
Emulator and debugging notes
|
|
****************************
|
|
|
|
For instructions on how to build for Android, see README.cross.
|
|
|
|
* Getting something running
|
|
|
|
Attach your device, so 'adb devices' shows it. Then run:
|
|
|
|
cd android/source
|
|
make install
|
|
adb logcat
|
|
|
|
and if all goes well, you should have some nice debug output to enjoy when you
|
|
start the app.
|
|
|
|
* Using the emulator
|
|
|
|
Create an AVD in the android UI, don't even try to get the data partition size
|
|
right in the GUI, that is doomed to producing an AVD that doesn't work.
|
|
Instead start it from the console:
|
|
|
|
LD_LIBRARY_PATH=$(pwd)/lib emulator-arm -avd <Name> -partition-size 500
|
|
|
|
where <Name> is the literal name of the AVD that you entered.
|
|
|
|
[ In order to have proper acceleration, you need the 32-bit libGL.so:
|
|
|
|
sudo zypper in Mesa-libGL-devel-32bit
|
|
|
|
and run emulator-arm after the installation. ]
|
|
|
|
Then you can run ant/adb as described above.
|
|
|
|
After a while of this loop you might find that you have lost a lot of
|
|
space on your emulator's or device's /data volume. You can do:
|
|
|
|
adb shell stop; adb shell start
|
|
|
|
Debugging
|
|
---------
|
|
|
|
First of all, you need to configure the build with --enable-debug or
|
|
--enable-dbgutil. You may want to provide --enable-selective-debuginfo too,
|
|
like --enable-selective-debuginfo="sw/" or so, in order to fit into the memory
|
|
during linking.
|
|
|
|
Building with all symbols is also possible but the linking is currently
|
|
slow (around 10 to 15 minutes) and you need lots of memory (around 16GB + some
|
|
swap).
|
|
|
|
* Using ndk-gdb
|
|
|
|
When you have all this, install the .apk to the device, and:
|
|
|
|
cd android/source
|
|
<ndk-bundle>/ndk-gdb --adb=<android-sdk-linux>/platform-tools/adb [--start]
|
|
|
|
In case you used --with-android-package-name, add --package=your.package.name
|
|
to the invocation
|
|
|
|
If you get the error
|
|
|
|
ERROR: Could not find gdb.setup under ./libs/
|
|
|
|
(and an empty "Compatible device ABI:" when run with --verbose), you need to fix
|
|
the quoting in the ndk-gdb script:
|
|
|
|
@@ -574 +574 @@
|
|
-adb_var_shell BCFILES run-as $PACKAGE_NAME /system/bin/sh -c "ls lib/*.bc"
|
|
+adb_var_shell BCFILES run-as $PACKAGE_NAME /system/bin/sh -c \"ls lib/*.bc\"
|
|
|
|
|
|
Pretty printers aren't loaded automatically due to the single shared
|
|
object, but you can still load them manually. E.g. to have a pretty-printer for
|
|
rtl::OString, you need:
|
|
|
|
(gdb) python sys.path.insert(0, "/master/solenv/gdb")
|
|
(gdb) source /master/instdir/program/libuno_sal.so.3-gdb.py
|
|
|
|
* Using Android Studio (and thus lldb)
|
|
|
|
Note that both might not yield the same results - so if lldb doesn't show you
|
|
useful info, try with the ndk-gdb method and the other way round.
|
|
|
|
- open android/source/build.gradle in Android Studio via File|New → Import Project
|
|
- make sure you select the right build variant (strippedUIDebug is what you want)
|
|
- use Run|Edit Configurations to create a new configuration of type "Android Native"
|
|
- on tab "General" pick module "source"
|
|
- on tab "Native Debugger" add android/source/obj/local/<hostarch> to
|
|
the Symbol directories
|
|
|
|
Then you can select your new configuration and use Run | Debug to launch it.
|
|
Note that lldb doesn't initially stop execution, so if you want to add
|
|
breakpoints using lldb prompt, you manually have to pause execution, then you
|
|
can witch to the lldb tab and add your breakpoints. However making use of the
|
|
editor just using File|Open .. to open the desired file in Android Studio and
|
|
then toggling the breakpoint by clicking on the margin is more comfortable.
|
|
|
|
* Debugging the Java part
|
|
|
|
Open android/source/build.gradle in Android studio via File|New → Import
|
|
Project and you can use Android Studio's debugging interface.
|
|
Just make sure you pick the correct build variant (strippedUIDebug)
|
|
|
|
The alternative is to use the jdb command-line debugger. Steps to use it:
|
|
|
|
1) Find out the JDWP ID of a debuggable application:
|
|
|
|
adb jdwp
|
|
|
|
From the list of currently active JDWP processes, the last number is the just
|
|
started debuggable application.
|
|
|
|
2) Forward the remote JDWP port/process ID to a local port:
|
|
|
|
adb forward tcp:7777 jdwp:31739
|
|
|
|
3) Connect to the running application:
|
|
|
|
jdb -sourcepath src/java/ -attach localhost:7777
|
|
|
|
Assuming that you're already in the LOAndroid3 directory in your shell.
|
|
|
|
* Debugging the missing services
|
|
|
|
Android library only include essential services that are compiled for
|
|
LibreOffice in order to reduce the size of the apk. When developing,
|
|
some services might become useful and we should add those services
|
|
to the combined library.
|
|
|
|
In order to identify missing services, we need to be able to receive
|
|
SAL_INFO from cppuhelper/source/shlib.cxx in logcat and therefore identify
|
|
what services are missing. To do so, you may want add the following
|
|
when configuring the build.
|
|
|
|
--enable-selective-debuginfo="cppuhelper/ sal/"
|
|
|
|
Which services are combined in the android lib is determined by
|
|
|
|
solenv/bin/native-code.py
|
|
|
|
* Common Errors / Gotchas
|
|
|
|
lo_dlneeds: Could not read ELF header of /data/data/org.libreoffice...libfoo.so
|
|
This (most likely) means that the install quietly failed, and that
|
|
the file is truncated; check it out with adb shell ls -l /data/data/....
|
|
|
|
* Startup details
|
|
|
|
All Android apps are basically Java programs. They run "in" a Dalvik
|
|
(or on Android 5 or newer - ART) virtual machine. Yes, you can also
|
|
have apps where all *your* code is native code, written in a compiled
|
|
language like C or C++. But also such apps are actually started
|
|
by system-provided Java bootstrapping code (NativeActivity) running
|
|
in a Dalvik VM.
|
|
|
|
Such a native app (or actually, "activity") is not built as a
|
|
executable program, but as a shared object. The Java NativeActivity
|
|
bootstrapper loads that shared object with dlopen.
|
|
|
|
Anyway, our current "experimental" apps are not based on NativeActivity.
|
|
They have normal Java code for the activity, and just call out to a single,
|
|
app-specific native library (called liblo-native-code.so) to do all the
|
|
heavy lifting.
|