A simple way to handle callbacks with Android and swig
9 min readWhy can't we just use one programming language?
Google wants us to implement our mobile apps in Java or Kotlin. If we do that, we get all the benefits of the Android platform at our fingertips. And Google traps us and our users in their walleted garden. Apple wants us to implement our mobile apps in Objective C or Swift - for the same reason. I don't think it's a coincident that all major platforms use not only proprietary API's for the user interface and system functions, but even different languages. Microsoft is no different, if you develop for Windows you must use C# (or suffer severe pain).
Despite the well intended gardening from Google, Apple (and Microsoft), their approach is not successful. Any app, exclusively available only for IOS, Android(or Windows), will fail to dominate it's niche. An app-company locked to one platform cannot win. (HTML5 and javascript has failed fundamentally to solve this problem). So all companies and even one man app shops with ambitions, will aim to launch their app for both mobile platforms, and then often for macOS and Windows too.
It's expensive (and ridiculous) to support different code bases for doing the same thing. Therefore, a common solution for many companies is to write most of the app in C++, and just implement the user interface and specific system interfaces in the language provided by Google, Apple and Microsoft. QT is in my opinion a better solution, but it's expensive to license QT for commercial use, and it's hard to make a QT app look and behave like a "native" IOS or Android app.
So, a lot of people look to C++ to solve their problem. C++ is fast (much faster than Java, Kotlin or C#), reliable, more mature than Objective C, and today quite modern and totally able to solve most problems.
Simplified Wrapper and Interface Generator - or just SWIG
The biggest problem with the C++ approach is that C++ libraries cannot be called directly from Java/Kotlin, C# or Objective C. So a typical mainstream app today, have three layers.
Let's take a closer look at how this is done with Android.
The wrapper layer must be aware of the details on how to translate between C++ and the local language, in this case Java for Android. The irony lurking in the details here is that Java cannot exist alone. Android's Java/Kotlin runtime, ART, is written in C++! So even when Google forces us to use Java or Kotlin for our apps, they use C++ to run it. Still, they have not provided a simple way to integrate the two layers.
Writing the wrapper by hand is possible, but it's probably not something you want to do. Even the simplest things looks scary, and it's very, very hard to get it right on a project with some complexity.
Example, handwritten c code that can be called to from Java
#include <string.h>
#include <jni.h>
/* This is a trivial JNI example where we use a native method
* to return a new VM String. See the corresponding Java source
* file located at:
*
* hello-jni/app/src/main/java/com/example/hellojni/HelloJni.java
*/
JNIEXPORT jstring JNICALL
Java_com_example_hellojni_HelloJni_stringFromJNI( JNIEnv* env,
jobject thiz )
{
#if defined(__arm__)
#if defined(__ARM_ARCH_7A__)
#if defined(__ARM_NEON__)
#if defined(__ARM_PCS_VFP)
#define ABI "armeabi-v7a/NEON (hard-float)"
#else
#define ABI "armeabi-v7a/NEON"
#endif
#else
#if defined(__ARM_PCS_VFP)
#define ABI "armeabi-v7a (hard-float)"
#else
#define ABI "armeabi-v7a"
#endif
#endif
#else
#define ABI "armeabi"
#endif
#elif defined(__i386__)
#define ABI "x86"
#elif defined(__x86_64__)
#define ABI "x86_64"
#elif defined(__mips64) /* mips64el-* toolchain defines __mips__ too */
#define ABI "mips64"
#elif defined(__mips__)
#define ABI "mips"
#elif defined(__aarch64__)
#define ABI "arm64-v8a"
#else
#define ABI "unknown"
#endif
return (*env)->NewStringUTF(env, "Hello from JNI ! Compiled with ABI " ABI ".");
}
All projects I have worked on use an ancient code generator, swig to generate the wrapper layer.
Swig is a monster. Last time I used it, I estimated that it would take me 2 - 4 weeks of reading and experimenting to get a reasonable understanding of how to use it correctly. And I am one who already understands most of the the principles it builds upon. Allowing Java developers to use swig to wrap "some library", by copying and pasting 10 year old snippets they don't understand from Stackoverflow is crazy. Especially now, when everything is Agile, and we don't have time to truly comprehend anything.
Just getting the memory management models in C++ and Java to work together is hard, and swig can only make some rough guesses about what C++ objects to "own" (and release) and what C++ objects to leave alone. For example, if Java get an object from a factory in C++, who will release it? If you call a C++ method with some Java object, how can the Java garbage collector later on know if it is referenced or not in the C++ library?
Interface file and simple wrappers
Doing simple things with swig is relative simple. For example, If we have this C++ class:
// Foo.h
class Foo {
public:
static int getAnswer();
std::string quoteMe(const std::string& quote);
};
We can create a very simple swig interface file to tell swig what to do.
/* Foo.i */
%module Foo_Wrapper
/* Required include files */
%{
#include "Foo.h"
%}
/* C++ Standard library wrappers */
%include "std_string.i"
/* Our native headers */
%include "Foo.h"
This will let us use code like:
public class MainActivity extends AppCompatActivity {
static {
System.loadLibrary("Foo_Wrapper");
}
Foo foo = new Foo();
@Override
protected void onCreate(Bundle savedInstanceState) {
...
// Example of a call to a native method
TextView tv = (TextView) findViewById(R.id.editText);
tv.setText("The answer to * is " + String.valueOf(Foo.getAnswer()));
// Example, send a text string and display the result.
// quoteMe() is a normal instance method that return a value.
tv.append(Foo.quoteMe("\nThanks for all the fish"));
}
}
The System.loadLibrary() method will load a shared library with our wrapper, and our Foo class, compiled for the architecture we run Android on.
Callbacks
So, in this case we create one instance of Foo, we keep it referenced as a variable in the main activity class (to avoid any garbage collection or memory leak issues), and conversion between simple types line integers and (a little more complex) strings works.
What about callbacks? After C++11, I use std::function<> and lambdas a lot to deal with callbacks in asynchronous code.
What if we want to call, for example, a REST API library written in C++ from Java?
Let us re-factor our previous example into a REST library that can POST Json payloads.
#include <string>
#include <functional>
namespace restapi {
class NativeRestApi {
public:
using completion_t = std::function<void (int httpCode, const std::string& body)>;
static int getAnswer();
std::string quoteMe(const std::string& quote);
void sendPostRequest(const std::string& url,
const std::string jsonObject,
completion_t completion);
};
} // namespace
This is a pretty standard pattern today, with modern C++.
So, how do we call sendPostRequest() from Java, and pass it a completion function that can call back to Java when the request is completed?
Swig has no idea what to do with our C++ completion_t type.
But swig has a concept about directors.
A director is a virtual class you define in C++, where swig creates Java overrides of the virtual methods. In our example, we call onComplete() in C++, and in Java, there will be a Java class overriding it, and it's onComplete() method will be called. It's not exactly what I would have preferred - I like lambdas - but it's good enough.
In a project I work on, I ran into this exact problem last week. Swig has the concept of directors, and Internet is full of examples on how you can use it, verbosely. You just create a bunch of new files in your project, copy & paste some 10 year old blocks of code, and we are set, right?
Well, not me.
I decided that I wanted exactly 0 new files for my wrapper, except for the swig interface file. So after two days reading and experimenting, I was able to find a distilled solution, that only require a small helper class, defined in the swig interface file itself.
Updated swig interface file:
%module(directors="1") NativeRestApi_Wrapper
/* Required include files */
%{
#include "NativeRestApi.h"
%}
/* C++ Standard library wrappers */
%include "std_string.i"
/* Our native headers */
%include "NativeRestApi.h"
/* C++ std::function<> callback support */
%feature("director") RequestCompletion;
%inline %{
class RequestCompletion {
public:
RequestCompletion() = default;
virtual ~RequestCompletion() = default;
/* This method will be called to in Java World by the overidden Java implementation */
virtual void onComplete(int httpCode, const std::string& body) = 0;
/* This method will be used from Java World to create a std::function<> object to the
* callback in our native API
*/
restapi::NativeRestApi::completion_t createWrapper() {
return [this](int httpCode, const std::string& body) -> void {
onComplete(httpCode, body);
};
}
};
%}
There are a few things to notice.
- The %module definition need the (directors="1") parameter to enable this feature.
- We must add "%feature("director") RequestCompletion;" before we define the completion class.
- The callback method is pure virtual.
- The code-block for RequestCompletion is %inline.
The RequestCompletion class has two responsibilities:
- It creates a C++ completion_t instance that Java can obtain, and then pass to sendPostRequest. That means that we don't need any more abstraction to provide the callback to C++. We can call sendPostRequest directly in Java.
- It is an instance of a director hybrid class, that we will derive a Java class from and provide just an override of onComplete().
The Java version of RequestCompletion:
class Completion extends RequestCompletion {
Completion() {
super();
}
@Override
public void onComplete(int httpCode, String body) {
; // Some logic to handle the event;
}
}
And, finally, some Java-code to POST some Json
class someCodeToPostJson {
NativeRestApi restApi = new NativeRestApi();
Completion completion = new Completion();
void postJson(String json) {
// Initiate an async 'rest' request with a completion
restApi.sendPostRequest("https://api.example.com/v1/testme",
json, completion.createWrapper());
}
}
That's all!
There is one potential problem, though: Memory management. Swig is smart enough to understand that something is going on with createWrapper() and sendPostRequest(), so it prevents the garbage collector from cleaning any instance of Completion that has been used. This is kind of swig, because if it did not, the C++ code would never have known when the callback was valid, and when it would crash our app. It does however create a potential memory leak. The simplest way to deal with this is to keep an instance of Completion available as a member variable or static member variable, in stead of rapidly creating new instances.
How to use swig with Android Studio
I found a really nice article about this here: Android NDK in Android Studio with SWIG.
In short, what I did was to create a new project in Android Studio 3, checked the C++ box, and then exceptions and rtt (I like those), and then I deleted the C++ files that Android Studio made for me, and made my own. I also modified the cmakelists file to handle swig, and added one section to the app's gradle.build script.
app/build.gradle
project.afterEvaluate {
javaPreCompileDebug.dependsOn externalNativeBuildDebug
}
Without this block, the build fails before the C++ wrapper layer is created.
A Real App
I have made a very simple app that shows how this all works together. The main view is just Android Studios skeleton code, with a multi-line TextView where I append the output of the interactions of the NativeRestApi methods. In my implementation, I start a std::thread for each request to sendPostRequest, sleep for 3 seconds, and then call the callback with the "result".
The C++ header and swig interface are shown above this section:
C++ implementation of NativeRestApi:
#include <chrono>
#include <thread>
#include "NativeRestApi.h"
namespace restapi {
int NativeRestApi::getAnswer() {
return 42;
}
std::string NativeRestApi::quoteMe(const std::string "e) {
return std::string("'") + quote + "'";
}
void NativeRestApi::sendPostRequest(const std::string &url,
const std::string jsonObject,
NativeRestApi::completion_t completion) {
std::thread{([completion{move(completion)}] {
std::this_thread::sleep_for(std::chrono::seconds(3));
if (completion) {
completion(200, R"({"object" : "something", "success" : true})");
}
})}.detach();
}
} // namespace
The CMakeLists.txt file:
cmake_minimum_required(VERSION 3.4.1)
set(SWIG_I_FILE "src/main/cpp/NativeRestApi.i")
set(JAVA_GEN_PACKAGE "eu.lastviking.core")
string(REPLACE "." "/" JAVA_GEN_SUBDIR ${JAVA_GEN_PACKAGE})
set(JAVA_GEN_DIR ${Project_SOURCE_DIR}/src/main/java/${JAVA_GEN_SUBDIR})
include_directories(src/main/cpp)
find_package(SWIG REQUIRED)
include(${SWIG_USE_FILE})
# Remove old java files, in case we don't need to generate some of them anymore
file(REMOVE_RECURSE ${JAVA_GEN_DIR})
# Ensure file recognized as C++ (otherwise, exported as C file)
set_property(SOURCE src/main/cpp/NativeRestApi.i PROPERTY CPLUSPLUS ON)
# Setup SWIG flags and locations
set(CMAKE_SWIG_FLAGS -c++ -package ${JAVA_GEN_PACKAGE})
set(CMAKE_SWIG_OUTDIR ${JAVA_GEN_DIR})
# Export a wrapper file to Java, and link with the created C++ library
swig_add_module(NativeRestApi_Wrapper java ${SWIG_I_FILE})
swig_link_libraries(NativeRestApi_Wrapper NativeRestApi)
add_library( # Sets the name of the library.
NativeRestApi
# Sets the library as a shared library.
SHARED
# Provides a relative path to your source file(s).
src/main/cpp/NativeRestApi.cpp
)
The Java code
package eu.lastviking.testswig;
import android.os.Bundle;
import android.widget.TextView;
import android.support.design.widget.FloatingActionButton;
import android.support.design.widget.Snackbar;
import android.support.v7.app.AppCompatActivity;
import android.support.v7.widget.Toolbar;
import android.view.View;
import android.view.Menu;
import android.view.MenuItem;
import eu.lastviking.core.NativeRestApi;
import eu.lastviking.core.RequestCompletion;
public class MainActivity extends AppCompatActivity {
static {
System.loadLibrary("NativeRestApi_Wrapper");
}
NativeRestApi restApi = new NativeRestApi();
Completion completion = new Completion();
class Completion extends RequestCompletion {
Completion() {
super();
}
@Override
public void onComplete(int httpCode, String body) {
runOnUiThread(() -> {
TextView tv = (TextView) findViewById(R.id.editText);
tv.append("\nRest result: "
+ String.valueOf(httpCode)
+ " "
+ body);
});
}
}
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Toolbar toolbar = (Toolbar) findViewById(R.id.toolbar);
setSupportActionBar(toolbar);
FloatingActionButton fab = (FloatingActionButton) findViewById(R.id.fab);
fab.setOnClickListener(new View.OnClickListener() {
@Override
public void onClick(View view) {
// Initiate an async 'rest' request with a completion
restApi.sendPostRequest("https://api.example.com/v1/testme",
"{}", completion.createWrapper());
}
});
// Example of a call to a native method
TextView tv = (TextView) findViewById(R.id.editText);
tv.setText("The answer to * is " + String.valueOf(NativeRestApi.getAnswer()));
// Example, send a text string and display the result.
// quoteMe() is a normal instance method that return a value.
tv.append(restApi.quoteMe("\nThanks for all the fish"));
// Initiate an async 'rest' request with a completion
restApi.sendPostRequest("https://api.example.com/v1/testme",
"{}", completion.createWrapper());
}
@Override
public boolean onCreateOptionsMenu(Menu menu) {
// Inflate the menu; this adds items to the action bar if it is present.
getMenuInflater().inflate(R.menu.menu_main, menu);
return true;
}
@Override
public boolean onOptionsItemSelected(MenuItem item) {
// Handle action bar item clicks here. The action bar will
// automatically handle clicks on the Home/Up button, so long
// as you specify a parent activity in AndroidManifest.xml.
int id = item.getItemId();
//noinspection SimplifiableIfStatement
if (id == R.id.action_settings) {
return true;
}
return super.onOptionsItemSelected(item);
}
}
The complete source code is available on github: SwigDirectorTest