Cross-Platform Libraries

Programmers are lazy. They really are, but in a good way. According to Larry Wall (you know, the Perl guy), laziness is one of the virtues of great programmers. Programmers do not like to repeat themselves, implement the same stuff over and over again, or waste time solving the same problems. But above all, they hate fixing the same bugs.

In theory, all that can be avoided by creating reusable code, and programmers do that a lot.

When we’re considering a single app, it’s pretty straightforward with language constructs like functions and classes. If we want to share code between iOS and OS X apps, we can easily achieve that with static libraries or, now that we have iOS 8, with frameworks. But when it comes to truly cross-platform code that can be used, for example, on Android as well, things get a little bit more tricky.

We wanted to figure out how to accelerate development by creating portable code. For the purpose of our learning experiment, we chose a work time-tracking app, that we created some time ago internally. It integrates with JIRA and allows for tracking time per ticket in the project. We already have working apps for OS X (TimeCentral written in Objective-C) and for iOS/WatchKit (WorkTime written in Swift). Now we would like to write a similar client for Android. It will be essential for this experiment to extract as much code as possible (in case of our app, this means roughly: HTTP request making, response parsing and model code) into a separate library that integrates with applications for all three platforms (OS X, iOS/WatchKit, Android).

POSSIBILITIES

Before we start, it’s worth noting that there are other possible methods of achieving similar results. The options you may consider include:

WebViews in the native app

This is probably the first and simplest solution that comes to mind when you think about cross-platform apps. It allows us to create almost entire app using Web technologies, which will work on all platforms, and that usually means some sort of JavaScript+HTML+CSS combo. A native application is just a bunch of webviews (or perhaps even a single webview) that display content. The biggest advantage of this approach is that you can write all of your application (model, logic, even UI) once and run it on every platform. It may require some minor tweaks and ironing out some platform-specific kinks, but these should not be a big deal for an experienced web developer. The downside is that you can’t use all of the native features offered by the OS. Apps written using this approach are usually noticeably slower than the native ones and may feel “weird”, because using the exact same UI on both Android and iOS is just, well, wrong. Each platform has its own design guidelines that you should stick to for the sake of user experience. It is extremely hard to implement a mobile app and make it feel great (in terms of UI/UX) on all platforms in one single swoop, regardless of whether you’re going the native or the web development route.

Backend for app logic

The solution to the UI vs. different platforms problem is to write (and design!) it natively for each targeted platform (Objective-C/Swift for iOS and OS X, Java for Android) and use the common model and business logic layers. To modify the previous approach as little as possible: we could implement a RESTful service to keep all necessary data and perform operations on them. Since the application is responsible only for data presentation, it needs to call the web service each time the user performs an action or when it wants to display some data to the user. Therefore, it requires a permanent Internet connection, even when the user just wants to check how long they are working on a current task (in case of our time tracking app), which makes those applications very slow, unresponsive and in general, a bit clumsy.

It looks like this solution may work for very simple and small apps, but when you have to deal with big applications with complex scenarios, this particular approach becomes very hard to maintain and is very error prone. Lastly, the constant Internet connection requirement may limit the application’s usefulness.

Cross-Platform Libraries

It seems that if we want to build an app that feels truly native, and at the same time avoid implementing everything from scratch on each individual platform, we have to follow the cross-platform library path. The library would perform data storage and general application logic tasks. Once again, the UI would be written natively for each platform.

This approach allows us to write applications that have UI customized to specification and UI guidelines of particular operating systems, and share elements that are common between platforms. It also gives us the possibility of writing an application that can work offline (obviously).

You can find a very good analysis of the possibilities of writing cross-platform libraries on Skyscanner’s blog.

When it came to choosing how to write the cross-platform part of the application, we decided to go this route mostly because of all the advantages it offers.

HOW?

We decided that we want to have a cross-platform library for making HTTP requests to JIRA, mapping and parsing responses to a simple model. Time measurement for the current task and the UI would be native for each platform. Before we jump right into making our cross-platform library, we should look at what our perspectives are. We need to pick a language that will suit all platforms, find out what tools might be helpful or necessary along the way and figure out how the library will talk with the native code.

Language

When it comes to picking a language in which our library will be written, there isn’t much of a choice. We cannot use Objective-C, Swift nor Java since none of these work on both Apple and Google platforms. The two safe options are C and C++.

Tools

We’re so used to our SDKs and take many functionalities for granted, including Cocoa’s Foundation.framework with goodies like NSURLConnection or NSJSONSerialization. We use them almost every day, but this time, we’re out of our comfort zone and cannot use any of those in our library as it needs to be platform-agnostic.

C++ doesn’t provide much out of the box when it comes to HTTP connections and JSON parsing, its standard library is very low-level. Higher-level functionalities are achieved by using appropriate libraries. Luckily, there’s a plethora of open source libs to choose from that implement capabilities we need.

libcurl

This lightweight and well-established C library gives us everything we need for our HTTP requests. With a surprisingly (at least from a high-level language developer’s perspective) overwhelming amount of C code we can perform GET and POST requests with Basic authentication, automatically handle redirections, query encoding etc. This will perfectly suit our needs and a thin C++ wrapper will make it easily manageable.

Another nice thing about libcurl is that it’s actually embedded in OS X by default. However, you’ll need sources to build your own binaries for both iOS and Android.

restful_mapper

restful_mapper is a light ORM for RESTful APIs with a nice JSON mapping functionality. We can use it pretty much like Mantle for C++.
It has several dependencies itself:

libcurl for HTTP requests
libiconv for character encoding conversion
yajl for JSON parsing

As you can see, restful_mapper has a libcurl dependency. That’s because it’s capable of performing HTTP requests. But the way it handles them would not suit our needs. It requires the RESTful web service to follow very specific RESTful API conventions. We don’t want to be restricted by that and remain flexible since we have no control over the JIRA REST API.
Also not clear how the underlying HTTP request mechanism works and how it handles concurrency, which our apps will rely on heavily. Thus, we’ll handle HTTP requests ourselves using libcurl directly.

libiconv comes standard on both OS X and iOS. For Android it has to be compiled from sources as Android NDK doesn’t provide it.

yajl is an open source library that needs to be built for your target platform.

Glue Code

Before we dive into putting it all together, we need to consider how the C++ code is going to interact with the native code.

On iOS/OS X, C++ can be used side-by-side with Objective-C via Objective-C++. All we have to do is change the extension of our .m file to .mm and we’re good to go, the C++ code will be valid and run as expected in those files. But that’s not the case for either Android or Swift.

Swift can interoperate with C and Objective-C, but not C++ or Objective-C++. If we want to use our C++ library with Swift, we need to provide a glue code layer in either C or Objective-C (the latter obviously a better choice).

Using C/C++ code in an Android application is possible thanks to the Android Native Development Kit (NDK). Calling C++ code from Java needs some glue code in the Java Native Interface (JNI). JNI is well known for being very difficult to write in even when we need to pass the simplest data object from C++ to Java. Fortunately, there are some tools available that will help you with that or even automatically generate entire JNI files for you.

For our purpose the best solution would be a tool that can automate the process of generating glue code for both Java and Swift anyway so that it’s less cumbersome and consistent for both platforms. That’s where Djinni comes in.

Djinni

Quoting its makers, Djinni is a tool for generating cross-language type declarations and interface bindings. It was made by our friends at Dropbox. They rely heavily on their cross-platform implementation and have impressive know-how on the subject. You should check it out.

The way it works is quite simple.

prepare an IDL file with cross-platform interface descriptions
execute the Djinni‘s src/run script to generate Objective-C, Java, C++ glue-code files.
make sure the generated C++ headers are implemented in your C++ library
add Djinni‘s support JNI or Objective-C files to your project

And you’re set.

Let’s look at these steps again, this time in an example.

IDL

The IDL file basically describes interfaces common for all languages. You can have data structures called records that will be accessible from C++, Java and Objective-C as well as interfaces with methods that you’ll be able to implement in C++ and call from your Objective-C/Java and vice-versa.

First of all, we need to define data structures that we’ll pass around that represent:

a JIRA ticket
a JIRA project
a worklog
the worklog’s author
credentials and domain for JIRA REST API HTTP request

So we have to create an IDL file, say JiraClient.djinni, and the items above can be defined inside like so:

Ticket = record {
  key: string;
  ID: string;
  summary: string;
}

Project = record {
  key: string;
  ID: string;
  name: string;
}

Author = record {
  key: string;
}

Worklog = record {
  ID: string;
  startedAt: i64;
  comment: string;
  timeSpentSeconds: i64;
  ticketKey: string;
  author: Author;
}

Credentials = record {
  domain: string;
  username: string;
  password: string;
}

Based on that, Djinni will be able to generate C++ header files with structs, as well as Java and Objective-C classes for these data structures.

Now that data structures are defined, we can move on to the client’s class interface. We’ll name it JiraClientInterface and it will be implemented in our C++ library, but will be available to be called from Objective-C and Java.

#Interface for a class that handles requests to JIRA REST API
JiraClientInterface = interface +c {
  # A custom constructor (supported only via static functions)
  static JiraClientWithCredentials(credentials: Credentials): JiraClientInterface;

  # Request methods
  myTickets(): list<Ticket>;
  ticketsForProject(project: Project): list<Ticket>;
  ticketWithKey(key: string): Ticket;
  projects(): list<Project>;
  projectWithKey(key: string): Project;
  worklogsForTicket(ticket: Ticket): list<Worklog>;
  postWorklog(worklog: Worklog);
  searchWithQuery(query: map<string, string>): list<Ticket>;

  # Status code of the last HTTP request
  httpStatusCode(): i64;
  # Raw string result of the last HTTP request
  rawResult(): string;

  # Names of possible exceptions
  static resultParsingErrorName(): string;
  static httpRequestErrorName(): string;
}

Similarly to data structures described earlier, Djinni will generate JiraClientInterface‘s C++ header file as well as Java and Objective-C class files. This time however, we also need to implement JiraClientInterface‘s methods in our C++ library.

Generating Glue Code

Now that we have the IDL file with our interfaces, we need Djinni to generate the code for us. You do that by executing the src/run script with appropriate parameters specifying, amongst others, output folders for C++, JNI and ObjC code folders.

After that, Djinni will do its magic™ and depending on your configuration produce up to 4 groups of glue code files, which in our case would be:

.
├── cpp
│   ├── Author.hpp
│   ├── Credentials.hpp
│   ├── JiraClientInterface.hpp
│   ├── Project.hpp
│   ├── Ticket.hpp
│   └── Worklog.hpp
├── java
│   ├── Author.java
│   ├── Credentials.java
│   ├── JiraClientInterface.java
│   ├── Project.java
│   ├── Ticket.java
│   └── Worklog.java
├── jni
│   ├── Author.cpp
│   ├── Author.hpp
│   ├── Credentials.cpp
│   ├── Credentials.hpp
│   ├── JiraClientInterface.cpp
│   ├── JiraClientInterface.hpp
│   ├── Project.cpp
│   ├── Project.hpp
│   ├── Ticket.cpp
│   ├── Ticket.hpp
│   ├── Worklog.cpp
│   └── Worklog.hpp
└── objc
    ├── CPLAuthor+Private.h
    ├── CPLAuthor.h
    ├── CPLAuthor.mm
    ├── CPLCredentials+Private.h
    ├── CPLCredentials.h
    ├── CPLCredentials.mm
    ├── CPLJiraClientInterface.h
    ├── CPLJiraClientInterfaceCppProxy+Private.h
    ├── CPLJiraClientInterfaceCppProxy.h
    ├── CPLJiraClientInterfaceCppProxy.mm
    ├── CPLProject+Private.h
    ├── CPLProject.h
    ├── CPLProject.mm
    ├── CPLTicket+Private.h
    ├── CPLTicket.h
    ├── CPLTicket.mm
    ├── CPLWorklog+Private.h
    ├── CPLWorklog.h
    └── CPLWorklog.mm

Now we need to make sure that our library implements the interfaces from cpp/. After that, we’ll be able to simply add our library with all its dependencies, the contents of cpp/, and objc/ together with Djinni‘s support files for Objective-C to the iOS/OS X project and we’ll be good to go. For an Android project we also have to add the generated jni/ and java/ files.

Library

Before we jump into the fun part, let’s recap.

We’re writing our library in C++.
We’ll use libcurl to handle HTTP requests
…and restful_mapper to map JSON into C++ objects.
Djinni will generate glue code for Java and Objective-C/Swift.

The Core

Let’s start by creating the header for the JiraClient class which will be at the heart of our library. It will be responsible for performing appropriate requests to JIRA’s REST API and wrapping the results into model objects. We’ve already described its interface in Djinni‘s IDL file as JiraClientInterface and generated the JiraClientInterface.hpp file, now it’s time to implement it.

We’ll start by creating a JiraClient.hpp file with a JiraClientInterface subclass named JiraClient. Naturally we need to add #include for "JiraClientInterface.hpp" generated by Djinni.

#pragma once

#include "JiraClientInterface.hpp"

class JiraClient : JiraClientInterface {

}

Now let’s add declarations for the virtual methods from JiraClientInterface together with a convenience constructor and some private fields and methods.

#pragma once

#include "JiraClientInterface.hpp"

class JiraClient : JiraClientInterface {
public:
  JiraClient(const Credentials & credentials)
  : domain(credentials.domain), username(credentials.username), password(credentials.password) {};

  std::vector<Ticket> myTickets();

  std::vector<Ticket> ticketsForProject(const Project & project);

  Ticket ticketWithKey(const std::string & key);

  std::vector<Ticket> searchWithQuery(const std::unordered_map<std::string, std::string> & query);

  std::vector<Project> projects();

  Project projectWithKey(const std::string & key);

  std::vector<Worklog> worklogsForTicket(const Ticket & ticket);

  void postWorklog(const Worklog & worklog);

  int64_t httpStatusCode();

  std::string rawResult();

private:
  std::string domain;
  std::string username;
  std::string password;
  int64_t _httpStatusCode;
  std::string _rawResult;

  void reset();
};

Great stuff. It’s almost time to move onto the implementation. Before we do that, however, there’s one thing that will make things easier along the way.

CURLPP

Using libcurl in good ole C can be a bit cumbersome. Managing multiple write callbacks and memory management in C can be a pain and even simple GET request code ends up looking complex and may intimidate developers used to higher level abstractions.

struct MemoryStruct {
  char *memory;
  size_t size;
};

static size_t
WriteMemoryCallback(void *contents, size_t size, size_t nmemb, void *userp)
{
  size_t realsize = size * nmemb;
  struct MemoryStruct *mem = (struct MemoryStruct *)userp;

  mem->memory = realloc(mem->memory, mem->size + realsize + 1);
  if(mem->memory == NULL) {
    /* out of memory! */
    printf("not enough memory (realloc returned NULL)\n");
    return 0;
  }

  memcpy(&(mem->memory[mem->size]), contents, realsize);
  mem->size += realsize;
  mem->memory[mem->size] = 0;

  return realsize;
}

int main(int argc, const char * argv[]) {
  CURL *curl_handle;
  CURLcode res;

  struct MemoryStruct chunk;

  chunk.memory = malloc(1);
  chunk.size = 0;

  curl_handle = curl_easy_init();

  curl_easy_setopt(curl_handle, CURLOPT_URL, "http://www.macoscope.com/");
  curl_easy_setopt(curl_handle, CURLOPT_FOLLOWLOCATION, 1L);
  curl_easy_setopt(curl_handle, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
  curl_easy_setopt(curl_handle, CURLOPT_WRITEDATA, (void *)&chunk);

  res = curl_easy_perform(curl_handle);

  if (res != CURLE_OK) {
    fprintf(stderr, "curl_easy_perform() failed: %s\n", curl_easy_strerror(res));
  }
  else {
    printf("%lu bytes retrieved\n", (long)chunk.size);
  }

  curl_easy_cleanup(curl_handle);
  free(chunk.memory);

  return 0;
}

But we can make a thin C++ wrapper to make our lives just a little bit easier. There’s a number of ways to do that, and a simplistic approach like this one will suit us for now. Let’s create a CURLPP class with a simple public interface

CURLPP.hpp

#pragma once

#include <stdio.h>
#include <sstream>
#include <curl/curl.h>
#include <unordered_map>

class CURLPP
{
public:
  std::string urlString;
  std::string username;
  std::string password;
  std::string postBody;
  std::unordered_map<std::string, std::string> query;

  CURLPP()
  : curl(curl_easy_init())
  , httpCode(0)
  {

  }

  ~CURLPP()
  {
    if (curl) {
      curl_easy_cleanup(curl);
    }
  }

  long getHttpCode()
  {
    return httpCode;
  }

  std::string get();
  std::string post();

private:
  CURL *curl;
  std::stringstream requestResultBuffer;
  long httpCode;
  void commonCURLPreparation();
  std::string commonCURLPerform();
};

and hide libcurl‘s complexities inside

CURLPP.cpp

#include "CURLPP.hpp"
#include <stdexcept>
#include <sstream>

using namespace std;

static size_t write_data(void *buffer, size_t elementSize, size_t numberOfElements, void *userp)
{
  static_cast<stringstream *>(userp)->write((const char *)buffer, elementSize * numberOfElements);
  return elementSize * numberOfElements;
}

string encodedQuery(CURL *curl, const unordered_map<string, string> & query)
{
  stringstream stream;
  for (const auto & pair : query) {
    const string & key = pair.first;
    const string & value = pair.second;
    char *encodedKey = curl_easy_escape(curl, key.c_str(), (int)key.length());
    char *encodedValue = curl_easy_escape(curl, value.c_str(), (int)value.length());
    stream << encodedKey << "=" << encodedValue << "&";
    curl_free(encodedKey);
    curl_free(encodedValue);
  }
  return stream.str();
}

void CURLPP::commonCURLPreparation()
{
  requestResultBuffer.str("");
  httpCode = 0;
  curl_easy_setopt(curl, CURLOPT_URL, (urlString + "?" + encodedQuery(curl, query)).c_str());
  curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
  curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, write_data);
  curl_easy_setopt(curl, CURLOPT_WRITEDATA, &requestResultBuffer);
  curl_easy_setopt(curl, CURLOPT_USERPWD, (username + ":" + password).c_str());
}

string CURLPP::commonCURLPerform()
{
  CURLcode res = curl_easy_perform(curl);
  if (res != CURLE_OK) {
    throw runtime_error(curl_easy_strerror(res));
  }
  curl_easy_getinfo(curl, CURLINFO_RESPONSE_CODE, &httpCode);
  return requestResultBuffer.str();
}

string CURLPP::get()
{
  commonCURLPreparation();
  return commonCURLPerform();
}

string CURLPP::post()
{
  commonCURLPreparation();
  curl_easy_setopt(curl, CURLOPT_POSTFIELDS, postBody.c_str());

  struct curl_slist *headerList = NULL;
  headerList = curl_slist_append(headerList, "Content-Type: application/json");
  curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headerList);

  string result = commonCURLPerform();

  curl_slist_free_all(headerList);
  return result;
}

From now on we can perform requests much more conveniently thanks to CURLPP.

CURLPP request;
request.urlString = "http://macoscope.com/"
request.username = username;
request.password = password;
string resultString = request.get();

restful_mapper

Now that we’re ready to make HTTP requests, let’s look at restful_mapper and how it can help us map received JSONs to C++ objects.
JiraClient‘s std::vector<Project> projects(); will serve as a good example. This method is supposed to fire a GET request to JIRA’s /rest/api/2/project method which returns a JSON with an array of projects, each one with this particular structure:

{
    "expand": "description,lead,url,projectKeys",
    "self": "https://macoscope.atlassian.net/rest/api/2/project/13300",
    "id": "13300",
    "key": "CPMCLAB",
    "name": "Cross Platform Lab",
    "avatarUrls": {
        "48x48": "https://macoscope.atlassian.net/secure/projectavatar?avatarId=10011",
        "24x24": "https://macoscope.atlassian.net/secure/projectavatar?size=small&avatarId=10011",
        "16x16": "https://macoscope.atlassian.net/secure/projectavatar?size=xsmall&avatarId=10011",
        "32x32": "https://macoscope.atlassian.net/secure/projectavatar?size=medium&avatarId=10011"
    }
},

We’re only really interested in the key, id, and name fields, as described in Djinni‘s IDL file

Project = record {
  key: string;
  ID: string;
  name: string;
}

so these fields need to be mapped to the Project struct generated by Djinni:

// AUTOGENERATED FILE - DO NOT MODIFY!
// This file generated by Djinni from JiraClient.djinni

#pragma once

#include <string>
#include <utility>

struct Project final {

    std::string key;

    std::string ID;

    std::string name;


    Project(
            std::string key,
            std::string ID,
            std::string name) :
                key(std::move(key)),
                ID(std::move(ID)),
                name(std::move(name)) {
    }
};

restful_mapper can help us with that, but here’s the catch, it cannot map straight to our Project struct. It needs a model object to be a class inheriting from restful_mapper::Model with fields and relationships of specific types.

So we need to declare a ProjectWrapper class, that follows restful_mapper requirements and add a Project project() getter, that will take the mapped data and return our Project struct object.

class ProjectWrapper : public Model<ProjectWrapper>
{
public:
  Field<std::string> ID;
  Field<std::string> key;
  Field<std::string> name;

  virtual void map_set(Mapper &mapper) const
  {
    mapper.set("id", ID);
    mapper.set("key", key);
    mapper.set("name", name);
  }

  virtual void map_get(const Mapper &mapper)
  {
    mapper.get("id", ID);
    mapper.get("key", key);
    mapper.get("name", name);
  }

  Project project() const
  {
    return Project(key, ID, name);
  }
};

Now that our model is ready to parse JSON, let’s make the request in JiraClient.cpp

vector<Project> JiraClient::projects()
{
  reset(); //cleanup JiraClient's transient data

  // Prepare request
  CURLPP request;
  request.urlString = "https://" + domain + "/rest/api/2/project";
  request.username = username;
  request.password = password;

  // Perform http request (CURLPP might throw an exception)
  string resultString;
  try {
    resultString = request.get();
    _rawResult = resultString;
    _httpStatusCode = request.getHttpCode();
  } catch (...) {
    throw runtime_error(httpRequestError);
  }

  // Map JSON to ProjectWrapper objects (restful_mapper might throw an exception)
  try {
    HasMany<ProjectWrapper> projectWrappers; // HasMany is a restful_mapper's type for an array of objects
    projectWrappers.from_json(resultString); // Feed JSON to restful_mapper
    vector<Project> projects; // Prepare a buffer for Project structs
    // Loop through ProjectWrapper objects, extract Project structs and store them in the buffer
    for (ProjectWrapper projectWrapper : projectWrappers.items()) {
      projects.push_back(projectWrapper.project());
    }
    // Return successfully mapped Project objects
    return projects;
  } catch (...) {
    throw runtime_error(resultParsingError);
  }
}

With all that JiraClient::projects() can now make an authorized HTTP request, parse JSON into restful_mapper‘s model objects which are then converted into Djinni‘s Project objects that are compatible with all platforms.
Now rinse and repeat with the rest of the methods. The idea is the same.

make an appropriate request
use restful_mapper to parse JSON into model objects
extract data from model objects to Djinni‘s objects and return them

An iOS / OS X Static Library

Assuming that we’ve implemented the rest of our JiraClient.cpp, it’s time to create an iOS / OS X static library, which we’ll name libjira_client.a. Our library is quite light, and consists of only 4 files:

JiraClient.hpp/.cpp
CURLPP.hpp/.cpp

and two dependencies

libcurl
restful_mapper

We want to be able to easily produce a library for both OS X and iOS, with the iOS library being a universal one for both simulator and devices. With a small configurable Makefile we’re able to achieve just that:

...

SDK_ROOT = /Applications/Xcode.app/Contents/Developer/Platforms
SDK_IPHONEOS = -isysroot $(SDK_ROOT)/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk
SDK_IPHONE_SIMULATOR = -isysroot $(SDK_ROOT)/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator.sdk
SDK_OSX = -isysroot $(SDK_ROOT)/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk

ROOTDIR = $(realpath .)
INC_IOS = -I $(ROOTDIR)/vendor/libcurl -I $(ROOTDIR)/vendor/restful_mapper
INC_OSX = -I $(ROOTDIR)/vendor/restful_mapper

ARCHS_ARM = -arch armv7 -arch armv7s -arch arm64
ARCHS_X86 = -arch i386 -arch x86_64
CFLAGS = -std=c++11

COMMON_IOS_FLAGS = -miphoneos-version-min=8.0

CC = clang
TARGET = libjira_client
OBJECTS = JiraClient.o CURLPP.o

...

One thing you need to keep in mind is to pass the path to the appropriate SDK (SDK_IPHONEOS, SDK_IPHONE_SIMULATOR, SDK_OSX) to the compiler and to include search paths for our dependencies (INC_IOS, INC_OSX). INC_OSX does not have a path to libcurl because OS X comes with it built in, whereas iOS does not and will need us to compile a binary.

iOS also requires us to pass a minimum iOS version (COMMON_IOS_FLAGS)

With the above taken care of, we can now call make and produce all the versions of the static library that we might need:

libjira_client-device.a – iOS device (armv7, armv7s, arm64)
libjira_client-simulator.a – iOS simulator (i386, x86_64)
libjira_client-ios.a – iOS universal (armv7, armv7s, arm64, i386, x86_64)
libjira_client-osx.a – OS X (i386, x86_64)

We may also call make ios, make ios-device, make ios-simulator or make osx if we want to narrow down the kinds of produced libraries.

With that said and done, we can move on and hook it up to our apps.

A Shared Library for Android

Producing C/C++ code for Android is not a very pleasant experience. As stated before, JNI is hard to use, but thanks to Djinni we were able to use our C++ code from Java quite easily. Still, there were issues: we quickly found out that Android Studio currently doesn’t support the debugging of C++ code. Also, by default, JNI error messages that you can read from logcat are very enigmatic. The blog post on the Android Developers Blog about Debugging Android JNI with CheckJNI is very helpful in that matter. It introduces a tool called checkJNI that helps catch common errors. To enable it, use the following command:

adb shell setprop debug.checkjni 1

For the purpose of our experiment, we’ve used the latest available NDK (r10d).

The first thing that needs to be configured if the default NDK settings don’t suit you is the Application.mk file. As stated in CPLUSPLUS-SUPPORT.HTML in the NDK documentation (you can find it in <ndk>/docs/), Android provides minimal C++ runtime support library. Specifically, the default runtime doesn’t provide:

standard C++ Library support (except a couple of trivial headers),
C++ exceptions support,
RTTI support.

It is, however, possible to choose a different C++ runtime. To do this, we need to specify APP_STL flag in the Application.mk file. We’ll use gnustl_shared library as it has all of capabilities mentioned above.

We will also need features form C++11. With the Android NDK we can access them fairly easily, all we have to do is to set APP_CPPFLAGS with the value -std=gnu++11 in Application.mk.

We should also define what platforms will be supported. By setting APP_ABI := all flag you will get libs for arm64-v8a, armeabi, armeabi-v7a, mips, mips64, x86, and x86_64. By default, the NDK compiles only for armeabi. It’s possible to specify which architecture your build should target. To do this simply define APP_ABI with the name of the specific architecture, eg. APP_ABI := armeabi-v7a.

Our entire Application.mk file looks like this:

APP_ABI := all
APP_STL := gnustl_shared
APP_CPPFLAGS += -fexceptions
APP_CPPFLAGS += -frtti
APP_CPPFLAGS += -std=gnu++11

Secondly, we have to configure the Android.mk file for each module used in the app. We put the source files for our shared library and the JNI glue code together. We also add Djinni support files. Here is the part of Android.mk for JiraClient specifically:

...
include $(CLEAR_VARS)
LOCAL_PATH := $(MY_PATH)
LOCAL_MODULE := JiraClient
LOCAL_LDLIBS := -llog -lz
LOCAL_SHARED_LIBRARIES := libiconv libyajl
LOCAL_STATIC_LIBRARIES := curl librestful_mapper
LOCAL_C_INCLUDES :=  $(LOCAL_PATH)/generated-src/cpp \
                     $(LOCAL_PATH)/djini_support \
                     $(LOCAL_PATH)/curl
LOCAL_SRC_FILES :=  JiraClient.cpp \
                    CURLPP.cpp \
                    djini_support/djinni_support.cpp \
                    djini_support/djinni_main.cpp \
                    Credentials.cpp \
                    Project.cpp \
                    Worklog.cpp \
                    JiraClientInterface.cpp \
                    Ticket.cpp
include $(BUILD_SHARED_LIBRARY)

LOCAL_LDLIBS defines libraries provided by NDK

llog – liblog – logging library
lz – zlib – compression library used by libcurl

Other dependencies are defined in LOCAL_SHARED_LIBRARY and LOCAL_STATIC_LIBRARY as prebuilt shared or static libraries. We’re building our library as shared library.

We have to specify the name of our library in LOCAL_MODULE. During the build process, the NDK adds the “lib” prefix to the module name when creating the library. The final result we get is libJiraClient.so file for each architecture that we can use in our Android project. We also have to add other prebuilt libraries that the JiraClient depends on to the project.

The OS X Objective-C App

To test the library in the OS X environment, we’re going to use TimeCentral, our internal app for tracking time with JIRA. Assuming we have our libjira_client.a compiled for OS X (i386 & x86_64), as well as all the dependencies: libyajl and librestful_mapper (libcurl and libiconv are available in the standard system SDK) all we have to do is add them to the project in Xcode. We also need to add Djinni‘s support files for Objective-C and the generated files from cpp/ and objc/.

With that sorted out, we can import Djinni‘s headers for Objective-C where we need them.

#import "CPLJiraClientInterfaceCppProxy.h"
#import "CPLCredentials.h"
#import "CPLProject.h"
#import "CPLTicket.h"
#import "CPLWorklog.h"
#import "CPLAuthor.h"

and we’re ready to start using libjira_client in our app. Let’s look at the same example we tackled describing the C++ implementation but now from the Objective-C perspective, that is fetching Projects from JIRA’s REST API.

As configured in Djinni‘s IDL file, all Objective-C glue code is prefixed with CPL. So Project objects are represented in Objective-C as CPLProject.

We start by creating a client instance.

CPLCredentials *credentials = [[CPLCredentials alloc] initWithDomain:@"macoscope.atlassian.net"
                                                            username:@"myUsername"
                                                            password:@"123456isTheBestPassword"];
id <CPLJiraClientInterface> client = [CPLJiraClientInterfaceCppProxy JiraClientWithCredentials:credentials];

on which we can simply call

NSArray *projects = [client projects]; // array of CPLProject objects

Under the hood [client projects] calls the vector<Project> JiraClient::projects() in our C++ library, thus performing an appropriate HTTP request and mapping the resulting JSON into model objects. Now we can easily access the projects’ values:

CPLProject *project = projects.firstObject;
NSLog(@"project's name: %@, key: %@ and ID: %@", project.name, project.key, project.ID);

However, there are still two things left to handle here.
Our C++ library is designed to leave the thread management to the user’s platform, so all calls are synchronous.

The second thing is error handling. [client projects] does not take an NSError pointer as you’d expect from other Cocoa APIs. It also won’t return nil in case of an error, because under the hood it relies on the C++ implementation of vector<Project> JiraClient::projects() that is built following the exceptions paradigm used in C++.

Asynchronous Requests

It’s really is up to the developer to decide how to approach this issue. The rules are the same as with any other synchronous call that we’d like to be performed in the background. The most common one would be dispatch_async

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
  NSArray *projects = [client projects]; // fetch Projects in background
  // do whatever with the projects
}

Exceptions

This one is really important. Cocoa is designed with exceptions intended for programmer errors only. That’s not the case with C++. Our implementation of libjira_client will throw an exception in case of an HTTP connection request or JSON mapping failure. Fortunately, Djinni automatically translates C++ exceptions to NSException and Objective-C supports them with @try @catch blocks.

There are two kinds of possible exceptions that libjira_client can throw, which can be distinguished by looking into the NSException.name property. libjira_client‘s exception names can be accessed via the CPLJiraClientInterfaceCppProxy class methods.

+ (NSString *)resultParsingErrorName;

+ (NSString *)httpRequestErrorName;

Now all we have to do is wrap our [client projects] call in a @try @catch block and handle any exceptions

@try {
  projects = [client projects];
}
@catch (NSException *exception) {
  if ([exception.name isEqualToString:[CPLJiraClientInterfaceCppProxy httpRequestErrorName]]) {
    // handle HTTP request error
  } else if ([exception.name isEqualToString:[CPLJiraClientInterfaceCppProxy resultParsingErrorName]]) {
    // handle JSON parsing error
  } else {
    // handle unexpected exception
  }
}

Objective-C Round-up

As seen above, integrating a C++ library with an Objective-C OS X app does not cause any trouble, especially with help from Djinni. The only thing that makes this approach different from the usual is the use of exceptions. However, Objective-C supports them and Djinni does a fine job with mapping C++ exceptions to NSException.

The iOS Swift App

To see how our library integrates with Swift, we’ll try and hook it up to our WATCH app called WorkTime.

The steps are very similar to the OS X Objective-C app. We add our static libraries to the project, but this time around, we also include libcurl.a, because unlike OS X, iOS does not come bundled with it by default.

Then we add files generated by Djinni

the contents of cpp/
the contents of objc/
Djinni‘s support files

Unless we already have an Objective-C bridging header in our Swift project, Xcode will automatically prompt us to create one. We’ll need it.

We must import any Objective-C headers we want to be able to use from Swift inside our bridging header. In our case, that’ll be:

#import "CPLJiraClientInterfaceCppProxy.h"
#import "CPLCredentials.h"
#import "CPLProject.h"
#import "CPLTicket.h"
#import "CPLWorklog.h"
#import "CPLAuthor.h"

The files structure in our project should resemble the OS X one with the addition of bridging headers and ExceptionProxy (which we’ll cover later on)

We can start using libjira_client in our Swift code.
Once again, let’s have a look at the example with fetching projects.

We start out by creating a client instance.

let credentials = CPLCredentials(domain: "macoscope.atlassian.net", username: "myUsername", password: "123456isTheBestPassword")
let client = CPLJiraClientInterfaceCppProxy.JiraClientWithCredentials(credentials)

on which we make a call to fetch projects

let projects = client.projects() as! [CPLProject]

It’s the same story we’ve seen in the Objective-C example. Under its hood, client.projects() calls the vector<Project> JiraClient::projects() in our C++ library, thus performing an appropriate HTTP request and mapping the result JSON into model objects, which is why we can force cast it to [CPLProject]. Now we can easily access the projects’ values:

let project = projects[0]
println("project's name: \(project.name), key: \(project.key) and ID: \(project.ID)")

Of course, we have to sort out how to handle the issues we dealt with in Objective-C, those being:

Asynchronous requests
Exceptions

Asynchronous Requests

These are exactly the same in Swift as in Objective-C. It’s up to the developer to choose how to handle them. One approach would be a simple dispatch_async

dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), { () -> Void in
  let projects = client.projects() as! [CPLProject] // fetch Projects in background
  // do whatever with the projects
}

Exceptions

These are a little bit more tricky in Swift than in Objective-C. Swift does not support exceptions, so we need to fall back on Objective-C for that. That’s what the ExceptionProxy.h/.m files are for. ExceptionProxy.h also needs to be included in the bridging header.

ExceptionProxy.h

#import <Foundation/Foundation.h>

@interface ExceptionsProxy : NSObject
+ (void)performCatchingExceptions:(id (^)(void))operationBlock
                            allOK:(void (^)(id result))allOKHandler
                  exceptionRaised:(void (^)(NSException *exception))exceptionHandler;
@end

ExceptionProxy.m

#import "ExceptionsProxy.h"

@implementation ExceptionsProxy

+ (void)performCatchingExceptions:(id (^)(void))operationBlock
                            allOK:(void (^)(id result))allOKHandler
                  exceptionRaised:(void (^)(NSException *exception))exceptionHandler
{
  @try {
    id result = operationBlock();
    allOKHandler(result);
  }
  @catch (NSException *exception) {
    exceptionHandler(exception);
  }
}

@end

This simple construct allows us to call anything from Swift and catch exceptions.
We do that by passing a closure that might throw an exception as operationBlock. Objective-C will wrap that in a @try @catch block, and trigger allOKHandler if no exception was raised, or exceptionHandler if it was.

This can be used in Swift like so:

ExceptionsProxy.performCatchingExceptions({ () -> AnyObject! in
    // Make a client request
    // Exception will be caught and passed to exceptionRaised handler
    // If no exception is raised, value returned from this closure will be passed to allOK handler
  }, allOK: { (result) -> Void in
    // No exception raised, result is safe to use
  }, exceptionRaised: { (exception) -> Void in
    // Handle exception
  }
)

which in an example will look somethings like this

ExceptionsProxy.performCatchingExceptions({ () -> AnyObject! in
    return client.projects()
  }, allOK: { (result) -> Void in
    let projects = result as! [CPLProject]
    // Use projects
  }, exceptionRaised: { (exception) -> Void in
    switch exception.name {
    case CPLJiraClientInterfaceCppProxy.httpRequestErrorName():
      // handle HTTP request error
    case CPLJiraClientInterfaceCppProxy.resultParsingErrorName():
      // handle JSON parsing error
    default:
      // handle unexpected exception
    }
  }
)

Swift Round-up

Integrating libjira_client with Swift proves to be only slightly more difficult than with Objective-C. The need to add bridging headers and fall back on Objective-C for exception handling make the process slightly more complex.

But in the end, it integrates well, all the functionality is there and we can definitely rely on such a solution.

The (Android) Java App

After building our library, we can add it to our Android project. We place .so files compiled for different architectures in proper subdirectories in project libs folder:

...
├──libs
│   ├── arm64-v8a
│   ├── armeabi
│   ├── x86
...

The first thing that we have to do when we want to use native library in our code is to initialize it. We do it by a calling System.LoadLibrary(String libname) in a static code block. libname is the alias assigned in the Android.mk file under LOCAL_MODULE

static {
    System.loadLibrary("JiraClient");
}

Afterwards, we can use our native functions from the Java code. All functions that we need to call are defined in JiraClientInterface.Java generated by Djinni. As an example we use this code for fetching the project list.

// This file generated by Djinni from JiraClient.djinni
...
public final class Credentials {

    /*package*/ final String mDomain;
    /*package*/ final String mUsername;
    /*package*/ final String mPassword;

    public Credentials(
            String domain,
            String username,
            String password) {
        this.mDomain = domain;
        this.mUsername = username;
        this.mPassword = password;
    }
    ...
}

// This file generated by Djinni from JiraClient.djinni
...
public abstract class JiraClientInterface {
    ...
    public abstract ArrayList<Project> projects();
    public static native String resultParsingErrorName();
    public static native String httpRequestErrorName();
    public static native JiraClientInterface JiraClientWithCredentials(Credentials credentials);

    public static final class CppProxy extends JiraClientInterface
    {
        private final long nativeRef;
        private final AtomicBoolean destroyed = new AtomicBoolean(false);

        ...

        @Override
        public ArrayList<Project> projects()
        {
            assert !this.destroyed.get() : "trying to use a destroyed object";
            return native_projects(this.nativeRef);
        }
        private native ArrayList<Project> native_projects(long _nativeRef);
     }
}

To use JiraClient in our app, we initialize it with user credentials

Credentials credentials = new Credentials("macoscope.atlassian.net", username: "myUsername", password: "123456isTheBestPassword");
JiraClientInterface jira = JiraClientInterface.JiraClientWithCredentials(credentials);

Then we can use it to fetch the project list

ArrayList<Project> projects = jira.projects();

We call all requests in a try-catch block so we can handle any errors that may happen. The full code block for fetching the project list looks like this:

ArrayList<Project> projects;
try {
    Credentials credentials = new Credentials("macoscope.atlassian.net", username: "myUsername", password: "123456isTheBestPassword");
    JiraClientInterface jira = JiraClientInterface.JiraClientWithCredentials(credentials);
    projects = jira.projects();  
}
catch (Exception e){
    String error  = e.getMessage();
    if (error.equals(JiraClientInterface.httpRequestErrorName())){
        // handle HTTP request error
    }
    else if (error.equals(JiraClientInterface.resultParsingErrorName())){
        // handle JSON parsing error
    }
    else {
        // handle unexpected exception
    }
}

With all that done, we’ve compiled our client application, installed it on the actual device and ran it. We’ve been using Nexus 5 with stock Android 5.0. However, additional problems quickly became apparent.

We realized that our app can’t connect to our JIRA instance. It turned out that the problem was related to the HTTPS handshake. We’ve been getting curl’s CURLE_SSL_CACERT error which means 60 - problem with the CA cert (path?). At first we were confused. We’ve been using the libcurl library compiled with OpenSSL ver. 1.0.1 and Atlassian services use certificates from DigiCert. By default, Android has trusted root certificates installed in /system/etc/security/cacerts/, so in theory it looked that everything should work just fine.

The problem is that prior to version 1.0, OpenSSL used MD5 for hashing. So does Android. Later OpenSSL versions switched to SHA-1. Our library was simply unable to find any certificates installed on the device because of the differences in the hashing algorithm. One workaround would be to compile libcurl with a version of OpenSSL older than 1.0.

After resolving the HTTPS connection issues, we found out that we didn’t get results for some requests. Namely, the ticketsForProject call, which takes particular project as a parameter and returns the tickets belonging to it. The thing is that for some projects, the requests were fine and for other we got errors. After analyzing the raw response, we saw that the server responded correctly, so we naturally directed our suspicions towards the JSON parser.

The problem was that the parser failed when there were Polish characters present in the JSON data. We checked the source code for the restful_mapper library to check how it uses libiconv to convert character encoding. It turns out that it defines a variable string local_charset = ""; in utf8.cpp, but there’s no external access to it. It passes this variable to libiconv for character encoding conversion. On Android, the conversion fails with the EILSEQ (Illegal byte sequence) error. However it seems to work just fine on iOS.

So we looked inside libiconv to see how it handles a situation where one of the charsets passed for encoding is "". As stated in its documentation: The empty encoding name "" is equivalent to "char": it denotes the locale dependent character encoding.. Normally libiconv checks in several places for the locale character encoding, eg. l_langinfo(CODESET) or determines it by checking the environmental variables LC_ALL, LC_CTYPE, LANG (see localcharset.c implementation for reference). If everything fails libiconv sets codeset to ASCII. And that’s what happened in our particular situation. The fact is that langinfo.h is not part of NDK and Android does not support locales. So every libiconv android ports we found (we used one from Danilo’s Tech Blog) simply ignore the locale character encoding check and set codeset = "ASCII" immediately. It seems that on Android you have to explicitly pass both charsets to libiconv.

One possible solution to this would be to modify the restful_mapper by adding a method to set local_charset from outside the library. Then, in Java, we’d get the system’s default charset as a string by calling Charset.defaultCharset().displayName() and passing the value to the C++ code via JNI. For now, we simply patched utf8.cpp by changing the problematic line of code to string local_charset = "UTF-8"; and recompiled the library.

Java Round-up

As you can see, thanks to Djinni integrating the native library with Java code becomes fairly easy. We did not get any errors with either JNI or generated Java code, in fact, we did not even have to worry about the dreaded JNI part. As stated earlier, debugging an Android app with native (C++) code is very problematic and that’s where most of our efforts were spent. Setting debug.checkjni 1 helps a lot, but when you come across problems with the native code, you have to prepare to spend a lot of time figuring out what’s wrong.

Conclusions

As we can see, sharing code between iOS, OS X, and Android is possible. In fact, it works just fine and after having at least one go at it, the whole setup becomes much less painful.

The Good

We got what we hoped for.

implement once – the code is reusable on multiple platforms
fix once – if a bug is found, there’s no need to fix it in all apps separately (and more importantly, lose time doing it more than once)
free improvements – any improvements made to the library, e.g. its performance, will benefit all platforms

Employing this approach has additional positive side effects, like improved performance. C++ is known for its speed. If our app had some computationally-intensive operations to perform, we may kill two birds with one stone here.
Code decoupling would be another improvement. Such a library is implicitly forced to be decoupled from any application code that we write. It’s independent and standalone, and as such it clears the path to another advantage, namely the facilitation of test implementation.

The Bad

But it’s not all peaches and gravy. There are some disadvantages that we need to be aware of.

C++ skills – the library we created is written in both C and C++. C++ is often considered convoluted and difficult, which may push many iOS and Android developers used to their native languages out of the comfort zone.
glue code – Swift and Java need it, Objective-C can do without it, but does better with a layer that channels the communication between C++ and the native language. However Djinni helps a lot in this respect.
a fatter app – libraries, as any code, make the app bigger. This should be kept in mind especially if the library reimplements functionalities that are already available in the system’s standard libs. Thus you should contemplate on what you want to handle with the library, and what parts can be delegated to the native code.
debugging – debugging with precompiled libraries is more difficult.
compatibility-breaking changes – you have to remember, that any changes made to a library will touch all apps that use it. So if you make a backwards-incompatible changes, all apps that depend on the library will need to adapt and conform to them.

The Ugly

Such a setup introduces a certain level of complexity. We need to prepare an environment to compile the library, take care of the dependencies and have them compiled, configure Djinni, hook all of that up to our project, and if we’re using Swift we also need to do the exception-handling-dance.

There’s definitely a lot that can be done to improve the process and make it more CocoaPods™-like friendly. Nonetheless, cross-platform libraries are a valid choice to consider.

Michał Hernas

Really cool article! I will try it on in future. Do you have any hints where to look to use C++ code in JS? I am searching for a way to use some basic computation lib (to ensure no rounding issues) written in one language in Obj-C, python (cpython?) and javascript (??)
- Jarosław Pendowski
  
  As for C++ and JS you could try using V8. But the binary can by quite large from what I remember.
  On the other hand if you want to use Objective-C, you have JavaScriptCore which have pretty easy to use Obj-C JS bridge.
  - Michał Hernas
    
    I know, I was searching for some smart universal solution and I am still checking all possible leads, at the end the best for us was to create implementation in Python, JS and Swift. We just put one set of fixtures (json) for tests and run the same test cases on all different implementations. Not the best approach for geeks, but the best in time/effort.
    
    About article, did you use this approach in some project or was it only made for blog post? I am curious is it really worth to setup all build processes, bridges and debugging? And then as mentioned in article, debug Android why ssl connection is not working, why json is not parsed etc.? How big project would have to be to make effort worth it I wonder.

Tags

Archives

Authors

It’s all about

POSSIBILITIES

HOW?

Language

Tools

libcurl

restful_mapper

Glue Code

Djinni

IDL

Generating Glue Code

Library

The Core

CURLPP

restful_mapper

An iOS / OS X Static Library

A Shared Library for Android

The OS X Objective-C App

Asynchronous Requests

Exceptions

Objective-C Round-up

The iOS Swift App

Asynchronous Requests

Exceptions

Swift Round-up

The (Android) Java App

Java Round-up

Conclusions

The Good

The Bad

The Ugly

Sign up for our newsletter

Tags

Archives

Authors

It’s all about

POSSIBILITIES

HOW?

Language

Tools

libcurl

restful_mapper

Glue Code

Djinni

IDL

Generating Glue Code

Library

The Core

CURLPP

restful_mapper

An iOS / OS X Static Library

A Shared Library for Android

The OS X Objective-C App

Asynchronous Requests

Exceptions

Objective-C Round-up

The iOS Swift App

Asynchronous Requests

Exceptions

Swift Round-up

The (Android) Java App

Java Round-up

Conclusions

The Good

The Bad

The Ugly

Further reading

8 Tech-Related Blog Posts You May Have Missed in Q1 on the Macoscope Blog

Simplify your life with fastlane match

Sign up for our newsletter