During development of Gears, we experimented with many different architectures for offline-enabled web applications. In this document we briefly look at some of them and explore their advantages and disadvantages.
As we experimented, we found some common themes. All the offline-enabled applications we created had the following design issues that needed to be addressed:
In most web applications today there is no real data layer.
AJAX calls originate throughout the code, without any single clean communication layer. This is usually fine for AJAX applications because there is only one data source: the server. In effect, the API that the server exposes to AJAX acts as a data layer.
In general, isolating the data layer is a good first step.
When you add a local datastore to your application, you will have a single place through which all data storage and retrieval requests pass.
For example, if your AJAX application issues a JSON request directly to the server to get all the accounts for a user, you might change this to instead ask an intermediate object for all the accounts for the user. This object could then decide whether to retrieve the data from the server, the local store, or some combination of both. Similarly, when the application wants to update the user's accounts, the app does so by calling the intermediate object. The intermediate object can then decide whether to write the data locally, whether to send the data to the server, and it can schedule synchronization.
You can think of this intermediate object as a data switch layer that implements the same interface as the data layer. As a first step, you can make the data switch forward all your calls to the data layer that interacts with the server. This step is useful since it is the code path that is followed when Gears is not installed or if the user doesn't want to enable the application to work offline. Note that a data switch is not strictly necessary (for example GearPad does not have a data switch layer).
The next step, as shown in the figure below, is to create a new local data layer that uses a Gears database instead of going to the web server for data. It's simpler if this data layer has the same interface as the existing data layer used to communicate with the server. If the interface is different then some translation needs to be done and you might as well do that inside this data layer.
To test this step, you can set the data switch layer to talk to this new (local) data layer. You might want to pre-populate the database to make things easier to test.
If the application is not structured with a data layer and adding a data layer is not an option, it is still possible to isolate the data layer by intercepting all the calls to the web server just before they are sent. For example, you could intercept a form submit (listen to the submit event) and decide if the application should use the local data store or the data on the server.
Implementing this approach involves finding all functions and methods that send requests to the server, and rerouting them. The challenge is that this method requires a lot of extra work, like parsing URLs, iterating over forms as well as generating the same result as the server would. In practice you end up reimplementing large parts of the web server on the client side. Regardless, this can be a viable option for existing AJAX applications that can't be otherwise rearchitected.
For practical reasons, every feature of an application may not become a feature that's available offline. You need to choose which features you want to support locally and implement the logic that decides when to use the local store and when to connect to the server. We call this the "connection strategy."
You might think that you would always want to use the local store since it is faster. However, there are many practical reasons why you may want or need to access the data on server instead. For instance:
Typically, the optimal solution is to use the local store as much as possible, since it's usually faster than a remote connection. However, the more work an application does locally, the more code you need to write to implement the feature locally and to synchronize the corresponding data. There is a cost/benefit tradeoff to consider, and some features may not be worthwhile to support locally.
One fundamental question that all offline-enabled applications must answer early is that of "modality".
In a modal application, when the application is online it communicates with the server. When it's offline, it uses the local store. Data must be synchronized when the app switches between modes.
The advantage of making a modal application is that it's relatively simple to implement and therefore a reasonable way to bootstrap the application to function offline.
The disadvantages are:
In a modeless application, the application works with an assumption that it is offline, or that it can lose the network connection at any time. The app uses the local store as much as possible, and does continuous, small data syncs in the background when the server is available. Data synchronization is also done when coming back online.
The advantages of modeless applications are:
The disadvantages of modeless applications are:
The sample application Gearpad is an example of a modeless offline application. It always writes to the local database, and then independently synchronizes the changes with the server.
Google Reader is currently an example of a modal application. It has a distinct offline mode that a user must explicitly enable. In Reader's case, implementing a modal state was a pragmatic choice, as it was faster to implement and made it possible to release an early version of Google Reader with Gears.
No matter which connection and modality strategy you use, the data in the local database will get out of sync with the server data. For example, local data and server data get out of sync when:
Resolving these differences so that the two stores are the same is called "synchronization". There are many approaches to synchronization and none are perfect for all situations. The solution you ultimately choose will likely be highly customized to your particular application.
Below are some general synchronization strategies.
The simplest solution to synchronization is what we call "manual sync". It's manual because the user decides when to synchronize. It can be implemented simply by uploading all the old local data to the server, and then downloading a fresh copy from the server before going offline.
Manual sync requires that:
The problems with this method and with the offline mode it creates, are:
Manual sync can be a good way to get started as it is relatively easy to implement. However, it requires the user to have awareness and involvement in the synching process.
In a "background sync", the application continuously synchronizes the data between the local data store and the server. This can be implemented by pinging the server every once in a while or better yet, letting the server push or stream data to the client (this is called Comet in the Ajax lingo).
The benefits of background synching are:
The downside is that the sync engine might consume resources or slow down the online experience with its background processing (if it's not using the WorkerPool). Using WorkerPool the cost of synching is minimized and no longer affects the user's experience.
There are many design choices to be made along the way to enabling an application to work offline. This document reviewed a few of the possible choices for the common design issues that arise. The decisions made for an application will need to reflect the user experience you want to achieve as well as the application's limitations and goals.