Processing plugin exports in the background

Hi everyone,

We’re looking into ways in which we can improve export time at the moment. There are a few small improvements we can make without any impact on your plugins, but there is a huge improvement we can make - but it’ll need plugin support.

Right now, pages are exported on the main thread in sequence - one at a time, one after the other. This is slow, and only allows us to use a single CPU core to do it all. If we can move the export off to background queues then we can make the export process much more efficient.

We’d really like all plugins to adopt this, but appreciate that for some it’s not going to be a small amount of work - so I suspect it’s going to have to be another opt-in feature.

Here’s what we’re currently thinking: during export, we’ll check to see which pages have plugins that have opted in to asynchronous export. Those will be done first, and we’ll kick off a few at once. Once those are done, we’ll go back to the legacy export - one at a time on the main thread.

This isn’t going to be in 7.1 but will make it into a future release at some point. We’ll have more information for you nearer the time, but the best thing you can do to prepare is make sure that your plugin doesn’t access any UI during export. For some this is going to trickier than for others, but it’ll be a worthwhile effort. Exports are going to be lightning fast! :wink:

Various versions of RW since version 5 have called various methods on various threads. It has changed wildly, often from point release to point release, without warning.

Currently ContentHTML will be called on both the main thread and the background thread depending on the version of RW, whether PlusKit is installed, and if a QuickLook Preview is being saved (RW6).

Needless to say, this has been a constant source of bugs (and grey hair).

I only ask for two things:

  1. Specify the behavior
    If you don’t call something on the main thread, the precise threading behavior and requirements must be detailed in the headers/specs/whatever. Changes to the threading behavior should be considered API-breaking – and increment the API version.

  2. Do not block the main thread
    There are some things that are difficult or even infeasible to do without some (often small) amount of access to the main thread (like anything involving webkit).
    If you block the main thread, then the pinwheel spins, even if you do all the work on background threads.
    I mention this because for years contentHTML was called on a background thread while QuickLook preview saving blocked the main thread.