⚡️⚡️⚡️ New in RapidWeaver 7.1 ⚡️⚡️⚡️

Today we’re releasing a private beta of the upcoming RapidWeaver 7.1!

We’ve added some amazing new features in 7.1 and we think you’re going to love them. Please have a read through and let us know what you think.

We’re aiming to get a public beta out within the next few weeks, depending on how you guys get on.

Download: [RapidWeaver 7.1 Beta (17930b)] (https://dl.devmate.com/com.realmacsoftware.rapidweaver/17930b/1467026976/RapidWeaver-17930b.zip) Released 27th June

New in this release

  1. Caching of the plugins preferred file extension
  2. Revising the RWPluginEditingViewControllers protocol

Lazy Loading

Starting in RapidWeaver 7.1 we will be lazy loading plugins. What this means is that your plugin is not instantiated until it’s actually required. Using this technique we have decreased loading time by an order of magnitude. In some cases we’ve seen loading decrease from nearly 4 minutes right down to 8 seconds!

What you have to do

In most cases, absolutely nothing. Your existing plugins will just work. However, if your plugin employs some kind of wizardry to examine the RW document or plugins of other pages, you may need to do some work.

Methods to stay away from

There are some methods available through the API that will force plugins of other pages to load. It’s advised not to use these methods so please check your plugin to see if their use is absolutely necessary.

- (NSString *)overrideFileExtension

This method is currently documented only in the RMSamplePlugin project. It’s called when we build links between pages, say for example the menu bar. If your plugin implements it, we’re forced to load the plugin before returning a link, thus negating the benefits of lazy loading. In most cases this method can just be removed. We’ve just added caching of the preferred file extension so the effects of it’s use are far less noticeable on subsequent loads.

Saving

Saving a document in RapidWeaver 7.1 is now performed asynchronously! Construction of document sandwich will happen on a background thread and the main thread will be blocked. Once the sandwich has been built, the main thread is unlocked and the background thread continues on to write the sandwich data to disk.

Uninitialized or missing plugins

When building the document sandwich, we will ask each loaded plugin for their sandwich. For unloaded plugins, we simply duplicate the page’s sandwich as it was when the user opened the document.

Plugin shared data persistence

Some plugins require the ability to share data between pages. While you can easily share data internally using a static property, there has never been a good way to persist shared data. With RapidWeaver 7.1 we’re introducing shared data persistence. During save, each plugin will get the chance to create an RMSandwich containing shared data to be persisted at the document level. Your sandwich will be saved in the document with the following path.

SharedPluginData/[PLUGIN CLASS]/

A new API

To make use of the new shared data, you’ll need to adopt the RWPluginSharedData protocol on your plugin. It’s important to note that this API doesn’t give you a way to pass information between instances of your plugin, that we leave to you. This API gives you an easy way to persist the information to the document. As static objects are shared across every instance of your plugin, we pass in the current document so you can restrict the sharing of data at the document level.

@protocol RWSharedPluginData <NSObject>

@required

/// Load the passed in sandwich for the specified document. This will be called on document load if the plugin has stored shared plugin data
+ (void)loadSharedPluginDataSandwich:(RMSandwich *)sandwich forDocument:(NSDocument <RWDocument> *)document;

/// Return a sandwich containing shared data to be stored in the specified document.
+ (RMSandwich *)sharedPluginSandwichForDocument:(NSDocument <RWDocument> *)document;

/// Clear any shared data for the specified document
+ (void)clearSharedPluinDataForDocument:(NSDocument<RWDocument> *)document;

@end

Existing shared data

If your plugin is already persisting shared data to one or more pages, you should consider migrating it to use the new shared data API. To help with migration, we’re providing the following protocol.

@protocol RWSharedPluginDataMigration <NSObject>

@required

/// Move any shared data stored in the page to the shared plugin data store
/** THIS METHOD WILL BE CALLED ONLY ONCE PER PAGE
 * Only plugins that conform to the RWSharedPluginDataMigration protocol will be migrated
 * After migration, the page will me marked as being migrated and this method will not be called on subsequent loads
 */
+ (void)migratePageSandwich:(RMSandwich *)sandwich toSharedPluginDataForDocument:(NSDocument<RWDocument> *)document;

@end

During document load, we will call this method for all plugins that adopt the RWSharedPluginDataMigration protocol. This method will be called exactly once for each page and gives you a chance to read the page’s data sandwich and migrate any shared data without actually loading the plugin. Anything you do in this method should be kept to a minimum to ensure documents load quickly.

Global Settings Area

Following on from the shared data above, we have also added the ability for plugins to provide a document-global settings area. You would put anything that would apply to every instance of your plugin in a document in here - for example, you might allow the user to provide a set of API keys or specific style options.

To implement this, we’ve started at looking at ways in which we can modernise the RapidWeaver plugin APIs to allow you (and us) the opportunity to take advantage of newer features in macOS. To this end, any plugins going forward will need to implement NSViewController subclasses for their main views. We’ve added two new protocols:

@protocol RWPluginEditingViewControllers <NSObject>
@optional
- (NSViewController *)editingViewController;
- (NSViewController *)pageInspectorViewController;
@end

-editingViewController corresponds to the old -userInteractionAndEditingView and is the main content area for your plugin. If you do not provide an NSViewController in this method, your plugin will not appear in the Pages list when added.

-pageInspectorViewController corresponds to the old -optionsAndConfigurationView, and as before you can choose not to provide anything here if you wish.

@protocol RWPluginSettingsViewController <NSObject>
@required
- (NSViewController *)settingsViewController;
@end

-settingsViewController is new: if you conform to the RWPluginSettingsViewController protocol and provide an NSViewController instance here we’ll automatically add you to the Plugins area of the sidebar when added. You may also build a settings-only plugin by only returning a NSViewController instance in this method (we’re looking at you, PlusKit).

This API is a work in progress, and we’re looking for feedback before RapidWeaver 7.1 launches.

1 Like

Will 7.1 fix Smart Publish not publishing some kind of files unless a complete republish is performed?

currently i only use the extension method when a user adds php to their page but has the html extension.

you have recommended i avoid the method.

so i should just let the user publish a broken page? or should i do?

@gibo I’m not 100% sure which files are not publishing correctly. Can you drop us an email with more info and steps to reproduce? Simon mentioned you reported something a while back with publishing, just want to confirm the current issue you’re seeing.

@isaiah I’ve figured out a reliable way to cache the value returned from -overrideFileExtension so you should be able to continue to use the method without issue. We’d still prefer this method to not be used but in some cases (like the one you pointed out) it’s necessary.

###A new build…

I’ll post up a new build soon with a few adjustments to the API to clean things up a little. I’ll also update the post above with the new information regarding the changes.

Thanks @tpbradley, I’ve forwarded the original mail to you.

Don’t get me wrong: these new features coming in RW 7.1 are great, but there are several bugs that needs your attention first, as they’re causing a huge spike in support request for our plugins:

  • minor changes not uploaded unless a complete republish is performed
  • corrupted addons when moving addons folder to a different location
  • cache busting feature causing Google Maps API to fail loading
  • can’t get a local resource identifier the way I’m getting it on RW6

We’ve reported them (several weeks ago for the most severe ones) but have no way to track the status of our requests.

1 Like

Ok guys, an updated build is available and revised docs above :wink:

Download: [RapidWeaver 7.1 Beta (17930b)] (https://dl.devmate.com/com.realmacsoftware.rapidweaver/17930b/1467026976/RapidWeaver-17930b.zip) Released 27th June

I confirm small changes are correctly published now.
Good work, RMS guys!

2 Likes

@isaiah have you had a chance to look at these changes? We’d like to push this out to beta testers sometime next week…

@dan yes, i’ve been looking at it since the beginning of the week.

the good news is that the new API is workable.
the bad news is that it’s going to take me at least a few more days to get this done.

the worse news is that using the current version of Stacks along with RW 7.1 causes data corruption. that’s a pretty big deal.

is there any way to make the current version of Stacks incompatible with RW 7.1? if Stacks loads at all then all with RW 7.1 subsequent stack nodes that are placed onto the page are basically corrupted data. it’s a pretty bad situation.

i’m afraid this is probably already causing issues with this dev release.

Yeah, I’m sure there is something we can do, cc’ing @tpbradley and @simon for input :slight_smile:

as of today (about 30 min ago) i have the new doc-storage protocol up and running and storing the ID data. :thumbsup: really happy with it! works perfectly – just what the doctor ordered.

but… i’m only about halfway through this change and i think there are some challenges yet to come:

handling Stacks 2 with RW 7.1 – i’m not certain that can work at all.

  • what happens when an RW 7.1 doc is opened in RW 7.0?
    maybe not a big deal. i think this will “just work”.

  • what happens when an RW 7.0 doc is opened in RW 7.1?
    this seems a tad more challenging.

Question: for @tpbradley / @simon: is there any way to force the pages to load the old way, like when the RWSharedPluginDataMigration is called? I could really use access to the whole document that one time to gather the data that has traditionally been stored on each page.

One more thing for @simon and @tpbradley :

Bug report:

/// Clear any shared data for the specified document

  • (void)clearSharedPluinDataForDocument:(NSDocument *)document;

There is a typo here. “Pluin” should probably be “Plugin”

Question:
When is this method called and why. It’s not clear what it’s purpose is.

Hey @isaiah,

Great news on getting the new storage API working!

I’m really not sure what would happen with Stacks 2, can you send me the plugin so I can test a few things? How many people still use it?

On the other two points, do you mean the other way round?

  • 7.0 docs should open in 7.1 and just work
  • 7.1 docs may open in 7.0 but some data might be missing (anything in the global document storage)

We did consider loading the whole document the once to allow plugins to sort themselves out but things start to get messy with this approach, hence the introduction of RWSharedPluginDataMigration.
This method is called on every page when the document loads. Once it’s been called the page is marked as being migrated. It’s a class method so it doesn’t actually create an instance of the plugin, but should be sufficient to extract any information from the plugin’s sandwich and insert it into the document’s global storage with minimal overhead.

Good catch on the typo here:

  • (void)clearSharedPluinDataForDocument:(NSDocument *)document;

I’ll fix that up. The method is called on document close and allows you to remove any document specific global data should you need to.

On the other two points, do you mean the other way round?

No. I stated it correctly.

(at least for the time being) 7.1 files they will be a superset of 7.0 files. So 7.0 will be able to open them. But how to create that superset the first time… i dunno.

How it works today: as the document opens and elements are created a global NSIndexSet is created containing each element IDs in the document. When a new element is created it is then easy to ensure a unique ID is created. newID = globalSet.lastIndex + 1;

in the Stacks API one of the specified guarantees of element IDs is that they are unique within the document.

in 7.1 i will store the indexSet in the global store. yeah!
but i cannot create it the first time as i cannot walk through the existing IDs on the pages that have not loaded.

i think our options are:

  1. add some way to force the opening of all pages for migration
  2. i fundamentally change the architecture of the unique IDs in stacks.

if #1 is really tough for you guys, then i’ll get busy on #2.
it’s not impossible. but it’ll definitely add a few more days to the schedule.

I’m really not sure what would happen with Stacks 2, can you send me the plugin so I can test a few things?

Here’s a link to the archive of old versions. It’s a good thing to know about: Stacks

How many people still use it?

Irrelevant.
This is a data corruption/file loss bug. No amount of users is OK for that.

More to the point: You’ve made massive additions changes to the API. It’s not necessarily possible to build plugins that span both new and old.

You need to provide something in the API to help us out.

In the Stacks API I provide a “minimum API” and a “maximum API” plist setting. Before loading the stack I check both to ensure that the stack should be loaded at all in the current version. I’m not saying you have to do that – just providing an example of what seems to work pretty well in practice.

Isaiah

I’m afraid I still think you’re very much mistaken on this. Take partials for instance. If you open a document in 7.1 and it migrates the partials from each of the stacks pages into the shared plugin data, you save it and then try and open the document in 7.0, it’ll fail to load the partials because 7.0 doesn’t know how to load the shared plugin data. The only way it could possibly work is if all the data stored in shared plugin data is duplicated somewhere in the pages, which is just daft.

Opening a 7.0 document in 7.1 - there’s migration for that and it should just work. There’s nothing tricky here, it’s expected behaviour.

For element ID’s, well IMO ID’s of this nature should be of the UUID variety. It solves a whole host of problems like the one you’re describing. When the document loads you do have the opportunity to perform a lightweight decode of the page’s sandwich in this method

+ (void)migratePageSandwich:(RMSandwich *)sandwich toSharedPluginDataForDocument:(NSDocument<RWDocument> *)document;

Are you for some reason unable to obtain the ID’s at this stage? It is called once for every page in the document, passing in the same sandwich we’ll pass to the plugins -createWithSandwich: method.

I have to say I completely disagree with your response to how many people use Stacks 2. It’s absolutely relevant because if only a handful of people use it, it’s not cost effective to spend hours writing code to cope for it - that’s just good business sense. I’ve just tested Stacks 2 in 7.1 myself and found that it does indeed run fine. Even the user stacks work, though I noticed they are created in a hardcoded path under the application support folder. Not something that will work correctly if the user relocates their addons folder. If there’s something in Stacks 2 that you’ve identified doesn’t work, let me know.

It should be fairly straightforward to create plugins that work against both 7.0 and 7.1. When the doc loads the following method will be called allowing you to create some global property for your plugin with data from the sandwich.

+ (void)loadSharedPluginDataSandwich:(RMSandwich *)sandwich forDocument:(NSDocument <RWDocument> *)document;

When a page plugin is instantiated, just check if the global property is set and you’ll know if your on 7.0 or 7.1 - If you’re on 7.0, create some shared property if it doesn’t exist and populate as you do now.

wow. ok. well, i’m sending the file corruption users over to you. :stuck_out_tongue:

@tpbradley Is this supposed to work in latest RW 7.1 builds? I’m trying to get this called with no luck.

Stacks 3.2 beta 1

Extra notes just for @tpbradley :

  • there is no need to block earlier versions of Stacks < 3.1 from opening files created by Stacks > 3.1. i managed to cobble together the backwards compatibility. features will differ between versions – but no data is lost. :tada::metal:
  • i’ve tried to maintain feature parity as close as possible as per request. the only difference is that partials now persist until the user deletes them. this is a much requested feature – so it’s all win. :tada::raised_hands:
  • if you have files created with earlier betas (files that have already run the one-time-only migration) then partials will just be picked up into the global store whenever they’re loaded in a page. which is A-OK, just letting you know for your own testing.

i’ve released this on my slack channel, and via auto-update to users already using a beta – but not yet to the general public. since there are no real differences for 7.0 users there’s not a real need.

assuming testing doesn’t uncover a show-stopper i think we can release this as final in the next day or two.

Download: http://yourhead.com/appcast/RW6/beta/Stacks3/Stacks_3.2.01_2997.zip
Release Notes: http://yourhead.com/appcast/RW6/beta/Stacks3/release_notes_3.2.01_2997

1 Like

@gibo: can you message me with a copy of your plugin and I’ll see what’s going on?