RW9: The all new RWKit!

RapidWeaver Developer Network
1

Ok guys, this is it!

RMKit and RWKit have been combined into a single framework called RWKit. I’ve stripped out everything I possibly can, simplified the interface where possible, added more comments and a bunch of new API’s.

We’ve gone from around 217 source files and 237 headers to just 21 source files and 33 headers so this is a massive, massive change from RW8. The vast majority of removals have either been old unused code or code that supported our own internal plugins.

Believe it or not, our sample plugin still compiles and runs in RW9 with minimal changes.

Because this is such a heavy change I’d like you guys to do one thing for me.

  1. Make a copy of your plugin project
  2. Remove RMKit and RWKit
  3. Add the new RWKit
  4. Compile
  5. Let me know your compile errors

Grab the new RWKit here https://share.getcloudapp.com/nOu8RyYL

Feel free to take a look over the headers and let me know if anything is missing - it’s really tricky figuring out what plugins need.

I’m aiming for a high level of backwards compatibility with RW8 and more work will be done on this over the coming weeks.

A couple of notes

RWKit is now a module so you can use @import RWKit;
RWDocument is now known as RWDocumentProtocol
RWPage is now known as RWPageProtocol
RWLinkStyle is now known as RWKitLinkStyle
RWStyledTextViewDelegate has been removed, it’s methods are no longer required (as far as I can work out)

Cheers
Tom

2 Likes
2

started with Stacks 4.0 branch. builds with zero errors, zero warnings.
i made changes to swap the framework and adjust the framework search paths

initially only changes due to modules. i had modules disabled. so i adjusted projects settings there.

results:

image
image946×1324 222 KB

mostly just missing .h’s – which is to say, “tip of the iceberg” error messages.
each of these likely represents several calls to classes that no longer exist.

it’s pretty overwhelming. not sure where to begin, but i’ll just detail a handful.
starting at the top…

  • RWMarkdown.h is gone. whelp. that sucks.
    i guess i can include my own markdown lib. i had been working on that in another side-project anyway, but will take some time.

  • RWTextAttachmentCellWrapper.h is gone
    this was the wrapper around images inside of styled text views. it used to be that i had to forward delegate methods from clicks on those things in order for styled text stuff to work well.
    any docs/change-notes on how to handle that now?

  • All of the categories for creating temp-files are gone
    i guess i can just create my own using foundation/appkit methods, but it was nice to have a unified app-wide way to handle that.

  • RWSiteFolder.h is gone
    i’m not sure how to get access to resources without that. any change notes around that stuff?

  • string categories for creating “friendly” filenames seem gone
    i guess i can write my own, but it was nice to have an app-wide way to do that

my initial takeaway:
ouch. i hope that some of these things are just hidden, name-changed, or my build settings are off. but, if all these things are actually gone, then i’d guess 6-9 months to fill in all the holes.

any docs/notes/details coming about how to migrate?

2 Likes
3

Thanks @isaiah!

This helps a lot, I can finally start to see which parts of the API plugins are using. Try not to feel overwhelmed by the long list of errors, in reality there’s going to be far less things to fix.

So, a few things to look into if you have a moment.

RWMarkdown
This class only defined 2 methods, both have been moved to RWAbstractPlugin. I can look at adding these to RW8 if that makes things easier?

#pragma mark Markdown Support

/// Returns HTML for the given markdown string
/// Availability: RW9
+ (NSString *)htmlForMarkdownString:(NSString *)markdownString;

/// Returns
/// Availability: RW9
+ (NSDictionary *)metadataForMarkdownString:(NSString *)markdownString;

RWTextAttachmentCellWrapper.h
Still there but it’s missing from RWKit.h, I’ll get that fixed up.

RWSiteFolder and RWSiteResource
There’s a far more capable and simper protocol for accessing resource, check out RWResourceProtocol.h
Along with these two methods on RWDocumentProtocol

/// Returns the root resource containing all other site resources
@property (nonatomic, readonly) id <RWResourceProtocol> _Nonnull rootResource;

/// Returns the resource for the given identifier if it exists
- (id <RWResourceProtocol> _Nullable)resourceWithIdentifier:(NSString * _Nonnull)identifier;

String / temp file categories
There were tonnes of these little helpers, and while they were indeed helpful there were far too many and mostly unused. Let me know which ones you need and I’ll either add them or give you the code.

Docs
Yes I’d like to get some good docs together on how to migrate/use the API but I need to know what’s busted first.

4

cool. easy.
just my opinion… RWMarkdown sure made a lot more sense from the outsider’s perspective.

getting resources working well took for. ev. er. – changing it completely with no step with new/old api working is going to be very difficult. i’ll have to do all that work again – then verify that both work in each version of RW – that means running tests in one app, then re-running again in older app.
again, just my opinion: it would make it so much easier on plugin developers if there were 6 months where both worked, if for no other reason it would mean a vastly less complicated testing during this migration.

if you have any advice on how to approach getting nine plugins migrated and working in both RW8 and RW9 please let me know.

  • i used the temp file stuff in every plugin - it just makes sense that RW coordinates that to me
  • i used the file naming stuff – very helpful for automatic naming of image files.
  • i pass on ALL of the little text encoding stuff to the Stacks API. the stack developers use these methods extensensively.

it looks like Keith published NSString+RMKit.mm with open source headers in 2008. i’ll probably just grab those old copies, write tests to compare them to RW8, fix bugs, etc. etc. – but easier than rewriting from scratch.

still, all this is just a whole bunch of work. time/money i don’t have. would be great if this stuff just still worked.

everything? or close to it.
tons of little protocol name changes, moved stuff, gone stuff, changed stuff.
so much work. so many new bugs. so much pain.

5

RWMarkdown / Classes in the API

So the problem with RWMarkdown is that it’s a class. Ideally there should be no classes in the API at all. As soon as a class has been added, it’s incredibly difficult to change without breaking anything. Just look at the history of RW releases, every release has broken plugins. Using protocols means I can change the underlying classes entirely, as long as it conforms to the protocol. It gives me complete freedom to add features, fix bugs and create a more stable app while maintaining 100% backwards compatability with plugins.

Resources

I know resources took you a long time to implement, I’m not surprised given the mess they were in. The resource API was shocking, full of bugs, spread across multiple files, awkward to use and most of it was private leading you to reverse engineer the API to make things work.

My advise at the time was to hold off because I knew it was going to be completely overhauled. You must have known that going ahead against my advise would mean a re-write in the future. You’ve had nearly 2 years to tell us what you’re doing and to try find an alternative.

There’s no way I can implement a duplicate API that mimics RW8 in RW9 - it would require a huge amount of complicated code throughout RW, and I believe Stacks 4 is the only plugin that would require it.

How do we move forward? I’m fairly certain I can add support to RW8 for a plugin compiled against RW9 API. So you would make your plugin work with RW9 API, send it to me and I’ll make it work in RW8.

Helpers

I would seriously advise against trying to reverse engineer any functionality, especially based on headers posted 12 years ago. If you need NSString+RMKit.mm you can have it - but I wouldn’t advise that either. Most of the code in this file has become irrelevant with modern web technologies. Anything useful has been moved into RW.

Tell me exactly what methods you require and I’ll look at making a public protocol for them, if there’s no suitable alternative already in the API.

6

sure. ok. i mean – i don’t agree – like, not at all, but whatever. i have zero interest in debating that.

i was only suggesting that the markdown methods were easier to find when they were in a file that had “markdown” in the name.

YES!!! please please please.

that’s is literally the only thing i asked for. please make a RW version where both new/old API exist – for as many of the changes as possible.

at the end of the day that’s all deprecation really is. and deprecation is the only request i made. and the only thing i care about. :smiley:

I think you misunderstood me.

some bits of realmac code were published as open source about a decade ago. not many people had access to the code, but to those that it it seemed like it was open.

the last copy of RMKit.mm i have is from 2009. it’s complete, with all the encoding stuff inside. it has the following header:

//***************************************************************************
// Portions Copyright (C) 2004, Keith Anderson
// Portions Copyright (C) 2005-2008 Realmac Software Pty Ltd
// Permission is hereby granted, free of charge, to any person obtaining
// a copy of this software and associated documentation files (the
// "Software"), to deal in the Software without restriction, including
// without limitation the rights to use, copy, modify, merge, publish,
// distribute, sublicense, and/or sell copies of the Software, and to
// permit persons to whom the Software is furnished to do so, subject
// to the following conditions:
// 
// The above copyright notice and this permission notice shall be
// included in all copies or substantial portions of the Software.
// 
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
// EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
// MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
// IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR
// ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF
// CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
// WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
//***************************************************************************

i obviously won’t use it if @dan doesn’t want me to. i’m not trying to reverse engineer anything, or steal any code. it just seemed pretty open-source-ish and it definitely solves the problem at hand. so i thought i’d mention it.

as for which bits i use:
the Stacks API provides pass through for, I think, the full set of string encoding methods that RW provided. It’s documented here: /stringEscaping/ · Stacks API

7

Ok great

Resources
So what I’ll do is make the sample plugin do some things with resource, compile it with RW9 and make it work in RW8. Should give us a good starting point. I’ll let you know when I have something working.

String Encoding
Can you let me know the exact methods you make use of. I could make an educated guess that

%id=textInput -encodeFile%

actually calls

-[NSString stringEscapedForFilename]

But it is purely a guess, specifics would be far more helpful.

8

i know you’d like a full list of all the API that i’m currently using that’s currently broken. if you have some way of finding that info out, i’d be happy to provide it.

there are currently several hundred errors – most of which are just missing filenames and renamed protocols. i’ve started the arduous task of just cleaning those things up. but with so much broken, it’s hard to gauge progress or know how much more of the iceberg might be below the surface unseen.

but here are the text escaping methods i’m using currently:

- (NSString*)stringEscapedForInclusionInSingleQuotedPHPString;
- (NSString*)stringEscapedForFilename;
- (NSString*)stringEscapedForHTMLElementText;
- (NSString*)stringEscapedForHTMLAttribute;
- (NSString*)stringEscapedForXMLElementText;
9

Awesome, that’s exactly what I needed.

These two methods

- (NSString*)stringEscapedForHTMLElementText;
- (NSString*)stringEscapedForHTMLAttribute;

simply call

- (NSString*)stringEscapedForXMLElementText;

All the code you need to get string encoding working again is here. It’s not the worst code but it’s non-ARC. We don’t need this stuff in RW anymore so it’s better to just include it in your plugin.

10

we’re probably going to introduce bugs, just so that this bit of code can jump from RMS’s github to YH’s. you’re cool with that? that’s what you want me to do?

sorry, i’m just double checking. i didn’t expect that response. afraid i’m misunderstanding something.

11

Yes, copy it into your plugin. You may want to alter the method names, maybe prefix them with YH so they don’t conflict with RW8.

That file was over 1000 lines of code, and I’m fairly certain that only 140 lines of it is in use by Stacks. We don’t have the capacity to maintain such a huge amount of unused code, especially when we’re not making use of it internally. Less code == less bugs :wink:

12

removing code is great. fewer bugs means happier users. and that’s the goal.
but this is moving, not removing. users will still be running this code, right?

my users are your users too.

13

Sure but it’s not where our focus should be, we can’t continue to support code that just one plugin uses. Every time Apple releases a new OS we need to make sure all this code works as it did before. This becomes practically impossible when I don’t know which plugins to test and when/how they call these methods.

I could write a bunch of unit tests to take care of things, but over time plugins will change what they use. What if there are no longer any plugins using these methods? How long do I spend maintaining it at the expense of new features? It just doesn’t make sense.

If there’s a common use case for some code then I’ll happily add it. I’d love to hear from the other plugin devs if they’re using anything in NSString+RMKit.h?

14

So I have something working! A plugin compiled against RWKit for RW9 can access site resources and add new folders / resources.

@isaiah, does the RWResourceProtocol cover everything you need to get site resources working again or is it missing anything?

Cheers

15

Sure but it’s not where our focus should be, we can’t continue to support code that just one plugin uses. Every time Apple releases a new OS we need to make sure all this code works as it did before. This becomes practically impossible when I don’t know which plugins to test and when/how they call these methods.

i’m not sure why you keep telling me this. you’re preaching to the choir, dude. i’m all for it.

my plugins have been through a bunch of big RW API changes. the 64 bit change was huge. everyone had to work together to make it seamless to customers. i shipped plugins with plugins inside of them. it was pretty wild. i think it gave @nikf an ulcer. sorry @nikf.

progress. teamwork. it’s a good thing. let’s do this!

i just asked for one thing. a deprecation period.
just leave your old code in for maybe 6 months after the change.

actually a single build would be sufficient.

so, in answer to your question, “how long”: 1 build. 1 day.
that’s it. i’m totally serious.

why? because i have 2000 errors in my build right now. it will be 8 weeks minimum before i get things to compile.

deprecation allows devs to work on migrating one bit at a time, fix the few bugs it creates, then work on the next bit, etc. biting off a little at a time, instead of choking on the whole thing at once.

making a huge API change all at once is crazy.
doing it without deprecations is crazy.
knee capping your largest plugin developer is crazy.

it’s just a lot more crazy than i’m used to – that’s all i’m sayin.

but, whatever. i’m just going to get on with it. i’ll let you know what i find.
might be a while,tho – after a week i only knocked off a few dozen errors. so, like only 1,964 to go… yeah.

Isaiah