Multicore/multiprocessor support please!

@webdeer, @ashleykaryl Ok, well if I run a preview of the “Banded Pages and Angles” page from the Sections Pro demo, it takes a total of 18 seconds before showing anything. 14 seconds of this was spent exporting extra files needed by the plugin. Weather this is down to the Stacks plugin or the Sections Pro stack I don’t know. All I know is 4 seconds was taken by RW and 14 inside the plugin so I’d say there is definitely a problem here. Running the same process through Instruments confirms my suspicions. During loading and exporting Stacks is sat at 100% CPU for a considerable time. There is most definitely a problem here. If you’re seeing other problems, please let us know the steps to reproduce it and send us your project files so we can analyse it.

2 Likes

I just timed that page with a 2010 MB and using RW6 it took 35 secs and with RW7 it took 37 secs before the page appeared!

@webdeer I’m not sure what your point is here. We may well have added 2 seconds to the processing time in RW7. If we were able to rectify that there would still be a 35 second delay. Out of the 35 seconds, how much time is spent in Stacks/Sections Pro and how much time in RW? I suspect 90% of the time is on the stacks side.

1 Like

Just confirming what I have thought was the case but now having timed the two, RW6 is indeed quicker. I thought that RW7 was supposed to be quicker but it never felt quicker to me.

I haven’t timed any previewing before but certainly 30 seconds odd to preview a page feels about normal to me. Any page with lots of content takes that sort of time.

I don’t have any tools here to decipher which part is causing the delay but if Stacks is taking up 90% of the time why is it not allocated more resources? I cannot imagine there are many people using RW who are not using Stacks virtually all the time and I see slow performance all over the place.

I used to work on Freeway with a lowly G4 400Mhz without spinning ball and in Rapidweaver I am seeing up to 30 seconds of delay previewing html pages with a few web resolution Jpegs that weigh less than a 1mb in total. If a computer like mine can kick around huge multi-layered files in Photoshop and simultaneously export HD animation from Maya it’s more than enough for web work.

Oh I completely agree guys, designing a web site should be an easy task for any modern system. And by modern system I mean anything that can run OS X El Capitan. RW 7 is most definitely faster for doing a lot of things but we’ve also added more functionality too. One of the new features is the introduction of a fully working php web server. This will definitely add a little processing time but this shouldn’t be significant.

We are however at the mercy of plugin developers. We can’t stop them from writing slow code or performing long running tasks If they decide to do so. What we can do is work with them to improve performance, which is what I intend to do.

@ashleykaryl I’m afraid it’s not a simple case of just allocating more resources. The plugin has access to all the resources a machine can give - it’s up to them to make use of it.

Surely that is also limited by RW not being multicore enabled, so in this case only 1/16th of my CPU is available to RW and shared with all plugins. I believe Photoshop handles a lot of tasks purely with Ram and offloads certain operations to the graphics card where appropriate. A computer with 32 gigs of ram should never see spinning ball previewing a web page.

@ashleykaryl There’s actually no real distinction between single/multi threaded apps on OS X. On a windows platform you used to enable your app to be multithreaded using a particular threading model. In OS X this is not the case. Every application is multithreaded but code needs to be written to make use of this. For instance, if I need to process 1000 items, I could do them one by one (single threaded) or tell the system to process them asynchronously (multithreaded). The system decides how best to process the items for that specific machine. So in your case it would most likely process 16 items simultaneously.

So RW and plugins do have every core available for processing tasks but the code needs to be written to make use of it. Because the export process has historically performed operations in a single threaded manor, parts of the export process have become reliant on this. Simon has been working towards fixing this so we can export multiple pages simultaneously.

What you’re referring to with Photoshop is GPU processing. The GPU is incredibly powerful at performing the same task across a large chunk of data. Manipulating an image is a perfect example of where to use the GPU. The system would break up an image into say 64 squares, then submit them to the GPU where all squares would processed simultaneously, recombining them to produce the finished result. I’m afraid RAM doesn’t really play a part in processing here, it’s merely a storage of data.

Typical time for me to preview a page in RW7 is 4 seconds. Seems reasonable. All of my pages use Stacks, although typically only a few (Markdown, Glider, Jump, Litebox, etc.). I typically have a few embedded videos and warehoused images. So stacks with simple pages seems to work fine when previewing. I don’t know if it’s slower/faster/same than RW6. I’m using a relatively underpowered MBA. On my more substantial iMac the preview time typically takes under 2 seconds, many times almost instantaneously.

Some folks are using pretty darn complicated combinations of stacks these days: so I would also expect those to preview more slowly. 30+ seconds sounds like a long long wait to me. That would drive me nuts.

I have been playing around with the Sections Pro demo extensively and have found that the each grouped Sections Pro stacks adds its own time to the previewing of the page. If you delete them one by one the page loads progressively faster. I have been using Sections Pro in a few of my pages and it does seem to be adding quite an overhead to the previewing of those pages.
I am a big fan of Big White Duck’s stacks and in particular his Sections Pro but there is obviously something he has missed with it’s release. I know in another forum on slow loading times he maintained that it didn’t add much of an overhead to the preview times but the doesn’t seem to be the case in reality. I’ll stick my neck out and tell him, I live in rural New Zealand so he will never find me…

I think there may be a misunderstanding here about Sections Pro. It is undoubtedly one of the more challenging stacks for RW in edit mode, but this is primarily because it is so aggressive in focussing on the final output being reduced to only what is actually selected.

The speed and quality of the published page should dictate priorities, even if this does require more processing in edit mode. While edit mode parsing is a CPU expensive task I really don’t see why this should generate the spinning ball in the modern day. Blocking processes should be a thing of the past as they are in all other modern software.

The logic is there to reduce the amount of redundant css written to the final page to virtually nothing; something that users have constantly requested. This is really just highlighting deficiencies in RW and I would point out that I’ve seen spinning ball and page preview delays in projects without a single Foundation stack.

I’m not sure why a stack would be challenging in edit mode? We keep getting back to the whole multicore support that is desperately needed as Stacks in particular gets more complicated. When Rapidweaver is faster on a 2011 Mac mini than a 2xXeon 2.93GHz Mac Pro (I even upgraded the processors last week!) something is very wrong. In all honesty we cannot but the blame on Realmac, they didn’t have to make it a plugin architecture but I doubt if it would have survived it they hadn’t. Looks like you have to buy an iMac to get the best performance, the ‘new’ Mac Pro is a lowly 11th on the 64bit single core benchmark…

Out of interest whilst testing the Sections Pro demo today, I noticed Rapidweaver being shown as 'unresponsive’in the Activity Monitor CPU window for well over 10 seconds before it continued to preview a page. Strange?

I don’t think there is anything outlandish that Sections Pro is trying to do, merely that RW is not currently able to handle the load and that’s a problem with RW.

On the subject of iMacs Vs Mac Pro I’ve had both and for serious production work I’d choose a Mac Pro every time. If I was just writing emails and surfing the web etc I’d get a Mac Mini and save some space. Are you pleased with the processor upgrade?

I have noticed Rapidweaver being flagged as ‘unresponsive’ in Activity Monitor for around 10 seconds when previewing the Sections Pro demo project so there may be more to this than a lack of multicore support (which is still needed).
I know what you mean about the iMac vs Mac Pro, I have had both and found the iMac too limiting and would probably struggle with the new Mac Pro as I have 5 internal drives, two Blu-ray/DVD drives and three external drives plus SD card reader and more in/on my Mac Pro.
The processor upgrade was a bargain in my eyes and was $60US well spent on two used 2.93 Xeons from a reputable US seller, it gives it a noticeable boost in everything apart from Rapidweaver! They were even easy to install.

@jspencer2 you do know the demo project is not meant for a real world case, right?? I highly doubt these issues have anything to do with the stack itself.

Firstly, I’d like to say thanks to the people that have commented in support of Realmac. But I have to say I’m getting a little bored of this now. There are far too many assumptions being thrown around without any actual evidence.

All plugins/stacks etc run inside the RW process so if they go slow, RW goes slow and it’s reported as such in Activity Monitor.

During preview & export we ask the plugin for any additional files it requires for the current page. I can measure the time the plugin takes to do this by adding some simple logging around the method call.

NSLog(@"Start");
pageSpecificFiles = [plugin extraFilesNeededInExportFolder:params];
NSLog(@"end");

The result of this is

2016-07-11 08:49:22.380 RapidWeaver 7[53561:715994] Start
2016-07-11 08:49:37.788 RapidWeaver 7[53561:715994] end

You can see from the log that asking the plugin for extra files took 15 seconds to complete. RW has no control over what the plugin does here, we can’t fix it or allocate more resources or anything.

From using Apple’s profiling tools I can tell you that 69% of the 15 seconds was spent on performing regular expression operations.

If you have any evidence that contradicts this please let us know so we can improve RW.

You can email us at support@realmacsoftware.com with any information or sample projects.

5 Likes