@markbuehl: That is one ginormous file. My guess is you will have continued problems. 2,000 pages is a lot but my guess is part of the problem is not the number of pages, but that you also have a lot of media stored within RW: video, audio, images.
With something this large there are probably a few steps you need to take to get things back under control. But I’d suggest the first step would be to “warehouse” your media. This means to put all your media on your server directly, and then link to them within RW.
One way to warehouse may be through the Repository stack (especially for images) or a traditional FTP app such as Transmit or Forklift. This long discussion has more details in places:
In addition this PDF guide from Will Woodgate may be quite useful:
I don’t know if it’s the newest version (1.0) but it’s simply the one I could find the easiest.
First, thanks for the comments. Second, this large project opens in a minute or two on my iMac. Opening and working with the large project is not the issue. No issues there. The only issue is how long it takes to convert the stacks into HTML code, now over 15 hours. I guess I have to live with that since I do not have a lot of images yet, mostly just links and data.
In researching what I wanted to use for this project, I did not find any information on RapidWeaver limits to the number of pages. Or if it was good to use on large projects. I knew going in I would have a large number of pages since I wanted a page for every community in Wisconsin, 3,438 pages to start with, plus history, lodging, restaurants, etc. Since the early 1990’s I have used a number of Website development programs and I like the idea of stacks to speed up development. I do not like Dreamweaver after using it for years or BBEdit for this project. Nor did I want to use WordPress or any online solution. So I guess I am using RapidWeaver for this project unless someone suggests something just as fast to develop a large number of pages. Sometime in the future once I get a base, I might start coding some pages in Dreamweaver or something like that again, maybe.
That way you could build a single community page (or perhaps a handful) that is filled in with the community data, images, and links from the database. This would hopefully mean a much smaller site that opens in second and publishes in minutes that is much more easily maintained – simply changing, adding, removing data from the database would update your site – perhaps without ever touching RW.
If the idea of an online database is distasteful – and there are really valid reasons that it may be – then a similar system can be built using a “static site generator”. This lets you manage content by simply editing text files on your server. This has all the benefits of a database – but just uses files and folders to do the trick.
Anytime you start doing repetitive tests – especially into the many hundreds of repetitions – and definitely when the file sizes grow over 1GB, it’s time to ask: Is this the right tool for the job? Is there another way to get this done?
Whatever you decide, the next time you have to wait 15 hours to publish your site, perhaps use some of those hours to research ways other ways to accomplish this task. If you post more about the details of your project I’m sure many here would have good ideas to help you on your way.
I don’t normally recommend hiring consultants – I like to think that most projects can be accomplished with a bit of effort – that’s the best part of RapidWeaver. But this sounds like a really big project – with a lot of time invested – perhaps investing a bit of money in hiring a consultant to help re-architect this massive beast into something a bit less beastly. There are lots of good for-hire consultants here on the forum and, in this case it would probably be money well spent.
There are a few folks that have quite large sites with rapidweaver. Most have broken the sites into Multiple project files. Might make things a bit more manageable as well as faster.
You may want to consider searching the forum for some hints.
First, carefully consider what everyone else here has said. There is a lot of wisdom and expertise involved.
Secondly, the site I maintain is 1500 pages and it is easy to maintain because I have it broken down into several separate project files. But the TOTAL size of all project files is about 550 mb. Why is your 2k pages 34 gb??? Isaiah is right… that is insane.
Third, I can’t remember the last time I updated (published) the entire web site. Something is added every day so I do publish daily but it is generally one or two new pages and 2-3 updated pages. I just use single page publishing. Navigation changes or anything else (partials, etc) that effect the entire site are rare so there is no need to upload the entire 1500 pages to add or change one. I have the site broken down into “reports,” “store,” “directory,” and such. And at that, if I need publish an entire “section” (project) I publish locally and then use ftp. But I reset, 90% of my daily “publishing” (upload) is a single page here… a single page there… You just need to consider if your change is affecting one page, a “section” of the site, or the entire site.
But again… consider the advice here. A 34 gb project taking hours to upload… that is just not right. Period.