The TV IV:Proposals/CPU Issue

A problem has arisen that could threaten the site's existance. We are located on shared hosting, and I've received a warning from Dreamhost that our account is using more than it's fair share of CPU. We are using more than 100 CPU minutes per day. We should be using something like 40. If we can't get it trimmed down, "your account may be moved to an evaluation server which could cause downtime." I'm not exactly sure what that would mean, but it sounds like trouble. I've looked into using some kind of PHP caching like Turck MMCache, but they all require that PHP run as an Apache module (which is possible), and that you modify the system php.ini (which isn't). I've tried a couple of other things like disabling full text searching, but that has barely made a dent. I'm bringing this up hoping that we can brainstorm a solution.

Possible solutions:

• Upgrade to Mediawiki 1.5, which was released recently.
  • It may have better optimization.
  • It may fix the bug that forced us to turn off client-side caching. (The epoch Last-Modified header problem)
  • Pro: We should probably do this anyway.
  • Con: My gut feeling is that this won't do much to lower CPU usage.
• Disable Calendar extension so that we can re-enable parser caching.
  • In order for the calendar used on the Main Page to dynamically update, I had to disable Mediawiki's parser cache.
  • Pro: I suspect this will make a big difference.
  • Con: Maintaining the front page calendar will become much more difficult.
• Move site to dedicated hosting.
  • Pro: CPU usage won't matter.
  • Con: Money. Costs about $100/month.

Updates

• On 12 Nov 2005, we were moved to an "evaluation server." We have tried several things, but nothing has helped the situation. At some point, our account at Dreamhost could be suspended. It looks like a dedicated server will be our only solution. If we stay with Dreamhost, the cost will be $99 per month, but only if we pre-pay for a year. Our best bet is to start on a monthly or quarterly plan, then try to raise enough money to pre-pay a year. You may have already noticed the Google ads on the site. They will help, but they are not enough. Other income sources will be needed. I welcome any suggestions anyone might have about hosting and fundraising. --CygnusTM 08:38, 21 Nov 2005 (EST)
• Thanks for the suggestions from everyone. We'd like to stay independent, so we've started to raise money to moved to dedicated hosting at Dreamhost. I've temporarily put up a donation link. This will be used to generate enough money to pre-pay for a quarter of hosting. The Google ads, upcoming Cafepress store, and Amazon (see the new Template:ASIN and Template:ISBN) and Dreamhost referrals should be enough to sustain the site after that. So we don't think we'll have to resort to constant grovelling for money. --CygnusTM 09:58, 1 Dec 2005 (EST)
  • Where do you want us to place the purchase links? BTW: I modified the ASIN template to say DVD instead of book. We should have another version that says CD. DCEdwards1966 00:05, 3 Dec 2005 (EST)
    • Thanks. I took the DVD out because an ASIN could be anything. (CD, book, DVD, etc.) Use the templates anywhere they are appropriate. That would probably be DVD pages mostly. --CygnusTM 00:41, 3 Dec 2005 (EST)
• We've now collected enough to pay for three months of hosting at Dreamhost. I have put in a request to get a server set up for us. We'll move to it ASAP. At the same time, we've received a few offers for hosting alternatives. If they had come in before I had opened the donations, we may have just gone with one of them. Since the donations went directly into the Dreamhost account, I feel obligated to spend it on hosting for the site, rather that just going with one of these alternatives. We will use the three months to evaluate our options and make the best decision for the site's future. --CygnusTM <talk> 10:36, 8 Dec 2005 (EST)
• Now that the site is hosted on a dedicated server this is surely a resolved issue and should be marked as such? Mazadillon 09:08, 16 January 2006 (EST)

Comments

• Definitely at least upgrade. Even if it doesn't fix the CPU issues, it couldn't hurt. Do you think there might be a way to do something like the calendar with parsing on, even if it's not exactly like it is now? --IndieRockLance 10:31, 18 Oct 2005 (EDT)
• If the problem can be solved without spending additional money, I'm all for it. However, if dedicated hosting is needed, I would not mind contributing to help defray the cost. DCEdwards1966 15:57, 18 Oct 2005 (EDT)
• We should try the calendar disable/parser cache option for at least a month and get a feel for the impact. I'm sure we (meaning someone smartish) can figure a heavy-lifting bot to take up a bit of the slack in the meantime or we can manage pages by hand. If it's going to have an impact, we should get an idea of how much runtime we have until this comes up again. I can't help but feel that we'd be living on borrowed time given the infancy of the site and the steady increase in traffic and searches that are to come if we continue to grow and develop into a trusted resource. But I'd rather we not jump hosts until we know what we're dealing with.--Ape Agitator 18:59, 18 Oct 2005 (EDT)
• Why not have a fundraiser like wikipedia often has to pay for hosting. If it's clear that all money will go towards the site and hosting, I see no objections. Maybe having a paypal button on the site permanently is another option. I think the site will probably grow greatly in the future, so moving to dedicated hosting in the long run is probably a good idea anyway. Jacoplane 17:26, 26 Oct 2005 (EDT)
  • I've been avoiding that, becuase I'm worried about the tax implications. As it is now, I think the donations would be taxable income for me. Unless I can write off the server expenses... Hmm... I need to look into this. --CygnusTM 09:32, 27 Oct 2005 (EDT)
• If it is possible to run the apache-module version of php, then I would try that too. I would imagine that it makes a big difference. --Christof Damian 09:59, 27 Oct 2005 (EDT)
• This is scaring me. I'm afraid that I'm going to log on one day and the wiki will be gone. I've been a lot of time into trying to improve articles and to write up episode pages for many shows, it would feel horrible if this site just went away because it became too expensive to manage (understandably). I have no solutions and that's what scary; none seem to exist. --MateoP 21:44, 24 Nov 2005 (EST)
• How about some amazon links for the dvd release page? Not that it'll generate alot of cash for the site but every little bit would count Drkstrm 04:11, 29 Nov 2005 (AST)
  • That is a possibility. A CafePress store may also be coming. We are approaching a crisis situation. I'm worried that we could be shut down at any time. --CygnusTM 09:32, 29 Nov 2005 (EST)
• On my mediawiki 1.5 installation I use eaccelerator now, that helped a lot with cpu usage. --Christof Damian 05:19, 29 Nov 2005 (EST)
  • I'm going to try this out. --CygnusTM 09:32, 29 Nov 2005 (EST)
• This is really worrying me. Me and many others have contributed a lot to this wiki and really don't want to see such a great project go down due to costs. I honestly don't think the fundraising is going to work; we don't have enough viewers at the present time. How about wikicities? --MateoP 21:50, 29 Nov 2005 (EST)
• So.. realistically if the CPU issue isn't resolved we're looking at the site needing $1200 a year to operate ( I rounded up to make things easy ). With the traffic that it's getting now would an amazon store + cafe press + donations get the site that far? --Drkstrm 01:07, 30 Nov 2005 (EST)
• Another thing just came to me... If tviv.org was registered as a business any donations could be classed as "subscriptions" or "memberships". At least that's how it could be done here (canada). Or maybe for the low low price of $10 each we can get a star or honerable mention on a locked page of some sort or even an email address. nothing for nothing as the saying goes..--Drkstrm 01:43, 30 Nov 2005 (EST)
  • Thanks for your suggestions. We're currently looking at some cheaper hosting being offered by a few of our members, and while it's obvious that this issue will never be fully 'resolved' (just look at Wikipedia), we're hoping that our new alternatives help open up to a more established userbase. --Wizardryo|✎ 01:50, 30 Nov 2005 (EST)
• Have you contacted Wikicities (home page here)?

They support Sister projects, like Memory Alpha and Uncyclopedia: this project might well qualify. HTH HAND Phil | Talk 05:01, 30 Nov 2005 (EST)

• By all means, contact Angela at Wikicities; I suspect they might enjoy having something TVIV to unload a lot of the fine-detail stuff about TV that tends to clog up Wikipedia.
  --Baylink 21:12, 1 Dec 2005 (EST) (baylink@wp, baylink@wc)
• Is there someplace specific we're to use the ISBN and ASIN templates ? like only on DVD set pages or only at the bottom with External Sites ? --Drkstrm 19:57, 2 Dec 2005 (EST)
• Have you looked at other hosting ? 1and1 is pretty good with dedicated hosting @ $69 an month . Every little bit helps. --Wenzi 10:21, 5 Dec 2005 (EST)
  • We're trying to stick with Dreamhost because they've been extremely considerate of us considering that we've sucked up a good amount of bandwidth and they've yet to do anything harmful to us. It also helps that they have a direct donation link that goes straight to the website allocation funds, not someone's Paypal account, so that eases a lot of worries some people may have about donating to a site. We're actually about a little more than halfway to paying for this site for an entire year under dedicated hosting, and, as you've said, every little bit does count. --Wizardryo|✎ 16:06, 5 Dec 2005 (EST)
• How much bandwidth does this site use per month? For most dedicated servers (or people who might be able to donate space on one), this is the main concern. --PacketCollision 09:30, 19 Dec 2005 (EST)
  • 10-12 GB/month. Bandwidth hasn't been a problem at all for us. Our current plan allows 100+ GB/month and the dedicated package we're moving to allows 500 GB/month. --CygnusTM <talk> 12:36, 19 Dec 2005 (EST)