Author Topic: The future of GMG....  (Read 9107 times)

0 Members and 1 Guest are viewing this topic.

Offline Mirror Image

  • Veteran member
  • *
  • Posts: 41532
  • Ralph Vaughan Williams (1872 - 1958)
  • Location: Northeast GA, US
  • Currently Listening to:
    The lark ascending into the horizon
Re: The future of GMG....
« Reply #40 on: February 27, 2013, 08:19:09 AM »
And the Purchases Today. I can't even imagine the deluge we are in for in March and December.

 :P
"In the next world, I shan't be doing music, with all the striving and disappointments. I shall be being it.” - Ralph Vaughan Williams

Offline PaulR

  • Veteran member
  • *
  • Posts: 2474
Re: The future of GMG....
« Reply #41 on: February 27, 2013, 10:52:25 AM »
how much space does the extra features within the forum take (like the FB sharing underneath each post, and tap-a-talk thing) take?  I imagine it only takes up a very small amount, though.

As for the purging/saving old important threads/posts, if exporting those posts into plain text isn't viable (which I am sure it is), would using a free service (like a proboards, zetaboards) with it just being a read only board (like the old forum) be viable?  It would keep the old posts available, and reduce space here.
« Last Edit: February 27, 2013, 11:12:17 AM by PaulR »

Offline Octo_Russ

  • Full Member
  • *
  • Posts: 187
    • Octo_Russ
  • Location: High Wycombe, England
  • Currently Listening to:
    Liszt & Messiaen
Re: The future of GMG....
« Reply #42 on: February 27, 2013, 03:09:14 PM »
There must be a way to put a system in place, where threads that haven't been looked at or added to for 12 months would just disappear off the forum, if there's no interest over a sizable timeframe, then why not free up space by getting rid of them.
I'm a Musical Octopus, I Love to get a Tentacle in every Genre of Music. http://octoruss.blogspot.com/

Offline Dungeon Master

  • Benevolent Dictator
  • Administrator
  • *
  • *
  • Posts: 366
  • Head Honcho
  • Location: Canberra
Re: The future of GMG....
« Reply #43 on: February 27, 2013, 04:22:33 PM »
Thanks for all the suggestions and thoughts.

A simple forum post takes up hardly any memory at all. It is almost just plain text. Its added to a database which also records who made the post, time/date, which thread it fits into etc. But even so, simple posts are not the culprits for the memory usage.

Images that are hotlinked are also not a problem as they are not hosted on the server, simply linked to. Same goes for the ASIN tag images. They are stored at Amazon and do not contribute to server storage.

Attachements are a bigger hog of hard drive space, but GMG's attachment folder is currently just 2GB, so thats not a big contribution either.

All the other bits and pieces (smileys, Tapatalk, Facebook/Twitter) are likewise a very minor contributor.

The mystery for me is why there has been a sudden increase in consumption of hard drive space. For years GMG (including both forums) fit into its allocated 40GB space. Recently (starting late last year) server HD space is being consumed much more quickly - currently about 5GB every 2 weeks! That is way faster than anything before. I can't imagine it is being caused by normal usage of the forum.

So the bottom line is that I do not plan to delete any posts or threads. Those threads that are not posted to or viewed will naturally sink away, but still be available to view as required.

The old forum (www.good-music-guide.com/forum) get very little traffic and the database and software take up a few GB. I have already given notice that I will delete the old forum in a month. Any posts you wish to salvage should be copied to either your own hard drive (you can use the Print option on the forum to get a nicely formatted version to save as a pdf file) or re-posted on this forum.

I have paid for a more comprehensive server package that will suffice at the current usage levels for a few months. In that time I hope to diagnose what is eating up the hard drive space and try to eliminate the problem. Then I can go back to the previous (cheaper) hosting package.

If I can't find a malicious or spurious cause, and the HD space is in fact being eaten up by normal usage, then my hosting costs will blow out to several thousands of dollars per year. In that situation, I will have to consider other options, which may include culling posts, restricting attachments, and extorting/exhorting more funds from members to help defray costs.

It is currently a mystery why we are blowing out, and obviously irritating that I have to pay so much more, but GMG is safe and I have no immediate plans to delete posts/threads or shut down the forum.

cheers
Rob
 Macs are better. They just are.

Offline Mirror Image

  • Veteran member
  • *
  • Posts: 41532
  • Ralph Vaughan Williams (1872 - 1958)
  • Location: Northeast GA, US
  • Currently Listening to:
    The lark ascending into the horizon
Re: The future of GMG....
« Reply #44 on: February 27, 2013, 04:53:16 PM »
Very good news, Rob, but I still hope you can locate the problem and fix it. In the meantime, I'm proud to hear that GMG isn't going anywhere. 8)
"In the next world, I shan't be doing music, with all the striving and disappointments. I shall be being it.” - Ralph Vaughan Williams

Offline Bogey

  • Veteran member
  • *
  • Posts: 13469
  • The plural of vinyl is vinyl.
  • Location: Colorado
Re: The future of GMG....
« Reply #45 on: February 27, 2013, 05:19:42 PM »
Have you asked other forum gurus why this may be happening, Rob?
There will never be another era like the Golden Age of Hollywood.  We didn't know how to blow up buildings then so we had no choice but to tell great stories with great characters.-Ben Mankiewicz

Offline k a rl h e nn i ng

  • Veteran member
  • *
  • Posts: 44765
  • Et quid amabo nisi quod ænigma est?
    • Henningmusick
  • Location: Boston, Mass.
  • Currently Listening to:
    Shostakovich, Frescobaldi, Stravinsky, JS Bach, Liszt, Chopin, Haydn, Henning
Re: The future of GMG....
« Reply #46 on: February 27, 2013, 07:20:31 PM »
There are no other "forum gurus": Rob is the only man!
Karl Henning, Ph.D.
Composer & Clarinetist
Boston MA
http://www.karlhenning.com/
[Matisse] was interested neither in fending off opposition,
nor in competing for the favor of wayward friends.
His only competition was with himself. — Françoise Gilot

Offline Bogey

  • Veteran member
  • *
  • Posts: 13469
  • The plural of vinyl is vinyl.
  • Location: Colorado
Re: The future of GMG....
« Reply #47 on: February 27, 2013, 08:15:35 PM »
There are no other "forum gurus": Rob is the only man!



Indeed.
There will never be another era like the Golden Age of Hollywood.  We didn't know how to blow up buildings then so we had no choice but to tell great stories with great characters.-Ben Mankiewicz

Offline Mirror Image

  • Veteran member
  • *
  • Posts: 41532
  • Ralph Vaughan Williams (1872 - 1958)
  • Location: Northeast GA, US
  • Currently Listening to:
    The lark ascending into the horizon
Re: The future of GMG....
« Reply #48 on: February 27, 2013, 08:33:55 PM »
There are plenty of forum gurus. I forget the guy who runs the TalkClassical site, but he always did an excellent job maintaining the site but the whole atmosphere on that forum is quite impersonal.
"In the next world, I shan't be doing music, with all the striving and disappointments. I shall be being it.” - Ralph Vaughan Williams

Offline Dungeon Master

  • Benevolent Dictator
  • Administrator
  • *
  • *
  • Posts: 366
  • Head Honcho
  • Location: Canberra
Re: The future of GMG....
« Reply #49 on: February 27, 2013, 11:12:10 PM »
Asking other forum managers wont help, unfortunately, as each site is unique with respect to the server, operating system and the forum software. GMG is the only classical music forum that runs using Simple Machines Forum software.

SMF have a very large community and support structure, and I may well need to make use of their expertise, as well as the most excellent support at PowerVPS, my hosting site. I, too, have not inconsequential knowledge about these things, but it will a matter of finding the time to dig into the rather complex mySQL database structure to work out where all the extra data is being laid down.

cheers
Rob
 Macs are better. They just are.

Offline Gurn Blanston

  • Haydn: that genius of vulgar music who induces an inordinate thirst for beer - Mily Balakirev (1860)
  • Global Moderator
  • *
  • Posts: 30782
  • Support your local Haydn Society
    • Gurn's Haydn Blog
  • Location: Texas, where else?
  • Currently Listening to:
    Haydn, I reckon.
Re: The future of GMG....
« Reply #50 on: February 28, 2013, 05:11:51 AM »
Asking other forum managers wont help, unfortunately, as each site is unique with respect to the server, operating system and the forum software. GMG is the only classical music forum that runs using Simple Machines Forum software.

SMF have a very large community and support structure, and I may well need to make use of their expertise, as well as the most excellent support at PowerVPS, my hosting site. I, too, have not inconsequential knowledge about these things, but it will a matter of finding the time to dig into the rather complex mySQL database structure to work out where all the extra data is being laid down.

cheers
Rob

Rob,
About 4 years ago, did we not have an incident which involved a hacker secretly using our space to run something of his own which also caused a net loss of hard drive space and bandwidth? Do you know in this instance that it isn't some form of the same thing? Seems like it.... :-\

8)
Help support GMG by purchasing from Amazon using this link

Visit my Haydn blog: HaydnSeek

Follow me on Twitter @GurnBlanston106

Offline springrite

  • Veteran member
  • *
  • Posts: 6059
  • Location: Flying all over the place
  • Currently Listening to:
    Lots of Bach, Brian, Mahler, Rubbra, Beethoven and Buddhist chants
Re: The future of GMG....
« Reply #51 on: February 28, 2013, 05:25:36 AM »
Rob,
About 4 years ago, did we not have an incident which involved a hacker secretly using our space to run something of his own which also caused a net loss of hard drive space and bandwidth? Do you know in this instance that it isn't some form of the same thing? Seems like it.... :-\

8)

Yes, I think so. There is one of the other gurus.  :P
Do what I must do, and let what must happen happen.

Offline Brian

  • Veteran member
  • *
  • *
  • Posts: 19750
    • Brian's blog
Re: The future of GMG....
« Reply #52 on: February 28, 2013, 05:32:45 AM »
I interpreted the "guru" comment as a reference to the fact that Simple Machines has its own support forum for stuff like this.

Offline Dungeon Master

  • Benevolent Dictator
  • Administrator
  • *
  • *
  • Posts: 366
  • Head Honcho
  • Location: Canberra
Re: The future of GMG....
« Reply #53 on: March 10, 2013, 02:04:38 AM »
Good news and bad news ---

The good news is I have identified the cause of the rapidly disappearing hard drive space. It was neither malicious nor was it normal forum use. It was caused by the forum software using more RAM than is allocated to me on the server. Every time it went over the RAM limit, the server laid down a very large (34MB) debug file. This was happening many times per day, and therefore eating up 10 GB of hard drive space per fortnight. I have now deleted those files, and our hard drive space is down below 40GB again.

The bad news is that the forum continues to use a lot of RAM. Possibly from increased traffic on the forum (from both members and non-members), possibly from a bug in the software. That means that I still have to pay for a more expensive server package, not for the hard drive space, but to cope with the extra RAM requirements.

It still perplexes me why the forum should quite suddenly (February 2 it all started) start using so much more resources. I shall continue my investigations. Stay tuned.

Rob
 Macs are better. They just are.

Offline mc ukrneal

  • Veteran member
  • *
  • Posts: 8222
Re: The future of GMG....
« Reply #54 on: March 10, 2013, 06:29:24 PM »
Good news and bad news ---

The good news is I have identified the cause of the rapidly disappearing hard drive space. It was neither malicious nor was it normal forum use. It was caused by the forum software using more RAM than is allocated to me on the server. Every time it went over the RAM limit, the server laid down a very large (34MB) debug file. This was happening many times per day, and therefore eating up 10 GB of hard drive space per fortnight. I have now deleted those files, and our hard drive space is down below 40GB again.

The bad news is that the forum continues to use a lot of RAM. Possibly from increased traffic on the forum (from both members and non-members), possibly from a bug in the software. That means that I still have to pay for a more expensive server package, not for the hard drive space, but to cope with the extra RAM requirements.

It still perplexes me why the forum should quite suddenly (February 2 it all started) start using so much more resources. I shall continue my investigations. Stay tuned.

Rob
It seems an unlikely cause, but you did change the smiley's and all that around that date. Glad to hear that at least part of the problem has been solved!
Offenbach gets a raw deal in recordings considering his talent! For a discussion of this outstanding composer too little recorded: http://www.good-music-guide.com/community/index.php/topic,5572.

Offline SonicMan46

  • Veteran member
  • *
  • *
  • Posts: 11862
  • Location: North Carolina
Re: The future of GMG....
« Reply #55 on: March 10, 2013, 06:52:15 PM »
Good news and bad news ---

The good news is I have identified the cause of the rapidly disappearing hard drive space. It was neither malicious nor was it normal forum use. It was caused by the forum software using more RAM than is allocated to me on the server. Every time it went over the RAM limit, the server laid down a very large (34MB) debug file. This was happening many times per day, and therefore eating up 10 GB of hard drive space per fortnight. I have now deleted those files, and our hard drive space is down below 40GB again.

The bad news is that the forum continues to use a lot of RAM. Possibly from increased traffic on the forum (from both members and non-members), possibly from a bug in the software. That means that I still have to pay for a more expensive server package, not for the hard drive space, but to cope with the extra RAM requirements.

It still perplexes me why the forum should quite suddenly (February 2 it all started) start using so much more resources. I shall continue my investigations. Stay tuned.

Rob

Hi Rob - thanks for explaining the server issue(s) and your efforts in trying to understand & explain (and hopefully correct) the problem(s).  I just cannot imagine that there have been  major changes in activity on the forum in recent weeks/months, so a server issue, a software bug(s), malware, or other explanation seems to be more likely, but of course just my unprofessional opinion.

But, just curious - I'm assuming that you are 'renting' space on someone's server?  I would assume that they have expertise to figure out these kinds of issues?  Not trying to be a pain, but my son runs servers for a major insurance cooperative in Indianapolis and is responsible for maintaining its servers and handling the malware issues - probably not the same situation - HOWEVER, again thanks for your dedication to the forum and hope that the problems will be solved quickly.  Dave :)

Offline Dungeon Master

  • Benevolent Dictator
  • Administrator
  • *
  • *
  • Posts: 366
  • Head Honcho
  • Location: Canberra
Re: The future of GMG....
« Reply #56 on: March 11, 2013, 02:16:20 AM »
The smileys were a red herring. They showed up in my error logs, but were not the cause of the hard drive or RAM issue.

Yes, I rent a server. In fact a VPS, a Virtual Private Server. I rent one quarter of a server, which is partitioned so that each of the 4 users on that machine has full control over their quarter (including operating system, hard drive, RAM and CPU allocation). Its like owning an entire server but at a much lower cost, and one quarter of the price. Running a full, dedicated server is well beyond my price range, usually $300-600 per month.

The hosts (PowerVPS) are very helpful in ironing out technical difficulties. However, when a bug is caused by a specific software installed by me, such as Simple Machines Forum that runs this forum, they are as much in the dark as I am. While they can help me diagnose why a server crashed (such as high RAM requirements) it is up to me to work out why SMF is using so much RAM. I often go the the SMF support community (that also runs of SMF, so their forum looks much like ours) to help diagnose problems.

Between my hosts, the SMF community and my own knowledge, I can usually work it out. Sometimes, its fun, being like an IT detective.

On second thoughts, its only fun when I finally find a solution.
 :P
 Macs are better. They just are.

Offline SonicMan46

  • Veteran member
  • *
  • *
  • Posts: 11862
  • Location: North Carolina
Re: The future of GMG....
« Reply #57 on: March 11, 2013, 05:59:38 AM »
Hi again Rob - thanks for the explanation above - I understand your server situation much better - Dave :)

Offline Gurn Blanston

  • Haydn: that genius of vulgar music who induces an inordinate thirst for beer - Mily Balakirev (1860)
  • Global Moderator
  • *
  • Posts: 30782
  • Support your local Haydn Society
    • Gurn's Haydn Blog
  • Location: Texas, where else?
  • Currently Listening to:
    Haydn, I reckon.
Re: The future of GMG....
« Reply #58 on: March 11, 2013, 12:33:21 PM »

It still perplexes me why the forum should quite suddenly (February 2 it all started) start using so much more resources. I shall continue my investigations. Stay tuned.

Rob

Had you done any upgrades or mods in that time period? I'm sure you checked all that, it just seems so odd that you can actually see the beginning of it and yet no action attaches to it. If it isn't something legitimate like that, than it must be something illegitimate. :-\

8)
Help support GMG by purchasing from Amazon using this link

Visit my Haydn blog: HaydnSeek

Follow me on Twitter @GurnBlanston106

Offline Que

  • Global Moderator
  • *
  • Posts: 14166
  • "One HIP dude"
  • Location: The Hague, Netherlands
  • Currently Listening to:
    Still nuts about harpsichord music and exploring Early Music.
Re: The future of GMG....
« Reply #59 on: March 11, 2013, 11:13:29 PM »
The forums seems to keep crashing at times...I'm officially concerned... :(

Q
À chacun son goût.

Buying Music From Amazon?
Please consider using these links. A small percentage of every sale using these links is passed on to GMG and helps keep this forum online.
Amazon US
Amazon Canada
Amazon UK