11/23/2015 16:45 Down again? UP 1322 28 Nov 2105!

[meno377]

Thanks for the update. To cockedandglocked and kapache, This failure is not a typical situation IMO. In addition, I would bet that Calguns along with it's hosting services don't have the resources that Google, Microsoft, and other huge datacenters have. It's very unfortunate that this type of failure happened to what it sounds like more than 4 Hardrives at once or (four mirrors). If Kestryll wants to start a fundraiser to support more hardware for more redundancy, I certainly don't mind contributing. 

Firmware being upgrade for alerts to be sent sounds odd. Why didn't they implement monitoring scripts that sends alerts when a drive drops from the raid? Seems they are making bs excuses to what happened. I bet a drive dropped from the raid it never got replace, and now that two drives from the array died the whole system went poof!

That was my thought too, it doesn't sound like an unfortunate series of coincidences, it sounds like they weren't diligent and are now paying the price. Coming up with excuses that are extremely unlikely, but we can't prove it didn't happen, to try to save face and not sound incompetent. Did they not realize that their system was incapable of warning them that a drive failure was detected? Does that mean that, until now, they've never head a drive failure? Heck, my server at work only has 4 disks, and I have to replace a dead one at least once every 2 years. And guess what, I even get an email when one goes bad 

I've known Michael the owner of PE/GV for over a decade, he has at one point donated two years of hosting free back when Calguns was small and needed time to grow. He's been a Calgunner for most of the time Calguns has existed and has helped dozens of Calgunners and 2A orgs get web stores and sites off the ground.

Nobody has to come up with excuses, for over ten years I've gotten straight answers whenever I've had a problem.

I don't care how unlikely it is I don't need them to prove jack squat because I've never been given a reason to mistrust Michael or PE/GV.
I have been given many reasons to trust them over the last decade so when I get a call from my friend and owner of the hosting company to tell me 'This is what happened' I believe him that that is what happened.

[justdave]

This is CG Purgatory.  All of you XD fanboys just need to give me your guns and go buy Glocks, then maybe.. just maybe all will be right in the world by this weekend.

[truthseeker]

So this is a fun situation, I can not access the control panel to log a trouble ticket and the host site is down so emails can't get through to them either.
On the plus side the owner of the company is a Calgunner and I have his cell number. 

As of right now the info I have is that it is a major cascade hardware failure and they are trying to determine how many things failed at the same time.
It's not just us or anything malicious that we're aware of and they are working on it as we speak.

Sounds like someone put BOTH ends of a ethernet cable into the same switch!!

[cockedandglocked]

Thanks for the update. To cockedandglocked and kapache, This failure is not a typical situation IMO. In addition, I would bet that Calguns along with it's hosting services don't have the resources that Google, Microsoft, and other huge datacenters have. It's very unfortunate that this type of failure happened to what it sounds like more than 4 Hardrives at once or (four mirrors). If Kestryll wants to start a fundraiser to support more hardware for more redundancy, I certainly don't mind contributing. 

I understand, but I'm just curious what's wrong with using a giant company like, say, GoDaddy to host CG? I know this guy is Kes's friend and a longtime supporter of CG, but sometimes business is business.

I may really love an ammo company, and the owner might be a really good friend, honest, and always been there for me for year and years, but when one of his cartridges blows up my rifle and I lose 2 fingers, I'll probably go find a new company. Just sayin'.

Not that I'm comparing CG being down for a couple days to losing an appendage (though for some I'm sure it seems just as bad or worse) 

[truthseeker]

A few questions that I know nobody here can answer:

-Why did they not have redundant hardware in place? No self-respecting computer company doesn't have a plan (and spare hardware) in place to prevent a hardware failure from taking out company operations. "they are trying to expedite replacement parts"... in other words, they only had 1 of whatever broke.

-If the storage disk array is what blew up, and they are "working to restore from backups", does that mean we lose hours, days, weeks, or months of data?

-Which IT guy there is getting canned?


I fully understand that hardware failures happen, I've experienced lots of them myself... but anyone who halfway knows what they're doing, would never experience a catastrophic hardware failure like this. Redundant systems should be in place, where one a hardware bank goes offline, catches fire, or whatever, the system kicks itself over to the redundant hardware and operations continue. Google, Amazon, Facebook, and many others never experience downtime, and I'm pretty sure it isn't because their hardware never fails.

Maybe time for CG to switch hosts?

Computer Engineer here, I agree 100%!

They are more than likely running Calguns.net on a virtual machine (on a hardware server with other clients virtuals) in a datacenter.

They should have had all the virtual machines on another set of "hardware machines" in a DIFFERENT data center on the East Coast so that the other "cluster" takes over if ANYTHING fails!

FOR PETE SAKE, THIS IS CALGUNS WE ARE TALKING ABOUT!! : )

[meno377]

A few questions that I know nobody here can answer:

-Why did they not have redundant hardware in place? No self-respecting computer company doesn't have a plan (and spare hardware) in place to prevent a hardware failure from taking out company operations. "they are trying to expedite replacement parts"... in other words, they only had 1 of whatever broke.

-If the storage disk array is what blew up, and they are "working to restore from backups", does that mean we lose hours, days, weeks, or months of data?

-Which IT guy there is getting canned?


I fully understand that hardware failures happen, I've experienced lots of them myself... but anyone who halfway knows what they're doing, would never experience a catastrophic hardware failure like this. Redundant systems should be in place, where one a hardware bank goes offline, catches fire, or whatever, the system kicks itself over to the redundant hardware and operations continue. Google, Amazon, Facebook, and many others never experience downtime, and I'm pretty sure it isn't because their hardware never fails.

Maybe time for CG to switch hosts?

Computer Engineer here, I agree 100%!

They are more than likely running Calguns.net on a virtual machine (on a hardware server with other clients virtuals) in a datacenter.

They should have had all the virtual machines on another set of "hardware machines" in a DIFFERENT data center on the East Coast so that the other "cluster" takes over if ANYTHING fails!

FOR PETE SAKE, THIS IS CALGUNS WE ARE TALKING ABOUT!! : )

Technically you're correct. But how much more money does it take to support that 24/7 365 days a year?

[truthseeker]

Don't know, but this is Calguns we are talking about!

Seriously though, meno377, why don't you go to Amazon EC2 and check prices if you are that concerned about price.

[cockedandglocked]

Technically you're correct. But how much more money does it take to support that 24/7 365 days a year?

I have no idea how much CG pays for their hosting now, nor what their bandwidth requirements are, but GoDaddy offers a dedicated hardware server for $209/mo, plus $4.99/mo for daily automatic backups:

32 GB RAM
Storage: 2 TB
Bandwidth: 20 TB/mo

Cheaper plans are available if the requirements are less.

https://www.godaddy.com/servers

I'm sure there are many similar options out there from other company's as well, that's just the first one that came to mind.

[GlockDoc]

Thanks for the update. To cockedandglocked and kapache, This failure is not a typical situation IMO. In addition, I would bet that Calguns along with it's hosting services don't have the resources that Google, Microsoft, and other huge datacenters have. It's very unfortunate that this type of failure happened to what it sounds like more than 4 Hardrives at once or (four mirrors). If Kestryll wants to start a fundraiser to support more hardware for more redundancy, I certainly don't mind contributing. 

I understand, but I'm just curious what's wrong with using a giant company like, say, GoDaddy to host CG? I know this guy is Kes's friend and a longtime supporter of CG, but sometimes business is business.

I may really love an ammo company, and the owner might be a really good friend, honest, and always been there for me for year and years, but when one of his cartridges blows up my rifle and I lose 2 fingers, I'll probably go find a new company. Just sayin'.

Not that I'm comparing CG being down for a couple days to losing an appendage (though for some I'm sure it seems just as bad or worse) 

Much worse than losing two digits ...

 

[swalt]

Perhaps its all a PM problem?  Not swapping out drives at the end of their useful lives?  I know 1 thing, all the drives being replaced now will fail at roughly the same time in the future.  They should be replaced before they "get an e mail from the server"  in some sort of staggered way. 

[Melvino the FAKE]

my friends..............don't worry, it will all be okay 

[meno377]

Don't know, but this is Calguns we are talking about!

Seriously though, meno377, why don't you go to Amazon EC2 and check prices if you are that concerned about price.

That's my point. Calguns probably doesn't have the extra funds for a dedicated server with redundancy.

[Somkey the Hores]

 I was lurking here yesterday and now I'm back. Maybe it's an attack by Procol Harum or Ralph Queida...?

[Melvino the FAKE]

Don't know, but this is Calguns we are talking about!

Seriously though, meno377, why don't you go to Amazon EC2 and check prices if you are that concerned about price.

That's my point. Calguns probably doesn't have the extra funds for a dedicated server with redundancy.

my friends, everyone knows that kestryll is independently wealthy, don't be a hater

[Melvino the FAKE]

I was lurking here yesterday and now I'm back. Maybe it's an attack by Procol Harum or Ralph Queida...?

My Friend, you sound muslimphobic.  Don't hate the peaceful ones

[Somkey the Hores]

Gawd, if I hear that one more time I'm going postal. As we said in the 70s, "Kill 'em all and let God sort 'em out." And, I'm not afraid of them.

[Satex]

Folks:
To all the armchair IT gurus, Kestryll and the hosting provider don't have much experience in this field and can really use your help.  Please put all of your suggestions in writing on the back of $100 bills.  If one bill is not enough for your ideas, please use multiple bills and mail them all to him for review, evaluation, and consideration.
Until then, put the smartphone/tablet/laptop down, go outside your mom's basement, enjoy a breath of fresh air, clean your guns, go out for a walk, maybe go out to the range...
You get the idea...
 

[Melvino the FAKE]

Gawd, if I hear that one more time I'm going postal. As we said in the 70s, "Kill 'em all and let God sort 'em out." And, I'm not afraid of them.

My friend, you sound violent................should you own guns my friend?

[Melvino the FAKE]

Folks:
To all the armchair IT gurus, Kestryll and the hosting provider don't have much experience in this field and can really use your help.  Please put all of your suggestions in writing on the back of $100 bills.  If one bill is not enough for your ideas, please use multiple bills and mail them all to him for review, evaluation, and consideration.
Until then, put the smartphone/tablet/laptop down, go outside your mom's basement, enjoy a breath of fresh air, clean your guns, go out for a walk, maybe go out to the range...
You get the idea...
 

My friend, you sound rich....................can we be friends?

[Somkey the Hores]

Gawd, if I hear that one more time I'm going postal. As we said in the 70s, "Kill 'em all and let God sort 'em out." And, I'm not afraid of them.

My friend, you sound violent................should you own guns my friend?

Are you from the planet Weenie or what?

[LBDamned]

geez... missed a day from this thread and - a bunch of stuff to weed through for meaningful update...

a locked thread with periodic updates would be helpful 

In the mean time, I get it - down for a couple days... I'm pretty sure I was one of the first ones to say:

seems like this might be a long one 

[Melvino the FAKE]

Gawd, if I hear that one more time I'm going postal. As we said in the 70s, "Kill 'em all and let God sort 'em out." And, I'm not afraid of them.

My friend, you sound violent................should you own guns my friend?

Are you from the planet Weenie or what?

My friend, what does that mean?

[Somkey the Hores]

Kindly go back under your bridge. How do you say Adios in troll?

[Melvino the FAKE]

My friend, why so hostile?

[Foulball]

Melvino!!!

How are you my friend?

Looks like you are getting along well with the natives.

[Melvino the FAKE]

Melvino!!!

How are you my friend?

Looks like you are getting along well with the natives.

Thank you my friend.  So many newbies that are not so friendly  Cheers my friend, Cheers!

[Foulball]

Melvino!!!

How are you my friend?

Looks like you are getting along well with the natives.

Thank you my friend.  So many newbies that are not so friendly  Cheers my friend, Cheers!

Cheers to you as well good sir.
I've missed your positive attitude on the other site.
Don't let them bring you down my friend.

[Aldemar]

OK, who will be the first to come up with a really good conspiracy theory as to why .net is down?

• Pelosi
• Øbama
• chemtrails
• .45 > 9mm
• fluoride
• mandatory vaccination
• Harry Tanges
• Moms Demand Don't Stop
• bacon

It was Senator Leland Yee!!! He took Calguns down as revenge for them not coming to his aid regarding his second amendment rights to buy and sell firearms in California

I think it was Jim Mitchell, the owner of Oaktree.

[Somkey the Hores]

It sounds as if Quakers are invading. Quick, call ICE.

[C-Arms]

Oh Lordy, please be up and running before Thanksgiving, please, please, please.....