JeffX
 |
|
sorry guys, I posted a few updates on our twitter page during the outage.
We had a drive go south on the database server. It took me some time to extract the server from the data center, source new drives, restore, repair, reconfigure, etc... and then reinstall the server. To make matters worse, I got the machine back into the data center at around 11:15AM today but there was a single file permission glitch on one of the webservers that prevented the entire operation from starting back up as it should have. So it took me a while to track THAT down.
I probably could have had it back online late yesterday afternoon but some config glitches on our backup side meant getting our backups took me a lot longer than it should have. I have brand new drives in the database server now, restored the latest backup (which was from around 2 or 3AM on Nov 20th., so we lost something like 8 or 9 hours worth of early morning comments), and hopefully resolved the config glitches which made the backups glitch.
|
98EX4cyl
 |
|
|
Thanks for all the hard work. Technology failures suck, i just lost my 7 year old Dell yesterday. New one arrives Monday.
|
xBeastx
 |
|
|
YAY!
|
TonyEX
 |
|
(1) Twitter? What's that?
(2) RAID-5+1. I run two RAID 5 arrays at home... identical Dell computers (Bought three used ones, one is for spares). One backs the other... triple redundancy. I'll never lose pictures of the kids again.
(3) At least the price of hard drives is coming back down, huh?
(4) Good job. Thanks.
|
superchg2
 |
|
|
Thanks for all your hard work, Jeff!
|
JeffX
 |
|
TonyE wrote:
(1) Twitter? What's that?
(2) RAID-5+1. I run two RAID 5 arrays at home... identical Dell computers (Bought three used ones, one is for spares). One backs the other... triple redundancy. I'll never lose pictures of the kids again.
(3) At least the price of hard drives is coming back down, huh?
(4) Good job. Thanks.
|
we have RAID but unfortunately I got caught in a technology snag on this one server. When i bought it, it was so "state of the art" none of the stable operating systems had reliable drivers for the raid setup, and we were badly in need of getting this server installed, so I ended up just doing some scripts to achieve 1:3 redundancy. The plan was to implement the RAID once the drivers were sorted out. Well, it turns out there were some version changes along the way which broke compatibility with our (ancient) CMS architecture. We needed to run a version of the SQL server which was incompatible with the newer versions of the OS (which had improved support for our RAID setup). Maybe with some big time mucking around we could make it work but it would require a prolonged downtime for this server. I could probably set up our "cold spare" to step in in the meantime but it's so much weaker than our current server it would struggle to deal with anything approaching our peak loads.
Anyhow, we're looking at options as to getting out of the "owned server" business so I don't have to worry about this stuff, but then we have to do a lot of work to figure out exactly how much something like Amazon Web Services (or similar) would cost us. I'm sure we'd have to put in a lot of work to optimize everything for performance vs. cost.
(3) Server-class hard drives are as expensive as ever. The capacities go up but the prices are still ridiculous. And we don't need the huge capacities - our entire database fits VERY comfortably on a 36GB partition. I've thought about trying some desktop drives just to "get me by" until we get bigger, badder servers, but I'm just not comfortable with that, so yesterday was sort of an expensive trip to MicroCenter.
(4) you're welcome. I'm just glad this failure happened yesterday as opposed to today. We would have probably been dark until sometime into next week.
|
TurkMan71
 |
|
All of the above pretty much is Greek to me...but thank you for your time, money, and effort!
I definitely come here first to get my Honda fix, and I know I'm not the only one who depends on you guys, so again, THANK YOU!!
Maybe a cookie bake sale is in order to raise some funds? I make a mean snickerdoodle...
|
trexdoc
 |
|
|
Does this mean we're finally getting new forum software?
|
JP
 |
|
TurkMan71 wrote:
All of the above pretty much is Greek to me...but thank you for your time, money, and effort!
I definitely come here first to get my Honda fix, and I know I'm not the only one who depends on you guys, so again, THANK YOU!!
Maybe a cookie bake sale is in order to raise some funds? I make a mean snickerdoodle...
|
What about the International Convention of TOV'ers!
|
TonyEX
 |
|
If you can make do by hosting your web sites somewhere.. go for it.
Admittedly my web sites are nothing like TOV but all i gotta do is to keep an eye on them, keep the spam filters up and periodically ftp or login to them.
I think it's cheaper on the long run and you don't have to panic.
Just make sure to use a stable provider. I've heard horror stories. I use AT&T.
|
A77
 |
|
|
How many of us (apart from Andrew and I) wondered if we were being locked out by our employers and needed to check TOV on our smartphones....
|
JP
 |
|
A77 wrote:
How many of us (apart from Andrew and I) wondered if we were being locked out by our employers and needed to check TOV on our smartphones....
|
lol
|
sadlerau
 |
|
|
I was having trouble with severe withdrawal symptoms! :)
|
revvin
 |
|
A77 wrote:
How many of us (apart from Andrew and I) wondered if we were being locked out by our employers and needed to check TOV on our smartphones....
|
QFT
|
CarPhreakD
 |
|
|
Heck, I thought I finally pissed off Jeff and Shawn enough for them to ban me!
|
xBeastx
 |
|
Now that you guys have solved that problem, would it be possible to consider updating/refreshing the site to make it look more 2012-ish haha? And that Model Release Matrix and pretty much all other things up in that menu are outdated.
But I only use the news and recent forum messages so what do I care lol.
|
sadlerau
 |
|
|
The look's just fine. I'm finely getting used to this new look after the original black look of the nineties! :D
|
Grace141
 |
|
sadlerau wrote:
The look's just fine. I'm finely getting used to this new look after the original black look of the nineties! :D
|
Same here. I agree the look is fine. The KISS principle allows more of a focus on the things that matter.
|
|
|
| |