Note: DOS means Denial Of Service.
Sent: Tuesday, July 13, 2010 4:09 PM
To: Don Russ
Subject: Re: please tell me
It went down Thursday. The issue is quite
complex with the security. Drew does work diligently on a daily
basis. Although I haven't heard anything definitive yet today, this does
appear to be yet another DOS attack.
Don Russ wrote:
Zippy, The national website has been down for days. I append our previous correspondence on a similar matter. Please tell me: Exactly when did it go down? Why does it take so long to get it up again?
And then, about noon on Wednesday, Bill Richart (PGR national board president) posted on the PGR Facebook page:
We're in contact with the company that makes our
security software to fix this permanently and as rapidly as possible. Any
information on tracking the source will be reported, but we wouldn't give them
the courtesy of a warning that we find them.
Funds have already been approved for the professional services, just waiting for them to do so. But, we don't plan to stop there, we're pretty close to a plan and funds to completely upgrade the site software across the board to improve the overall site and stability. If we need to pay for this to be completed in a timely manner, that's going to be done also. Drew has been spending more hours than a full time job on this and the site relocation to WA, in addition to proposals for the upgrade.
He has asked for volunteers to assist, but hasn't gotten me a list of needed qualifications yet. Send me an email with that and experience, along with where you are and I will gratefully get it to him. Thanks for the questions and patience.
Bill continues on Facebook Thursday morning:
To reply to a couple of the questions posted. The security software company
is currently analyzing the malicious traffic. We'd address any indentified
source legally, if applicable. Most of these types of attacks come from
innocent users who have had their computers hacked and "zombied" into these actions without the owners knowledge. We just don't have definitive information
we could share at this point.
I have written an email that will probably be posted here today and possibly emailed to the membership outlining what has happened during these two attacks, what has been done, and what the future plan to ensure the site supports the mission is. The BoD and web admin are reviewing as they generally do to make sure we are sharing the most accurate info.
Immediate support for the mission. Remember, the PGR mail system is working. You can contact us at a varietly of addresses where folks will help you support the mission. The mods@, info@, and contact@ mailboxes are fully able to assist and reply to you. They won't be able to provide additional info on the website, but they will always help if they can. Secondly, we are looking at establishing some additional mission listing capabilities for those who are not facebook members and don't want to join. We've got to provide information to the membership without requiring them to join anywhere more than the PGR.
I'm sure we could benefit from additional volunteer assistance, but haven't discussed this yet. Our Director of Ops, Zippy, might have some ideas there. I'll ask her about that and if she can be available through here.
Thanks for the questions and offers. The PGR mission only happens through the efforts and good hearts of the membership. Sincere Respects,
Then, an email broadcast from Bill at 9:00 PM on Thursday:
As we are all painfully aware, the PGR site has been down for the second extended time within the past few months. This email is to summarize why we are down, what steps have been taken, and where we are at in repairing and preventing future disruption of the site.
The PGR website has been under extended attack referred to as Denial of Service (DOS). A DOS attack is intended to overwhelm the site traffic or hardware by generating a huge flow of traffic to the site or by tying up the server's resources with input it can't understand or resolve, effectively paralyzing it. Many other websites have been hit with this same type of attack, and we do not believe it is directed against the PGR specifically.
The PGR site infrastructure has not been modernized effectively as we've grown. Other issues, notably the Trademark Defense, have consumed a huge portion of the PGR operating budget which comes from the official PGR Gear Store and donations. Please understand that we are not asking for any increased donations, just explaining some of the limits we are operating under.
We had already begun a project to address the site "age" regarding equipment and software. A new site location had already been "powered up" but not completely functional at the time of the first attack. As part of the response to the first attack, the site relocation of web services was expedited and new filtering servers were put into place. Only our mail system has not been completely moved at this time as we work with the major internet providers regarding our address change and "white list" as a bulk mail client.
The latest attack started on last Friday evening. By the time the situation was defined and on hand resources were tried, we were into the weekend and were not able to establish new professional consulting till Monday. We have currently provided advanced information for analysis along with further installation and modification of the security filters. We expect a detailed outside analysis shortly and are in consultation with the provider of our security filters for what we expect will be a long term solution. Funds are already approved, we just have to get the best and most effective steps to implement.
Currently, the site hardware has been almost completely upgraded. Once we get through this current crisis, we will immediately focus on completing the site migration to the new facility and are reviewing proposals for replacing and updating the site software to include the web site operating software. We are currently looking at two different paths in software to responsibly and effectively improve the site stability and maintenance, improve navigation and ease of use, and provide additional tools and support for members and local leadership, while also providing advanced security features.
We want the site to be more member friendly, focus on the information you need for the mission, and be there 24/7 for you. Once the new site is proven, we will deploy a back up site dedicated to the mission information only for reliability. The hardware for this back up site is already on hand.
I extend a personal apology for not preventing these disruptions of the PGR mission. Though we had been aware of the "age" of the site and need to expand, it had not been effectively addressed before these unexpected attacks. You can't cover for every event, but the site age was something we were already concerned about. We will do all that we can to return the site as soon as possible, continue mission notification through the facebook page, and develop other remote notifications that do not require a member to join any other site to receive mission info and updates on the PGR.
Thank you for all that you do and for the patience and strength you've shown in supporting the mission in spite of this disruption of our website.
President ~ Board of Directors
And then, an email broadcast from Bill on Monday afternoon.
At this point, the national website has been down ELEVEN days.
We really try not to send to many of these all member emails. We prefer letting you subscribe to the topics/threads you choose. With the website still down, we do want to share a brief update of progress and status.
First, a very sincere thank you to the many folks with web knowledge that have offered assistance. Every email has been shared with the web team and will each receive a response; some in this phase of defending against this attack, and some in the moving forward to the PGR 2.0 website. A second thanks to all the folks who have mailed in questions and messages of support. All are appreciated and will also receive replies. You continue to show why the PGR mission will never fail!
As of Friday, our web admin had multiple volunteers working on two different paths to return the PGR site. Without going into too much tech detail; one team is trying to defeat the attack through further work on the web application firewall and the other on work with server interfacing or how data communicates to the website. Both paths are proceeding independently of each other to return the site as quickly as possible and then to make it as dependable as possible.
Communication has been challenged, but we will work through it. The mail system remains operational and you can always contact us through the email@example.com, firstname.lastname@example.org, or email@example.com. Additionally, we will post information on the PGR Facebook page, and we are bringing the Google calendar back up also. We will maintain the Google calendar as a back up to prevent any possible future disruptions until the site has demonstrated it's security or until the backup site comes online. We will do everything possible to provide the mission information you need to honor our heroes. The address for the calendar is http://tinyurl.com/pgrcalendar
Thank you again for all that you do to support the mission, especially through this challenge.
Bill "Whirlwind" Richart
President ~ Board of Directors Patriot Guard Riders
Déjà vu – less than two months ago
back to ALL MISSIONS