January 5th, 2012 – an update to my original shutdown message with some FAQ’s to deal with several common questions landing in my inbox:
Q: Will you be keeping any packet gear on the air?
A: Yes, the WA4ZKO-15 APRS digi will remain on the air as is, no changes.
I am considering keeping the 6m 1.2k and a UHF 9.6k port in place with a stand-alone TNC acting as a simple dual-port node/digi/mailbox for a basic local packet messaging hub. It would be 100% RF, easy to run on backup power, even offer some very useful solar/battery power options, fully isolated from the internet, no IP network dependency or exposure, and provide a more than adequate mini-PBBS/node for the county.
UPDATE 02/10/2012: The new “mini-PBBS” is on the bench in testing. It will be moved to the Dry Ridge site for further testing in a few days. I am working towards making it 100% solar/battery powered.
UPDATE 03/24/2012: After a month of burn-in testing at the Dry Ridge site the new “mini-BBS” is officially on the air. It is 100% solar/battery powered and offers access on both 6m 1200 baud and UHF 9600 baud. This gives us a local messaging hub and cross frequency/speed gateway and digipeater that are BOTH internet and power grid independent. WA4ZKO-1 is the BBS. WA4ZKO-4 is the KA-Node, and DRGATE is the crossport digipeater. The perks of not having to maintain a PBBS PC at the tower site are nice and even nicer for the electric bills!
UPDATE 04/29/2012: The new system is working well and proving itself very reliable. We have been running most of our county ARES communications over it using Outpost as the client software.
UPDATE 05/18/2012: The new system has been on for nearly 3 months now and has proven itself to be rock solid with zero glitches, no restarts needed, and not a second of downtime. Just like with the KPC3+ TNC powering the local APRS digipeater (with uptime measured in years!) the KPC-9612+ just runs and runs and runs! During the recent COMEX drill we passed several p-mail, NTS, and ICS-213 messages over the system using both 1200 and 9600 baud connections.
PC based packet BBS systems have their pros for sure, but they also come with a lot of cons too. Most of the potentially EMCOMM useful pros come in the form of additional routing capabilities, but ultimately that is very dependent upon having a well configured and maintained BBS network around it which we don’t have.
Historically most “useful” PBBS traffic around here has always been local “sp” BBS mail (aka p-mail). Without a reliable global BBS forwarding network in place, a full service PC based BBS with a bunch of links is just overkill. A properly designed/maintained statewide packet BBS network could be useful for sure, but clearly that is unlikely to ever happen in Kentucky. Thus scaling back from a PC based PBBS just makes sense for around here. The newest Kantronics TNC firmware feature set provides some compelling reasons to consider the KPC3+ and KPC-9612+ units for a small local only packet BBS.
Out of state or regional messaging can be handled via other means like HF NBEMS. While I have some concerns over NBEMS and other manual P2P approaches for extended duration and/or high message load EMCOMM scenarios, they can be useful for moving non-local traffic if needed. If nothing else, with NBEMS we will at least have some assurance that non-local messages got delivered (or not). This is far better than “pressing send and praying” when using what remains of the very broken national packet BBS network.
The more I look at the evolving internet threat landscape out there, recent EMCOMM events across the country, our local needs, how packet is (and is not) being used for real EMCOMM needs, and how broken the “global” packet BBS scene is, the more I feel the “mini-PBBS” and NBEMS approach is the best for our area. As such, the BPQ32 BBS is being shut down in favor of a standalone TNC based approach. The Robust Packet HF gear will be put to better uses.
Q: Will my VHF/UHF Winlink nodes remain?
A: No. I feel without HF as a backup, VHF/UHF Winlink is usually a poor choice for serious EMCOMM messaging use.
The recent failures of numerous VHF RMS Winlink (RMS Packet, etc) sites during Hurricane Irene & the big Halloween snowstorm in the northeastern USA should be more than enough proof. For those watching closely during those two fairly predictable scenarios that became real life events, it quickly became obvious how fatally flawed most RMS Packet Winlink installations are. VHF/UHF Winlink may be fine for day to day tinkering if you wish, but most installations are no where near reliable enough for serious EMCOMM use.
My other issues with Winlink for EMCOMM use can be summed up as:
- Internet dependencies and security exposure of such a system. P2P capabilities do not address all of these.
- Software stability issues remain after several years of development.
- Software development “roadmap” is unclear and spastic at best.
- Little focus on being cross platform friendly.
- Poor programming platform choice (.NET).
- Gateway sysops frankly have little to no idea of what content is crossing over the RF side of their systems. High potential for abuse and rule violations.
- Fragile and mismanaged internet infrastructure under the system. Several related outages in 2011.
- System DNS continues to have configuration related issues.
- Winlink itself has a bad reputation for QRM and poor frequency choices.
- Winlink leadership is often arrogant or outright hostile when approached with concerns or questions.
- The HF MARS is the only facet of Winlink that makes much sense for serious EMCOMM needs.
I’m not going to say Winlink has no place in EMCOMM. The problem is the way most folks are going about it doesn’t make a lot of sense if we are really about being “the backup for the backup” when it comes to radio communications. IMHO, the HF side of Winlink makes the most sense for EMCOMM, but only when done right. Plus this is all arguably best suited for the HF MARS spectrum.
Moving on. With ham radio basically being “the backup for the backup” for our served agencies, I am growing more and more concerned about a trend that I’m seeing in too many AR EMCOMM circles. That trend is a growing dependence upon internet links, internet applications, and frankly any IP based transit in general. The rapidly evolving cyber-warfare threat landscape should make AR EMCOMM folks think twice about deploying/relying upon the internet or frankly any TCP/IP transit for critical EMCOMM date or voice transit needs.
Forget the usual “total internet failure” scenarios for a minute. Forget the DDOS attack scenarios that could take out or severely degrade key internet infrastructure. What if the internet is there but far too “hostile” to use due to a nasty terrorist or foreign government “sponsored” worm on the loose that is attacking a variety of operating systems and router/firewall flaws using a collection of zero-day exploits? Now imagine the CAT5 cable on every PC in your EOC or command post has to be unplugged for security reasons and risks from “internal” infected/compromised systems? Can you continue to communicate in such scenario?
Ham Radio EMCOMM/ARES/RACES folks might want to give these “cyber” scenarios some serious thought. If you think they are just theoretical then you haven’t been paying much attention to the threat landscape lately. These are some of those “when all else fails” scenarios where serious ham radio “help” will be needed. If we are “the backup for the backup” then will your group be ready to help out or have you set yourself up to be just as impacted as they are?
After a recent controversial dictate came down it is clear that even the Army MARS leadership is starting to shy away from Winlink due to internet exposure. There is a ton of spin out there on this topic from many folks with a variety of agendas. A lot of it was pretty comical to watch and was usually just attempts to ignore or distract from the key point in the recent Army MARS HQ memo. That point was the need to transition away from communication links and methods that use or have internet exposure and focus more on RF only approaches and standards.
Sadly most of the Winlink response to this MARS directive seemed more about dodging the point above and preserving the status quo versus an honest examination of valid concerns. I find it eerily similar to the challenges facing the NTS system and how out of touch to downright self-centered versus served agency centered a lot the response has been.
UPDATE: After a lot of FUD, wailing, and gnashing of teeth from the Winlink user base the Army MARS HQ brass essentially gave in for now. I have it on pretty good authority that HQ folks were appalled at a lot of the response to a relatively simple directive. They finally just said the heck with these guys as we have bigger fish to fry now and just let the “status quo” go for now. I suspect the final chapter of that saga has yet to be written.
Many ham radio EMCOMM folks need to wake up to the new realities of today’s world. The year 2011 was full of “wake up calls” for amateur radio’s current and future relevance in EMCOMM. Changes are coming one way or the other and ham radio can wake up, adapt, or continue losing its relevance. If we are to be the “when all else fails” communications go to guys/gals for our communities then many groups need a huge dose of reality on the threats out there. Unfortunately I fear too many will have to learn the hard way.
Do you still think Packet Radio has a future in EMCOMM?
Sure it does. With some of the advances in packet radio client software (Outpost comes to mind) it is more useful than it has ever been. Just a lot of folks/groups out there that need to come to grips with the fact that the failed approaches of the packet “Glory Days” back in late 80’s and early 1990’s are gone. Those days are not coming back and IMHO that is a good thing for those that can learn from history. As I’m noted for saying “Packet radio is not dead, it just evolved.”
I think the folks wishing for a big nationwide BBS network are going to continue to be disappointed. I used to be a member of that group, but at some point you have to face reality. Today the successful packet BBS systems will be the ones that meet a specific and defined goal versus just talk, wasting a lot of airtime moving content few if anyone is interested in, and generally just supporting the local utility company. They will usually be small locally focused BBS systems, maybe a statewide network at best.
Q: Will the KY Packet Network (KYPN) continue to exist?
A: KYPN has been around since 1992. We have had some form of packet radio on the air non-stop since then and I don’t foresee that changing. KYPN, at least locally, has not been disbanded and I have no plans to do so. It has “evolved” over the years and I suspect it will continue to evolve and adapt.
The KY Packet Mailing List is going away. There simply is not enough interest in a statewide packet network at this time, so it serves no real purpose.
Q: Will you be supporting other packet systems?
A: Not beyond the remaining Dry Ridge systems and maybe a local node in the south end of the county. I simply don’t have the time to take good care of my own systems plus help everyone else with theirs. Maybe some day when I’m retired
Q: What is this “MSS” you talk about?
A: As I’ve stated several times in the past few years “it’s time for Sysops to be Sysops or turn their stuff off if they can’t operate and maintain it well.” Poorly ran and maintained packet radio systems (of which we have many across the country) are not an asset. These systems do more harm than good and run off potential new users. I coined a term for the above. It’s called “Missing Sysop Syndrome” (MSS). MSS has done and continues to do a lot of harm to the reputation of packet radio, especially the global Packet BBS system. If you are not going to do it right, then pull the plug.
Q: Will this shutdown impact the Williamstown node too?
A: Yes, in fact I will probably turn it’s RF ports off over the weekend of Jan 7/8th.
The Dry Ridge node shutdowns will occur by Jan 20th.
Q: Jeff, I don’t care about EMCOMM use of my Packet BBS?
A: Great! That’s fine if you have and can keep enough non-EMCOMM users around to justify things. Hint, in most areas I wish you luck. Outside of a few sentimental “old hat” users, EMCOMM is probably your only remaining packet BBS user base. My experience is the “tinkerers” and casual BBS users come and go like the weather. I suspect you’ll just be supporting your local utility company most of the time. Sorry, but reality is what it is.
Q: Are you mad at anyone in particular?
A: Not mad at all. I guess somewhat aggravated would describe it better. My aggravations are mainly with the “global” Packet BBS scene, the widespread Missing Sysop Syndrome (MSS), and they have been stated for some time now. After a “New Years” review of where things stood this year I decided it was prudent to shut the full service BBS and Winlink gateways down, take a more sensible approach, and put my limited hobby time back towards my first love in the hobby – VHF/UHF DX’ing.
Q: Are you getting out of KY ARES?
A: Not at this point. When recruited by my EC, I was abundantly clear that I would not have a great deal of time for day to day ARES stuff. Some months I will have more time for it than other months. I often joke that ARES roles are usually best filled by someone “retired.” That “time” resource has to be applied to where it does the most good. Around here we have to make do with very limited resources on many fronts and I wear a multitude of “hats” anymore.
Q: You gave up too soon?
A: Ah come on who do you think you’re kidding? We gave things here in Kentucky a “2nd chance” starting back in the spring of 2008. I offered HF access, multiple 1200/9600 ports on a variety of bands, and modern software. I even offered loaner hardware (radios/modems/laptop/PCs) to anyone serious about setting something up and willing to learn something new. Sorry but I would call that nearly 4 years of effort. Next question please.
Q: Dude you are kind of harsh on this topic?
A. I realize that some in the packet/Winlink world would prefer I not voice my opinions, but that’s just not me. I come from a “unforgiving of failure” background in Public Safety. A communications driven world where “talk” is incredibly cheap and often dangerous. Plus for over a decade now I have made my living by cleaning up IT or communication system related messes. I often find that cleaning us these messes often starts by stating what no one else wanted to say. Some folks appreciate my brutal frankness, some will resent it. Sorry, you can like that approach or not…my paycheck remains the same.
WA4ZKO