So, you like Technology and all the really, really, cool things it enables? Me too. But Tech has a dark side that is rarely discussed. All the "0's" and "1's" that we love so much can all disappear in a New York minute. And so, if you do not have a robust disaster recovery plan ("DRP") enabled and in place (obviously just having a plan is not enough), then sooner or later the Digital Economy is going to shoot you in the head. Guaranteed. All knowledge workers have inadvertently deleted, corrupted, and/or otherwise lost a document that they were working on. As painful as that may be, that is not what this article speaks about. No! Here we are talking about a massive loss of data that cannot be recovered! We are not discussing a few hours (even days) of lost work on behalf of a knowledge worker. We are talking thousands, if not hundreds of thousands, of lost man-hours. Poof. Gone. And don't forget the Patient and caregivers! How will care be provided when there is little to no history of what has already been given to the patient, their medications, other tests, records, and results?
OK. But what does this have to do with compliance? Well, that depends on what compliance regime we are discussing. If it is HIPAA (or GDPR), and the lost data involves Patient records, then you are in a world of hurt. There is no way that you can comply with the HIPAA Privacy Rule. For example, if you cannot provide a patient their records upon request then you are in violation of the Rule. Section §164.524 mandates that you provide a patient with records that they have requested. The last thing patients should worry about is the privacy and security of their medical information, especially when they're about to be admitted to the hospital. Obviously, if the records are no longer available then you can't provide them. Moreover, if the patient(s) complained to HHS about this, then HHS would consider that to be willful neglect on its face. So? The so what is that HHS, under these circumstances, is required to audit you. Generally, an OCR investigation results in corrective action plans and fines to settle HIPAA violations.
You almost certainly will be found to be in willful neglect; wherein the Civil Monetary Penalties ("CMP") start at $50K per identical violation. Multiply that times the number of individual patients whose data was lost, and you are going to get a really large number. OK, so those of you that have been following along for the last decade might be thinking: "yeah, the CMP will be large, but it will be capped at $1.5M."That's the maximum amount that an organization could be fined per identical violation." Would this kind of data loss constitute a Breach? Probably not, but you will still get audited AND you will likely have to notify patients that their data has been lost. Given the current state of healthcare compliance, additional violations are likely to be found. You are going to have a bad day!
Don't trust and verify. I know that President Reagan said "Trust and Verify" concerning any nuclear treaty with the Russians, but when it comes to backups, it is better NOT to trust and ALWAYS verify. About a month ago I got an email from Dropbox saying that 35K files had been deleted. I mirror my server to Dropbox so needless to say I was a bit disconcerted. Was I panicked? No. Why? Because when it comes to data, I have backups of backups and some are stored offsite. I am paranoid about it. I can't afford to lose client files, plus all the intellectual property developed over more than a decade. Dropbox was simply a convenient way to always have a mirror of my server available in case I traveled, and, for whatever reason, I couldn't get access.
When I got that email, I quickly checked my server to ensure all was well and temporarily forgot about it. A few days later I had to send a large file that I couldn't email and so I wanted to send it via a Dropbox ("DB") link. Whoa. When I logged into DB something looked wrong. I can't see all my files. Something is definitely wrong. I contacted DB support. They confirmed that 35K files had been deleted. I informed support that I still had those files on my server so how could they possibly have been deleted? First things first. Because it had been less than 30 days since the "delete event" DB said they could restore the files. I told them to go ahead and restore. They restored all 35K files. But things still did not look copacetic. There were folders that I knew existed on my server that I couldn't see in DB. So, I start the whole support dance of providing screenshots that demonstrate that what I am telling them is true. Apparently "on their end," they say they can see the files. I said, well that's nice but I can't. Why don't we do a "screen sharing" session so that DB support can see what I see? Nope. DB support says for security reasons they can't do that. What? They already have access to my account. What security reasons?
I tell support to escalate. Once escalated the dance continues. More screenshots. More proof. By now I have lost track of which DB support person I am talking to. In any case, they send me some links, not to the folders that I can't see, but to files beneath said folders. I click on the link and voila, now I can see my files. Life is good. I am ready to sing DB's support praises. Not so fast. I log in a few days later and I am back to not seeing the folders I couldn't see previously. Plus, now I am seeing duplicate folders. Things are definitely not right.
I am using a Synology NAS (Network Attached Storage model DS412+) to sync to DB. I decide to contact Synology support because the finger-pointing had already started. DB says that Synology is using an "unsupported API" which is news to me. Really? Unsupported? Evidently, DB just wants you to sync from workstations (e.g. Macs or PCs) but not from a NAS. I tell DB that that strategy is "brain dead" because for most businesses their mission-critical data is stored on a server (i.e. the NAS is actually nothing more than a file server). So here I sit. A month later. Still no resolution. I have spent about 15 hours "messing around" with support and there is no end in sight.
OK, because I have backups of backups, I have options. I can cancel DB and look for a more robust solution that works with my NAS. In fact, I believe Synology has some solutions that address this issue and that is likely the path that I will take. That said, I was a huge fan of DB. They were (and are) a market leader. I thought it was a safe bet using their "stuff." The moral of the story? There are no "safe bets," only the paranoid survive. Don't trust and always verify!
In the end, DB was not able to make me whole. I am going to cancel the service and look for another solution. DB support was diligent BUT apparently, things were so badly broken that Humpty Dumpty was never going to be put back together again. I can assure you that you will either see (or read about) this movie repeatedly going forward. We trust vendors, even arguably high-quality vendors, way too much. Don't trust and verify is a much better way to go!
