The Hattiesburg Clinic in South Mississippi has its hands full taking care of over 500,000 Mississippians facing an array of health challenges. With such a burden, the last thing the sprawling clinic needs is file transfer problems.
Progress spoke with Josh Hazel, Director of Clinical Server Systemsabout the challenges they faced scripting file transfer solutions, how they overcame them with file transfer automation, and MOVEit which the clinic adopted in 2014. We also talked with Stephen Klauk, Data Extractor for the clinic’s database system.
Progress: How were you doing file transfers before MOVEit? What were some of the issues with your previous approach?
Hazel: We went live on a new EMR (Electronic Medical Records) system, EHR or Electronic Health Record system called Epic in 2011. At that time, we switched over to a Unix server environment and began scripting a lot of our transfer processes. We were writing all these Unix scripts and had directories everywhere.
It was time-consuming to write the scripts, test them, and be thorough and thoughtful with the process. It's not simply a copy command. You want to make sure that it's received and do all the failsafe checks and things of that nature. That was the driving factor.
We did that scripting for a while. The more our EHR grew — Epic is a very modular product — and the more modules we added, the more we found ourselves doing that sort of automation. It became very time-consuming.
Progress: You had to keep writing new scripts for new use cases with your EMR system?
Hazel: Exactly. Files were going to different vendors, different systems, or just doing different things. A year or so later, we brought on a hospital system we have a close partnership with. They came onto our EMR as a joint-type environment. That blew that up even more as far as more things to have to script. It just kept growing and growing and became very unwieldy.
Progress: Before you adopted MOVEit in 2014, were you already using WS_FTP? Or did you move to that in conjunction with MOVEit?
Hazel: We spun up the WS_FTP server at the same time if I'm not mistaken.
Progress: What is WS_FTP used for and how is that different from what you used MOVEit for?
Hazel: MOVEit is mostly transferring from one system to another, or from one system to a vendor – things like that. We may have a vendor that delivers stuff to us, in which case we use the SFTP server for that. We have a kind of a drop box form to connect to.
Progress: Who is using MOVEit?
Hazel: It's exclusively the technical team. We may have some departments that have some needs that we set up tasks for, but we manage it and use it. A lot of it is extracts coming out of our EMR system.
Klauk: We send out telephone-call extracts and send out claim extracts to our clearing houses. We receive back reports and files to pull back into our EMR system from a lot of our vendors as well.
Progress: How do your partners feel about MOVEit?
Klauk: They're tangentially aware we have it. We have often used MOVEit's tools to be able to go back and say, "Yes, we did transfer this file." And we can see, back and forth, if there has been trouble in transmission. We can use the logs to determine where the issue has been.
Hazel: It's mostly transparent to the recipient what product we're using. But I do think they have a level of comfort with the reliability. Let’s say there's an outage somewhere or our vendor's system is in maintenance mode or something, and we're not able to connect to their system. We'll get a notification from MOVEit saying, "Hey, this task didn't go. They failed to connect or whatever." They'll reach out to the vendor right away, and sometimes before the vendor even knows they have an issue.
They can reach out and make sure everything's good, and then resend the file. That proactive approach gives our recipients a lot of confidence. And we can go back and retrieve everything. We archive everything, so we're able to go back and retrieve files we send as needed.
Progress: How did you discover MOVEit? Did you have a general RFP for a managed file transfer or a secure file transfer solution, and look at a variety of vendors?
Hazel: When we sat down to write a new script for a new process, we would just pull our hair out and say, "We have got to find a better solution." But by the time we put the effort into getting those scripts done and tested and implemented, we'd be onto another project. It was hard to catch our breath.
When we brought that hospital system on in 2012, they already used MOVEit. As we were comparing notes for new processes, they told us about MOVEit. I think we downloaded a trial and loaded her up and were impressed with the simplicity of setting everything up and how much time it freed up being able to quickly and easily automate those tasks.
We queried them about the product and their experience. It was rock-solid, stable, easy to set up a task, and set up a host. They didn't have any issues with it. It did all the logging and auditing. All of that was already done whenever you just set up a new task, whereas before we would have to write that kind of stuff and create new log files every time.
It sounded like something that would alleviate a lot of our frustrations and free up a lot of our time.
Progress: What got you interested in the automation? Were you tired of doing all the scripting in three different types of systems, Windows, SQL Server and Unix, and MOVEit Automation eliminated all or most of the scripting? Was that what kind of got you interested in the MOVEit Automation portion?
Hazel: That is 100% what got us interested.
Klauk: It was a duck-to-water situation once we set up the server and set up a couple jobs. It was like, "This is so easy to do."
Progress: Obviously HIPAA is a big deal for your organization. Is there a HIPAA benefit to MOVEit?
Hazel: From a validation standpoint, it's easy to look back at reports and audits to see how something was transferred or what mechanism — SSH or whatever — was used, as well as validating encryption. When we wrote scripts, we knew that we were doing it. But if you had to go back and show somebody a report on it or get validation, it was cumbersome to pull that together.
Progress: How long would it take to write a script?
Klauk: For the processes I was doing, it would take a half hour to an hour to get a script together and get it tested on a Windows PC. With MOVEit, it's in minutes.
Progress: Then you have a process that took minutes to create, but once you start running it, it's proven, it's repeatable, and you can trust it.
Klauk: Yep, and it logs itself and keeps track of everything it has done.
Progress: When you create a new automated process, is there a big advantage in the fact that you've already done a bunch of others, so you're just modifying one for a slightly different use case?
Klauk: Definitely. A lot of skeletal copying of the framework goes on, and we just make minor modifications to where we're sending it to.
Hazel: Stephen's gotten real fancy with it. When I set up tasks and hosts and such, mine's pretty straightforward: pick up file from here, send it to here, save an archive copy, and send an email. Stephen's gotten real fancy with the programmatic part. He's parsing files and bringing in two different files from different sources and matching them up and sending them to a third destination. He's gotten real fancy with his task creations.
Progress: Stephen, you're doing all that right through MOVEit Automation without having to resort to additional scripting?
Klauk: For the most part. I have done a little bit of Visual Basic scripting within there, but MOVEit has made it easy to incorporate that.
One of the biggest examples was the statement files we had to send off to a vendor. They had to be zipped up into a single file and it had to have a file count appended to the name of the file. We were able to create a quick Visual Basic script that would put all that together and then use MOVEit's internal zipping function to put that file together and send it out.
Progress: Can you specify any benefits through stats or metrics?
Hazel: We haven't done that in a while. We're going on, I guess, seven years now. In the early days, we tracked back how long it took us for these various scripts we wrote for tasks, and then we would create a new script or a new process in MOVEit. And it would take seven minutes versus the 45 minutes to an hour or 90 minutes.
It may only be a 30 or 45-minute script you have to write but finding 30 to 45 minutes of uninterrupted time to do it was the challenge – finding those pockets of time. Scripting and programming in general is something that's hard to jump in and out of over a course of days. Say you're trying to work on a task, and you can't get to it for another two or three days. You’ve got to get back in and remember what you were trying to do.
With MOVEit, we set up the host one time and then anytime we have to interact with that host, the new task is point-and-click to where you want it to go. Early on, we did track, and we had significant savings. I don't recall the percentages, but it was 80-plus percent improvements just in time and resources. We've almost taken it for granted now because we've not had to do that in so long. This is now the standard for us.
We don't even think about writing those long, arduous scripts. We just go straight to MOVEit and create a task and set it up. It's a no-brainer. We'll do it while we're on a call with a vendor and set it up on the fly. It is a different environment we're in now because of this product.
Progress: How important is it to free up time through file transfer efficiency?
Hazel: Freeing up that significant amount of time allowed us to focus our efforts on other things, particularly when we've got an EMR the size that it is and growing.
Klauk: One of the best examples comes from looking at the dashboard for MOVEit. We've done 14,732 transfers today. Back with old ways — attempting to go through all the logs and stuff just to find out how many transfers we've done, or if they've had errors — took quite a while. Whereas with MOVEit, I can now see at a glance what's been going on for today.
Progress: How long would it take if you were even able to do that kind of tracking? How much time to get an approximation of what actually happened?
Klauk: We had three people doing transfers, and not at the volume that we do today. It would take somebody quite a while to go through all of that and keep on top of it.
Hazel: If we had to do that seven years ago, we would have to dig through log files of disparate systems, and different scripting environments and applications – and dig through to compile all that into a single report. It'd be time consuming. Again, trying to find time to write 45-minute-plus scripts is already a challenge. And then certainly, going back and doing an audit and having to compile that kind of stuff, for any given period, would be a challenge. It would eat up a lot of time.
And Stephen's right. The dashboard is great because we can jump in and immediately, from all the sources that we're sending data, see what that activity is. We can see any failed activities. With cash runs or file transfers, we can see it failed from the dashboard and immediately drill down into it, which is of huge benefit. All our tasks are set up to send notifications, some on success and failure, some just on failure.
We have a lot of systems we get alerts from and notifications from. It's easy to get into alert fatigue with receiving alerts just from all the different systems. I've got my Outlook set up to automatically sub sort alerts into different buckets, and it's still 15 buckets instead of coming from 15 primary alert notification systems. It is a lot to keep up with. Whereas if we have an insurance file that doesn't get transferred, those we want to know about quick. Having immediate drill-down access to these things is great.
Progress: Visibility is important to you. But do you also use reports for other people in the organization such as your bosses?
Hazel: As things are running smooth, we don't get asked for reports. But we've got this at our disposal to provide that information if the need comes up. As long as things are running smooth, it's one of those out of sight, out of mind things – nobody's really worried about it. If we had issues or something we needed to address, we would have this at our fingertips to share. But we just don't have a lot of issues at all with it.
Progress: What do you say to folks that don't have a regulatory reason to adopt Managed File Transfer?
Hazel: Just because you don't have to doesn't mean you shouldn't. They may not be regulated to have to be secure. It's just not a bad idea, and especially when it's this easy. It's no more difficult to send it secure and do it properly than it is to not. So why would you not?