For weeks, users had been getting randomly disconnected when using Microsoft Dynamics CRM software, and losing their data. The IT team couldn’t figure out if the problem was in the database, on the web server or the network. And the network management software they were using at the time was offering no help.
Their 2 month to-do list went something like this:
Review event logs line-by-line, trying to correlated events with reported problems
After exhausting all options, someone on the team downloaded a free trial of WhatsUp Application Performance Monitor software. After installing and configuring the software, the IT team isolated the problem to a change in a stored procedure that dramatically increased the SQL server query runtime. It turns out a DBA loaded a new stored procedure on a production system rather than a test system.
If you’re experiencing intermittent application performance problems, you may want to avoid asking Google for help and consider the solution found by the folks in this tale.
Steve Wexler, who writes the excellent IT Trends & Analysis blog posted yesterday on the topic of shadow IT (aka ‘cockroach IT’). We’d like to share excerpts from the blog that feature the perspectives of our president Ennio Carboni. But you should definitely check out the entire blog too for interesting perspectives and statistics from industry leaders like Cisco and Symantec.
From Steve’s blog:
According to a PwC survey, 50% of IT managers admit that half of their budget is wasted on managing Shadow IT (the unsanctioned use of IT products and services), said Ennio Carboni, President, Ipswitch, Inc. [network management division]. The ‘inability to track, manage and eliminate unapproved software and devices drastically eats bandwidth, slows networks and increases the IT financial/resource burden.’ Gartner says by 2020 90% of IT budgets will be controlled outside of IT and Forrester says IT could be obsolete by then.
However, Carboni said he doesn’t consider Shadow IT as much a problem as a business opportunity for IT. One of the disciplines he’s been able to keep alive is to cold call customers every week, and he speaks to hundreds every month. “Shadow IT is a real trend… (and) I’m a Shadow ITer.”
Carboni said the reason why Shadow IT is growing is because the IT process today for most companies is wrong. Rather than the budgeting approach we’ve been using for the last 20-30 years, organizations should ask what approach do we need that would enable our employees to be more efficient and effective. “I actually view this trend as a positive one, the consumerization of IT.”
In general, IT needs to embrace Shadow IT and become a positive force for standardization through listening, he said. “It’s very inexpensive and very easy for employee to go out there and take on a SaaS-based solution for a short time. Instead of having CISOs appear as a version of big brother, they heed to work on enabling the workforce to bring their own devices and managing the risks.
“After speaking with hundreds of customers, I believe security will go extreme, it will fail… and then we will have more reasonable and progressive policies”. He believes it will evolve like virtualisation, becoming much more proactive and with greater accountability.
Ipswitch offers four steps for minimizing the impact of shadow IT:
Flagging unauthorized apps before they cause problems;
Perfecting visibility into network bandwidth utilization
Monitoring for rogue devices – who is accessing what, when and via what device; and,
Identifying root causes of outages and slowdowns faster and speeding time to resolution.
Businesses exchange files. That was the basic premise of my last post, and the foundation on which I made a case for Managed File Transfer (MFT) as a critical category of infrastructure software behind B2B processes. I’d like to expand on that this week, to talk about the role people play in file-based business processes, and what that means for MFT.
Business processes (sometimes) involve people. Seems simple enough. So what?
For starters, that means MFT systems have to support people with tools that help them to do their job, while protecting all of the things that make MFT necessary in the first place. To do this, a complete MFT system needs a range of client options that allow end users to exchange files simply, and transparently. Along with this, they need the peace of mind that, in the background is machinery to handle:
Reliable and verifiable receipt of delivery
Visibility in all the comings and goings of files across critical, file-based business processes
Non-repudiation and proof of file integrity
Imagine an auto insurance company handling its most basic business process: claims adjustment. Without getting into messy details, the basic process involves the receipt of claims and validation that the claim can be paid, often after several loops of workflow with law firms, body shops, the customer, and so on. Files change hands in these workflows – and all of them are in some way material to the claims-adjustment process – as the insurance company works toward a final disposition on the claim.
The process involves a number of people, from the claims adjusters at the insurance company, to the clerks at partner law firms, to the poor customer sitting at the requesting end of the workflow, wondering if he is going to have to pay out-of-pocket for the unfortunate incident in the mall parking lot (I’ll call him Joe Fenderbender). Files will likely flow between several pairwise combinations of these players, and there may even be multi-party access to the same files at some point in the process. But the players are not equal partners in the exchange, and have different needs when it comes to tools to support their role.
Email alone isn’t sufficient for getting business done
Today, a lot of business gets done with email, and a lot of files move as attachments. Consider that a vast majority (84%) of the respondents to our third annual survey about data security send classified or confidential information as email attachments. But could you possibly imagine a worse tool for structured file exchange? Just think about the signal-to-noise ratio, for starters. Not to mention, most mail solutions handle very large files poorly – a real challenge in an age of HD video and tens-of-megapixel cameras masquerading as smart phones. Mail servers were never meant to be content-management systems. While it may be possible to conduct business via email, it is probably not going to serve every aspect of our example process equally well. However, email likely plays a role.
Consider, for instance, file exchange that takes place between the claims adjuster and our hero, Joe. It is probably reasonable to assume the insurance company has no control over the technology on Joe’s end of this exchange. So the easiest thing to do may be to employ plain-old mail as a way to communicate status, or receive/deliver materials. For security or compliance reasons, the insurance company won’t want to do this using traditional attachments (consider for example if Joe has whiplash, which could introduce patient information into the equation). Instead, they will employ a secure-attachment option that provides Joe with a link to files securely stored in the MFT repository, for his eyes only. The company may even want to provide him with temporary access to a Web-based upload/download space where files can be staged for the duration of the adjustment process. You can imagine the role email might play in this workflow of the process, since Joe is a temporary participant, with unknown skills and equipment on his end of the exchange.
Managed File Transfer streamlines file-exchange workflows
For contrast, consider the relationship between the insurance company and a partner body shop. In this case, the relationship is a little more permanent, and the two partners will likely have a more streamlined workflow in place, possibly with a durable shared upload/download space, and client technologies that make exchange very easy. In this case, the exchange might be better served by traditional FTP clients, or possibly a simpler background-synchronization option that links folders at the two parties’ locations through a central store, hosted by the insurance company. Email may still play a role, but because of the volume between these trading partners and the durability of their relationship, it makes more sense to use tools that are more tightly tied to the MFT system. These tools – and this integration – should make their exchange workflows quicker and simpler, so that all exchange is bagged, tagged, and verified in a more structured way with minimum friction.
You could even imagine a claims investigator from the insurance company employing his smart phone to collect photographic evidence at the body shop, or at the accident scene. A mobile MFT client would tie those collected files directly to the claim to speed processing, and keep all the evidence and artifacts in one place, stored securely.
Let’s pull back and take stock for a moment. In one simple business process, we have just imagined several types of file sharing workflows, and several types of tools to support those activities. We’ve covered:
Web-based file upload/download, and structured storage for universal access
Folder synchronization for simple, durable exchange between tight partners
Traditional FTP clients for cases of bulk upload, or automated exchange
Mobile access and file origination for employees in the field
My simple thesis is: All of these tools are necessary for end users of an MFT system today. And it follows that a complete MFT system will support a variety of exchange models and tools to make these options possible.
The digital world changes quickly, and IT departments find themselves on their heels a lot these days. IT has to serve the core processes of the business, protect the business with security and compliance coverage, and do all that with tools that bring end users along for the ride.
When it comes to MFT, bringing end users along has become more of a challenge than it used to be. That’s because end users have been targeted over the past few years by a number of consumer-grade, cloud-only file sharing services offering incredible ease of use for a very narrow synchronization and sharing use case.
As research firm Ovum states, “The shift in the balance of power from corporate-led to consumer-driven IT innovation has in part been caused by the cloud, since it has provided consumers and line-of-business managers with an alternative range of services that run independently and are competitive with the portfolio of applications provided from on-premise corporate IT infrastructure.”
This movement has largely been driven by the proliferation of mobile devices, and the desire to have all of one’s stuff on any of one’s devices, available at all times. It’s no wonder a Gartner survey found CIOs expect mobile technology will be the most disruptive force in the enterprise for the coming years. Users have eaten this stuff up, and that has led them to expect a level of usability, polish, and capability when it comes to the exchange of files.
That users employ these tools for their own “personal cloud” is fine, but when usage bleeds into business workflows, IT managers tear their hair out. These services store critical data offsite, have no guaranteed security, and are not under the visible control of the business. But the damage is done, and IT knows whatever they deploy to gain back control over business file transfer has to meet the changed end-user expectations for ease of use, convenience, and seamless fit in users’ existing workflows.
Modern MFT systems like Ipswitch MOVEit provide a full range of client options, including all of the options mentioned above, to serve our customers’ business-critical workflows – even the workflows that involve people who expect more today than ever before.
I appreciate feedback. Are you concerned with the tools your people are using to move files? Are you confident that they enable your employees to be productive while ensuring IT and the business meets their security, internal auditing and external compliance requirements?
Today’s tale from the front lines of network management comes to us from a company that’s been in business for eleven decades. They’re a logistics company with offices around the U.S. And they literally started with a horse and a cart. They have bought several companies along the way, keep their eyes on the future, and learned the value of technologies that help them grow.
Buying other companies means buying incompatible systems that are managed with different tools. The IT team at the company decided they needed to do three things before system issues got out of hand:
Choose one network management product
Centralize monitoring for all locations
Commit to an internal service level agreement
They agreed that the network management product they picked would have to scale for dozens of locations and hundreds of devices, handle the distributed environment, and give them the ability to be more proactive. They assumed they’d buy a solution from one of the big vendors, but, as one IT manager said, “Not only were these solutions high-priced, they required extra dedicated staff and extensive training. It was not cost effective for a company of our size.”
Then a network administrator spoke up and recommended WhatsUp Gold from Ipswitch. He had used it at his last company. According to the IT manager, “We downloaded a trial and liked it right away. We quickly saw that WhatsUp Gold could monitor what we need to monitor – it had what we needed without unnecessarily complicated features or excess cost.”
So the company that started with one horse and a cart, and grew because they knew how to look ahead, picked WhatsUp Gold. Scalable? Check. Distributed monitoring? Check. Support for proactivity? Check.
Oh, and the price not only fit within the budget, they actually saved money by replacing lots of licenses for different tools with just one license for WhatsUp Gold Premium and the plug-ins they needed for managing network bandwidth, configurations and applications.
Todays tale from the front lines of network management comes to us from the UK. When a new network manager joined a British toy retailer with a huge store in London and nearly a dozen more spread throughout the UK, he immediately came face-to-face with complaints that business users were being dropped off the network intermittently.
He frustratingly tried to chase down the intermittent drop-offs that would later seemingly resolve themselves. He found the firm had purchased Ipswitch WhatsUp Gold network management and monitoring software some time before he came on board but it hadn’t yet been put into production.
Although he was new to the product, he and his team were able to set it up quickly. WhatsUp Gold attacked the intermittent performance problems, performing a trend analysis linking performance metrics to the dates and time when a user got dropped from the network. This helped uncover the root cause and prevent future occurrences.
The IT team leveraged a single dashboard view to see the status of every device and real-time alerts to catch problems early. Combined with reports to correlate network performance with network problems, the mostly generalist IT staff can optimize network performance without specialized trouble-shooting skills.
IT service levels boosted
WhatsUp Gold continues to make his team proactive in addressing and solving problems quickly. He reckons that his team’s quality of service has risen to the same or higher level than larger, more specialized teams he has managed. “WhatsUp Gold deserves credit,” he says. “It does everything it says it does on the box.”
If you want to play with network management software from Ipswitch, download WhatsUp Gold free for 30 days.
Today’s tale from the front lines of network monitoring comes from a sysadmin from a large CT-based financial services company. One day not long ago he was performing routinely scheduled website maintenance at his office in Tampa, FL. Everything seemed rather normal. The web servers were humming along.
Until the complaints started pouring in. About a problem they couldn’t see. And neither could he.
As it turns out, users – including middle management – hadn’t been able to get access to the website for 45 minutes. And they were seriously unhappy.
“Naturally, I was blamed for the outage,” he reports.
He used WhatsUp Gold from the network management division of Ipswitch to see the problem he had only heard about. A few minutes later, after checking out the single dashboard and glancing at a few reports, he saw that HTTP/HTTPS/Ping had gone down on several sites. He traced the problem to an F5 load balancer issue.
In a flash, he restored access to the site.
“[Without WhatsUp Gold] it would have never been solved and I would have been blamed for a production outage!CYA – cover your ass – because the blame was pushed off of me and my maintenance!”
“Also, people actually being able to work again was a nice bonus.”
When it comes to understanding the value of Managed File Transfer (MFT), sometimes the best approach is to see how it has helped organizations address their real-world issues. In this case, we look at how a publicly funded healthcare organization – NHS Wales – tackled security challenges associated with the transfer of highly sensitive data.
Healthcare organizations responsible for protecting such data have their work cut out for them – they mustcarefully rationalize and secure information flow while working to optimize patient care and manage operating expenses. It’s not an easy task.
As part of providing care services for three million citizens of Wales, NHS Wales is tasked with securing files and data without impeding its 65,000 staff nationwide working across hospitals, university hospitals, GP surgeries, clinics, dental practices, palliative care, pharmacies and more.
Like employees in many organizations, NHS Wales’ staff struggled to securely share information between sites around the UK – especially when large file sizes or sensitive information were involved. As a workaround, they resorted to posting paper copies of records, or couriering discs of MRI scans and x-rays.
NHS Wales knew it needed a better way of working, one that simplified daily tasks for its employees while also meeting compliance goals for ISO security standards and UK data protection requirements.
The organization found its solution in a robust and secure managed file transfer system. Once NHS Wales put the new system in place, “More and more agencies began asking us if they could take advantage of the service too. We then began rolling it out to other health boards and trusts so that they could also share information securely with their third-parties, such as police crews, ambulance operators, social services, and others – and we simply haven’t stopped!” reports Andrew Glencross, senior IT security specialist at NHS Wales Informatics Service.
As an IT analyst firm, we query companies large and small on a range of issues. One of the areas of risk we consistently see is around the transfer of files associated with business processes. To understand why these risks exist, we need to explore the difference between simple file transfer and managed file transfer.
The Importance of Managed File Transfer
Business processes are dependent on the transfer of files – important business records sent between applications, between people, or between applications and people. A strong case can be made that the vast majority of so-called “communication” networks are really much more about transferring files than they are about sending messages or other types of communication. This is particularly true since most file transfers occur between applications and not between people: roughly two-thirds of file transfers – for content like purchase orders, invoices, travel documents, tax information, etc. – are sent business-to-business, not user-to-user.
File Transfer Needs Improvement
File transfer is so integral to the proper flow of business processes and corporate communication that it must be a high priority for any company. Yet, there are numerous problems with existing file transfer processes including the following:
Many files are sent without encryption
IT cannot control the lifecycle of the transferred files
Lack of auditability for the file transfer process
Some content is not archived in accordance with corporate policies
Chain-of-custody cannot be maintained for some content
Transferred files cannot be inspected by Data Loss Prevention systems
This lack of control results in a greater likelihood of data breaches and a breakdown of business process efficiency.
The “M” in “MFT”
Clearly, companies of all sizes need to address these problems. They can do so by implementing a file transfer process that can be managed in accordance with the variety of requirements that enable them to maintain the integrity of data as it is being transferred within and between organizations. However, this is usually not possible via typical file transfer solutions like traditional FTP or many of the growing number of cloud-based file transfer tools because of their inherent limitations. Instead, true, enterprise-grade file transfer requires MFT, which is distinctly different from typical file transfer solutions in the following four ways:
Security An MFT system enables the transfer of files using secure protocols that will encrypt content both in transit and at rest. This is essential in order to maintain the integrity of information as it passes from sender to recipient, and as it is stored on various servers.
Compliance An MFT solution allows an organization to maintain compliance with the growing number of statutes that are designed to protect sensitive information from being intercepted by unauthorized parties.
Control A key distinction of an MFT system is the control that it allows over content: its expiration, who can access the content, where it can be sent, and the ability to report on content flows, etc. In short, MFT solutions permit complete control over the lifecycle of content in order to minimize the risk of non-compliance or loss of sensitive content.
Integration with workflows
An essential element of a true MFT solution is its ability to integrate with corporate workflows to ensure that content can be sent in support of corporate requirements. For example, a purchase order system that requires the sending of purchase orders and other documentation to recipients must integrate seamlessly with this system in order to minimize disruption with existing processes.
The Next Generation of File Transfer
File transfer is changing as organizations migrate away from insecure, legacy FTP systems; email (which has become the de facto file transfer solution in many organizations); and lightweight, consumer-focused file sharing solutions. Instead, companies are moving toward true MFT solutions that:
Integrate well with existing corporate workflows and content-transfer processes
Allow IT to maintain control of the entire file-transfer lifecycle
Ensure appropriate corporate governance for all content
Enable end users to employ file transfer simply and efficiently
I will have the pleasure of discussing these issues along with two individuals whose organizations have recently implemented MFT solutions at a Webinar on August 22nd and would welcome having you join us for the discussion.
To learn more view the on-demand version of the webinar:
Today’s tale from the front lines comes from a customer who manages a dispersed wireless network for a large U.S. city’s public school system.
To give you an idea regarding the scope of the network, it has more than 4,500 wireless access points across the city. Users’ personal devices (aka BYOD) play big into the wireless infrastructure with a wide array of laptops, tablets and smart phones attached to the network.
Before the school system purchased WhatsUp Gold Network Monitoring and management software, every time users reported problems with wireless availability and performance, members of the IT staff had to get in their cars and drive around the city in order to diagnose the issues. This was not an infrequent exercise. It was daily.
As if IT folks didn’t have enough to do around the office, this team had to get behind the wheel and hunt for needles in a haystack dozens and dozens of city blocks wide in order to make sure students, teachers and administrators had a strong wireless signal.
The school system implemented WhatsUp Gold and put its integrated wireless infrastructure management features to immediate use, including:
Real-time alerts for access point health.
Dynamic network maps to identify users and devices connected to each access point.
Out-of-the-box reporting on user and device bandwidth consumption
A single dashboard for displaying access point subscription, signal strength, hardware health and more
We were pleased to know the hard working team had traded in their cars’ dashboards to navigate them through the wireless morass for a single dashboard located inside their office.
One evening not long ago, Fred took his laptop home and visited adult entertainment websites. Before logging off, his machine was quietly infected by malware. Fred came back to work the next morning and connected to the network.
Soon thereafter the network started to crawl.
Application performance became intermittent.
And the help desk got bombarded with user complaints.
To sort out the problem, the company’s IT manager used Flow Monitorfrom Ipswitch for network monitoring and traffic analysis. A quick analysis provided a graphic representation that showed the source of the problem. There was a very large amount of UDP traffic coming from Fred’s laptop. The machine was removed from the network.
Network and application performance went back to normal. And IT quietly shared a few best practices with Fred.
Just what is managed file transfer (MFT)? It’s easy to think of MFT as little more than file transfer on steroids, or a super slick FTP server. But MFT is more than that because the problems IT administrators solve with MFT demand more. Our customers don’t move files for fun – they move files to get work done.
MFT is a category of middleware software that ensures reliable, secure and auditable file transfer to enable critical business process. But even though File Transfer is at the core of MFT, it’s the M in MFT that sets the category apart.
Back in the Day…
There was a time when an organization in need of file transfer infrastructure would reach for a basic FTP server by default. That was the answer if you needed to make files available to partners, create a space where partners could drop files into a process, and script all around those activities to keep things moving while maintaining some sense of security. But as file volumes went up, and the range of processes that involve file exchange broadened, so too did the number and variety of software solutions that could help to accomplish the goal.
In recent history, we have seen the emergence of a new category, so called Enterprise File Synchronization and Sharing (EFSS). This category of mostly personal tools helps individuals share files between their myriad devices, including smartphones, tablets, home and work computers. While easy for end users, these mostly cloud services have become a real problem for IT departments. That’s because the simplicity, openness, and device-friendliness they allow come at a real costs to the control, visibility, and security protections that are the IT department’s responsibility.
New Demands for Security and Compliance
In addition to pleasing end users, IT also has to please the businesses they serve, and on that side of the ledger things have grown more complex too. Today, the variety of business processes that depend on reliable file transfer is up and the volume of transfer activity is up. The need to manage all of this activity under a tighter security and compliance regimen means nothing can be left to chance.
Where simple FTP was once sufficient, today IT has to reach for more capable infrastructure that mixes the end-user simplicity of EFSS with the reliability of FTP and the business-process focus of integration middleware. But they need to do this in a way that doesn’t inadvertently make what has traditionally been a solvable problem into a messy, bespoke custom development situation. The last thing they want to do is to engage “solutions vendors” with their bag of forty tools, complemented by expensive internal developers and systems integrators.
This is where MFT fits in.
MFT is a purpose-specific class of middleware focused on the reliable transfer of files between business parties, using simple, secure protocols and easy-to-understand models of exchange. But it’s fortified with security, manageability, scalability, file processing and integration, and business-reporting options that allow IT to deliver more sophisticated, controlled file-transfer solutions without slipping into the custom-code abyss.
In a series of upcoming posts, I and my colleagues will explore each facet of MFT, including:
Tools for end-user access: The ways users can participate in MFT-driven business processes using the skills they already possess, and tools that leverage already familiar activities, like sending email attachments or working in local folders.
File-transfer automation and workflow: Explores the ways that file transfer can be put to work, either through the handling and preparation of files for further processing, or the standards-based handoff of files, metadata, or both to the next step in a business process.
Reporting and analytics: Will look at the importance of visibility into the volume, history, and current activity of a 24/7 MFT flow into and out of your business, and the importance of end-to-end visibility in linking that traffic to your business.
MFT administration: Will explore a range of topics, from security and compliance to topologies that deliver high availability, performance under load, and efficiency of operations.
Ipswitch customer engagement engineers and members of our WUGspace community have been sharing their stories from the IT front lines. These brief vignettes cover a network-related problem that was solved using one or more of our products.
These stories aren’t meant to be commercials (we have plenty of room elsewhere on our website for that) but more of an insider view into the day to day challenges faced by IT pros and how they can make their jobs easier to do.
Today’s edition stems from an engagement with a fast food franchise where our folks came to help with a network Internet bandwidth issue.
For those of you who are WUGspace community members, share your own story from the front lines and immediately earn 100 WUGspace points and a $25 gift card. If you aren’t a member, we’d love to have you join in the dialogue.
True Stories from the Front Lines of Network Performance Monitoring: How a Fast-Food Franchise Closed the Lid on Pandora’s Box
We recently worked with folks from the corporate office of a national fast-food franchise who were very concerned over their Internet bandwidth costs. They were considering purchasing additional bandwidth to keep up with demand.
They came to Ipswitch to help understand the source of their growing internet bandwidth consumption so they could address the issue and gauge how to solve it.
Using our Flow Monitor software, we were able to identify the source of the problem in minutes. The culprits were users streaming Pandora from the desks.
Turns out a little satellite radio can really chew into the bandwidth that others need in order to do their jobs. If the franchise had bought more bandwidth they’d be doing so in order to support an unauthorized application.
Using Flow Monitor, they were able to enforce bandwidth usage policies and detect the use of unauthorized applications – and lowered the bill from the ISP.
By the way, we’re music fans.
Just wanted to put that out there in case we were giving the wrong impression.