Remote User Experience Monitoring

How can you monitor user experience from home?


The last several weeks have been rather complex. As I imagine you know, a new virus popped up on the world stage and it has been making its way around the world. Good hygiene, isolation to prevent spread, and keeping yourself healthy go a long way in keeping us all safe when viruses arrive. As companies are faced with keeping their employees and their customers safe, many are cancelling large events until the outbreak calms down. As an example, Microsoft has cancelled several Ignite The Tour events, including Amsterdam, which should have started tomorrow:

Companies are also making work force adjustments, including having employees work from home in cases where this is possible. Examples of large companies promoting working remote is heavily highlighted in the news. But smaller companies do not always get a lot of publicity. Large or small, how do you monitor the user experience when your user population is remote?

Intelligent Remote User Experience Monitoring

Here at Perfrax we make a rather intelligent user experience monitoring solution. The solution was designed to be the easiest to deploy to every user’s device while gathering the most accurate information possible. I may be biased, but I think we hit our goal rather well. I am a geek by my nature, so I enjoy seeing technology truly shine. And shine TrueDEM™ has. With no knowledge of world events, and no understanding that more people would work from home, TrueDEM™ started noticing a change in response times. Instead of sending endless notifications about response time changes, the AI engine looked a little wider as we have taught it to do. And it focused this to a change the distribution of ISPs being used. That is actually the right analysis, because we are seeing a change in the mix of ISPs at customers both large and mid-sized.

TrueDEM’s AI identified a change in the mix of ISPs in use to be the cause of increased fluctuation during business hours. Can your product do that?

I would generalize the change as less “corporate-grade” connection-hours and more “home-grade” connection-hours. In my experience home connections can experience more latency and higher variability. Personally, I think this is absolutely acceptable. You pay a fraction of the price for a home connection compared to a business connection that offers enhanced latency controls. In this circumstance our data does back up my generalization, we do see an increase in latency and variability, in most cases. (Although some users do have a better connection at home.)

Let’s forget about the global issues that are causing temporary disruptions, and instead consider companies that always have large work from home employee populations. Such employers live the challenge of monitoring remote user experience every day. This is not a spike due to world events, it’s their daily existence. This existence presents a new monitoring paradigm for a distributed employee population. As a company we live this same existence. So we documented a few truths you need to know.

The Truths of Remote User Experience Monitoring

Truth #1 – Simulations in Company Offices Can’t Help Remote Users

If you are using only synthetic transaction monitoring, or consider it to be the primary type of monitoring you use, I have a question for you: How can artificial transactions, running from a location where your users are NOT working, deliver an accurate picture? Answer: they can’t. But don’t take our word for it, simply read the marketing materials from vendors who primarily focus on simulated transactions. They will remind you to put the probes where users are located. But in a remote workforce scenario, the users are not in the office and a robot/probe/agent ends up with almost nothing in common with the user you need to support. Synthetic Transaction Monitoring operating from company offices has always delivered limited value to begin with. But this monitoring methodology is highlighted as particularly inadequate when we need to monitor employees working remotely.

Probes, agents, devices, robots, even our own TrueDEM™ Canaries, running in the company offices deliver ZERO value for the users working from home.

What happens in a situation like we have now, where there is a sudden need to work remote, do synthetic transactions help then? Actually, they might be even less useful in today’s situation. Statistically speaking, the probes or agents on this network will have the best “work hour” performance they have seen in a long time. Why? Your users are working from somewhere else. As such a lot of load on corporate connections is now missing. Less load leaves more available for your probes. And in this case synthetic transaction monitoring products will show an improvement in performance. But your users are working from home connections which many times do not respect QoS tags. Or prioritize voice traffic over say, your kids streaming video on demand content.

I will note that simulation based or synthetic transaction monitoring (STM) is not devoid of value in all cases. When synthetic transactions are used with a true monitoring product, it can be a nice-to-have. It can provide an additional context that can help in a subset of support needs. But you shouldn’t be paying much for the additional context offered by synthetic transaction monitoring. In fact we offer our synthetic transactions – running on TrueDEM Canaries – for free. When this reasonably simple data is combined with the more advanced data direct from users and their devices, it can add some interesting contrast to the data you have. But it is never the starting point. After all, monitoring user experience by very definition requires monitoring users.

Truth #2 – You Must Monitor EVERY User Device

To support users in a remote workforce, a company must be directly monitoring the user experience on the user’s devices. You can not short cut the telemetry and still provide acceptable support to your user population. A product that gathers actionable data, direct from the user and their device is the only digital experience monitoring solution that will help you and the people you need to support.

For emergency cases such as the ad-hoc work from home scenarios we see today, users may need to use a personal device to connect to work. After ensuring the device clears the necessary security protocols, you must then be able to install and monitor personal devices – without violating their privacy. A monitoring product must be able to gather user experience data for work related activities, and not gather extraneous information.

Truth #3 – Installation Must be EASY

A digital experience monitoring solution that gets deployed direct to user devices must be easy to deploy and manage. The installation, patching, updating and support must be possible whether the user is in the office or at their home. Sadly, with many real user monitoring solutions you need a fleet of consultants to install the product. Heck, with some of the synthetic transaction monitoring products you need hours of phone support to install a single buggy robot probe. That doesn’t work well when employees are in the office. Customers of dedicated probe based monitoring products rarely find the time to get probes deployed to all locations. But remote user experience monitoring? Literally impossible. With a remote employee population, as you cannot send IT staff to everyone’s house. The software must install easily, work every time, without local support or hours on a support call.

If your company is a work-from-home enterprise, how are you solving this challenge?

Can your digital experience monitoring solution be installed remotely in seconds? Is is private enough to capture data only related to user experience with Office 365 or your other SaaS platforms? Is it smart enough to detect an increase in variance, on the real user experience, gathered on the actual user devices? And then isolate this to a change in the distribution of ISPs in use? All while understanding not to issue mass alerts? Users are not having a poor experience. But the change is important enough that IT should be provided an informational notice as the trend may be something to ‘keep an eye on’ If your current product cannot do this, you are using the wrong product.

Want to Monitor Remote Users, for Free?