Remote Assisted Reality Adds A New Data Dimension to Enterprise Applications

For over the past two decades, corporations worldwide have invested millions of dollars in long drawn out multi-year programs implementing large scale enterprise applications such as ERP (Enterprise Resource Planning), CMMS (Computerized Maintenance Management Systems), CRM (Customer Relationship Management), MRO (Maintenance and Repair Operations) systems. Over the years the footprint of these applications have grown in maturity, accuracy and have for most practical purposes are believed to have served their purpose well for most organizations who have embarked on these programs.

Business processes have been re-engineered with increasingly lean models, generating higher efficiencies, lower cost of operation, reducing cycle times and waste and providing significant benefits when coupled with the state of the art enterprise applications. At the root of such improvements you will hear phrases such as single-source-of-truth, system-of-record, real-time-system-integration, workflow automation, data governance and stewardship, audit trails, data accuracy and the like. These controls and measures have done are still continue to deliver massive gains for corporations globally.

However all of these enterprise applications are reliant on alphanumeric datasets and in some cases “blob” objects if the underlying database supports such data types. In cases where the database supports “blob” objects at most you are talking about recorded multimedia files such as videos, pictures or even audio clips being captured through a variety of capture devices and somehow being uploaded in the database to be connected to the appropriate transaction. Very rarely do applications support meta-tagging of such multimedia in order to make them searchable through search algorithms for future analytics and reporting. Lack of such tagging or data mining algorithms renders these multimedia content grossly under-utilized.

In no circumstances however, have any of these enterprise applications in the ERP, CMMS, MRO, LIMS, CRM suite of products, achieved any level of maturity in their support for collaboration through live streaming of real time scenarios, to solve real life business problems occurring in field operations. The reliance for these applications for live collaboration is in asynchronous CRUD (create, read, update, delete) transactions, stretching to live chat / messenger services, phone calls or Heaven-forbid low resolution Skype / Facetime calls. Technicians on location or the customer making a support call has to make a valiant attempt to verbally explain the presenting problem to a remote supervisor. The supervisor in turn has to make judgment calls based on a text based or verbal description – quite a risky propostion and prone to errors. There is nothing more immersive than a direct audio visual view of the presenting problem.

This is exactly where Remote Assisted Reality wedges in to fill the void of the installed enterprise application software suite. RAR does this by extending the static alphanumeric data of a service ticket into a rich and immersive experience of the situational intelligence streamed live, securely, in high definition audio/video from location to one or many remote experts duly qualified to assess the problem and provide the most appropriate solution. The RAR application can be integrated from a master data and transactional data perspective to the backend ERP, MRO, CMMS, LIMS or CRM system over REST or SOAP web services and even ODATA, offering a seamless switching of control from one application to another.

From a user experience perspective the RAR application appears to be an embedded functionality of the application that the users are already used to – only enhanced significantly with the live streaming functionality. From the user perspective launching the RAR application from the primary enterprise application only requires a one touch or a single click. The underlying integration layer takes care of all the heavy lifting of rendering the content and the functionality of RAR.

Every live interaction in RAR can be recorded in the RAR secure server under full control of corporate IT and be accessible in the transaction history with a one touch launch from the main enterprise application. Every recorded multimedia content in RAR can be tagged for future searches and even be included in an extended version of the Transaction Analytics or Big Data analytics.

The ramifications of an integration of RAR to the multi-million dollar investment that corporations have already made over the past couple of decades on ERP, CMMS, MRO, LIMS, CRM systems are many and the benefits becomes clearly evident. The ability to collaborate with multi-disciplinary experts anywhere in the world from any internet connected device equipped with a web browser through live streaming of high definition audio/video content is a significant cost saver. Being able to use junior technicians on location and yet leverage senior experts remotely increases operational efficiency. Quicker problem resolution metrics by using RAR integration significantly boosts customer satisfaction. Corporations can extend their sales and service territories knowing that support can be delivered more efficiently from an operations and financial perspective leading to larger economies to scale and hence leading to higher competitive advantage in the market.

Remote Assisted Reality In the Internet of Things

That tiny ripple in the Internet Of Things (IoT) has grown into the tidal wave of today, well poised to morph into tsunami proportions in the not too distant future. One does not have to be too adventurous to conceive that a day will come when the number connected machines would demand the need for IP addresses higher than IPV6 to accommodate them all.

The growth of human population in this planet has strained the availability of quality facilities and other essential services that sustain our modern day lives. Similarly the exponential growth of connected machines in the IoT has already started to strain the availability of qualified personnel to service and maintain malfunctioning machines and get them back in operation. Faulty machines thereby leave black holes in the IoT network and depending on the application such as SCADA, malfunctioning control equipment can disrupt essential services due to control system shutdowns.

The truth of the matter is that it would become increasingly challenging to have enough qualified expertise who could be brought to bear in short notice to fix real problems occurring in the field. The number of downed devices in the IoT would far outnumber of the duly qualified service personnel who could be dispatched on location to fix the problem. One way to resolve this shortage would be to increase the quality and longevity of the connected IoT devices, in order to reduce failure rates. But then investing heavily on higher quality also has an impact on increased product costs and hence would demand and increase sales prices. However that could trigger pushback from the consumers who are looking for the lowest cost of ownership. As competition drives the prices down and product costs increase, businesses would be faced with eroding gross margins, reducing profitability, which would take increasing quality levels to a point of diminishing returns. Quality managers and COO level executives are always challenged with identifying that point of optimum quality that returns sufficient gross margins for the business.

Given the true reality that machines are not perfect and everlasting, it is evident that machines will require maintenance, they will fail for a plethora of reasons and would need to be monitored, controlled, maintained, repaired and even replaced in some cases. These business processes are points of analysis and decision-making and quite honestly there isn’t enough supply of quality human expertise and experts to be physically present on location to meet the demands of keeping up with the machines.

So what do you do as smart strategic humans responsible for the operational life cycle of the machines deployed in the field? You leverage! You leverage the modest supply of high quality human intellectual capital and expertise in your organization by offering them the concept, technology and business process plugins provided by Remote Assisted Reality (RAR). Instead of having these product and service experts commute all over your service area following a prioritized task list, RAR enables them to provide real time consultation from anywhere in the world to a distributed network of junior service technicians who are on location with the equipment that needs attention.

RAR enables this real time consultation to occur by provisioning live audio/video high definition streams to be routed to the qualified expert who can watch the real time situation through any browser. This real time situational awareness facilitated in high definition by RAR provides the expert with the most immersive experience of the situation. Not only can the consulted expert provide live and close-guided assistance to the remote junior technician on location to resolve the situation, they can also invite other qualified experts into the same live session to watch the same live stream and together collaborate to provide the necessary guidance. Nothing pre-programmed, no preset patterns to recognize – RAR involves raw human intelligence and expertise to solve real world problems in the field.

In some cases the customer themselves can be provided with a pair of specialized RAR smartglasses as part of a premium service plan. In the event of a machine failure or trouble the customer can initiate the live stream while on call with customer support. More often than not, customers will report the presenting problem, not the root cause. Watching the live stream initiated by the customer, duly trained customer support personnel can provide real time Level 1 live support to the customer and help resolve the problem without even having to send a service technician. The customer is up and running right away much to their satisfaction. In the event that Level 1 support cannot resolve the problem, they could use RAR to engage Level 2 support to watch the same live stream and together collaborate to resolve the problem right away. If the problem actually requires a service technician to be dispatched, the live stream provides a more accurate understanding of the right skills that that service technician must have to resolve the situation. Furthermore the live stream can also ensure that when the service technician shows up, they are also carrying the right parts and equipment to resolve the problem on the first visit to get the customer up and running sooner than before.

Every live stream can be recorded in the RAR application and connected to the service ticket for future reference, training, SOP and regulatory compliance and Big Data Analytics. The multi-party collaboration is facilitated securely in RAR where the business retains total control of every bit of data, every frame of video, every frequency of audio being used during the session.

The result is faster turn-around times of downed equipment back in operation leading to higher customer satisfaction, reduced cost of service, extreme leverage of high quality human expertise, significantly increasing their work-life balance all of which contribute to higher gross margins for the business and reputation in their operating niche market. RAR enables all of these benefits for the organization with secure streams and transactional data stored in on-premise servers or in the cloud (private or public) where the business has full and total control of the data at all times making it a gold mine of multimedia data for future use.

As the Internet of Things population density increases with more and more devices connecting to the network, RAR provides a compelling value proposition to provide faster, more efficient and lower cost of ownership and service to businesses who adopt RAR. We are creating these machines with greater and better sophistication, which in turn makes them high maintenance entities. The leverage of RAR to meet the demands to keep the IoT up and running to deliver on its objectives is worth a key consideration for your operational budget both in the short, medium and long term planning.

Remote Assisted Reality Applied In SCADA Systems

As the ‘Rise Of The Machines’ – that too Intelligent, Decision Making, Remote Controlling Machines, advances by leaps and bounds through the Internet Of Things, it is important for us modern day technology architects and programmers to remember and revere the greats of the last century who started it all – the visionaries who introduced the concept and technology of SCADA.

The SCADA (Supervisory Control And Data Acquisition) concept stems from the industrial need and objective to acquire operational data electronically from remotely located possible points of system failure, transmit that data securely to a supervisory computer system in nearest-real time, which would process the data, compare it with stated tolerance limits and make a pre-programmed decision to take corrective action or not.

At Level 0, the SCADA system relies on the control factor (temperature, pressure, flow, current, voltage, load, proximity – you name it) sensors to continuously monitor variations and report measurements to the Local Control.

Level 1, representing the Local control consists of RTU (remote telemetry unit) and/or PLC (programmable logic controller) units, which at the most basic operational level is responsible to accept the inputs from Level 0compare with tolerance limits of the control factor in local embedded firmware and make an “pre-programmed decision” to trigger appropriate control relays to implement a change on the necessary components (valves, voltage regulators, thermostats etc) at Level 0 that would bring the control factor back within tolerance. In some extreme cases, Level 1 can also implement a shutdown of Level 0 components under its control. In addition, Level 1 components also communicate measurements, automated actions taken, up to Supervisory Control over a secure network for additional control room / command center activities in the SCADA application.

Level 2 of the SCADA system offers the HMI (human machine interface) where digital signals coming from the hundreds or thousands of Level 1 components in near-real-time are collated and presented in a form that can be interpreted by human operators in the control room or command center. The information is displayed in a multitude of large monitors or wall panels in summarized and detailed format. In addition to monitoring the incoming data from Level 1 the SCADA application typically would include an Alarm Management or Incident Response system.
Level 2 operations of the SCADA system is also a prime spot for applying technology related to Remote Assisted Reality (RAR). In situations such as false alarms being triggered at Level 0 or relayed from Level 1, or even in case of abnormal measurements being transmitted to Level 2 from the lower levels, Remote Assisted Reality can play a major role to derive real time situational intelligence from remote locations. A Level 2 operator observing the anomaly in the control panels could instruct a local technician to activate their RAR smartglasses initiate a live stream in high definition and invite the Level 2 operator to watch the presenting problem.

In addition to recording the entire trouble shooting operation, which may be used later for training, repair, replacement or even regulatory audits, the Level 2 operator could provide real time assistance to the technician on-location to conduct repairs or corrections. If repairs cannot be done right away, the Level 2 operator could create a service ticket for a higher qualified technician to be dispatched through the CMMS (Computerized Maintenance Management System) with precise detail about the presenting problem. Through RAR technology, any relevant parties such as supervisors, inspectors, procurement advisors, parts and service personnel and even external parties could be invited to watch the situation live and in real time to collaborate and arrive at the most appropriate route to resolution. Every action, every conversation, suggestion is recorded by RAR software and connected to the incident log for future reference. As you can understand being able to resolve situations in the field through such remote live assistance by watching live audio video streams in high definition can significantly reduce service costs, reduce risk, increase equipment availability and protect service levels – all contributing to higher gross margins in operation.

Level 3 of the SCADA system is all about analytics, decision making and providing reports for both internal and external audits and creating supporting collateral for preventive maintenance and regulatory compliance. The multimedia logs collected through the RAR technology from Level 2 can add an entirely new dimension of data to those analytics, extending the static alphanumeric data of the traditional SCADA system into a set of recorded live streams, photos and annotated images contributed by the real-time integration to a RAR application.

Modern day SCADA systems at Level 4 are almost always integrated (or at least they should be) to a backend CMMS (Computerized Maintenance Management System) or MRO application at a minimum, in addition to other systems such as GIS (Geographical Information Systems) and FIS (Financial Information System). When integrated with a Remote Assisted Reality application, both CMMS or MRO applications can also derive significant benefits from the extension of their alphanumeric datasets into the multi-media data set in the RAR application for holistic analytics and intelligent business decision making.

Service orders created in the CMMS or MRO application show up in the field technician tasklist on their mobile device. If the technician needs real time assistance or consultation with a remote expert who is not physically present, the technician would activate their RAR smartglasses and invite the expert through the RAR application to watch the live stream in high definition. The expert could invite other multi-disciplinary experts in the same live session, all watching the same live stream from anywhere in the world to consult and arrive at the most appropriate resolution, while the RAR application records every interaction.

In another scenario for the Level 4+ operations the remote expert could voluntarily guide a junior technician through the steps necessary to resolve the presenting problem at Level 0 or Level 1 simply by watching the live stream originating from the on-site technician’s RAR smartglasses. The recording of such interaction can ensure strict adherence to the applicable SOP or regulatory compliance guidelines – data that may be used in the future for a variety of purposes.

So as you can understand as the SCADA systems from Levels 0 through 2 become more and more sophisticated to do better and better at what they already do best, Remote Assisted Reality based technology can offer significant economies to scale in the entire operational management portfolio.

As the Machines Rise higher and higher with greater levels of artificial intelligence programmed in them, Remote Assisted Reality offers the much needed bridge to leverage real, live human expertise to resolve real world operational challenges that do not follow preset or pre-programmed patterns, occurring in real time. Regardless of how intelligent machines become they will always remain a subordinate to the human intelligence and expertise and that is where Remote Assisted Reality will continue to thrive, evolve and grow to capture and deliver real time situational intelligence for low cost service, issue resolution, security and compliance.


The Economist posted a well-thought article on their website on April 14th 2016 calling Virtual Reality and Augmented Reality as close cousins. The article had an interesting analogy of the immersive experience of a simulated world on Virtual Reality to the well-acclaimed movie “The Matrix”. The same article draws an analogy of the more real world yer pre-programmed experience of Augmented Reality to the widely popular movie “The Terminator”.

The Matrix and The Terminator

Keanu Reeves journeys through a myriad of simulated environments in the Matrix where circumstances keep changing dynamically based on his decisions. Once Reeves is inside the Matrix his real physical world is cut out from his experience – that pretty much sums up a Virtual Reality experience. On the other hand, the robotic eyes of Arnold Schwarzenegger’s Terminator scans the environment, feeds it to an internal computer that recognizes patterns, invokes the appropriate algorithm matching that pattern and delivers instructions on a heads up display on the field of vision, guiding the Terminator on its next action. When faced with situations that are not in the Terminator’s firmware, it’s next move is that of a rudderless ship and is left to its own devices. Quite rightly then, the pre-programmed and pattern recognition based experience of Augmented Reality can be compared to the story of the Terminator. Kudos to The Economist for that article, however it appears that the author is not yet aware of the concept and technology of Remote Assisted Reality.




Where then would Remote Assisted Reality fit in this tough competition of the Reality space for Enterprise Applications?

The Martian – A Blockbuster Rendition of Remote Assisted Reality

Let’s reflect on the 2015 blockbuster – “The Martian” that catapulted Matt Damon to the ranks of the all time greatest actors of our time. Damon’s character is left stranded in the red planet which is at least 34 million miles away from his home planet – assuming the best case scenario of Mars being at its perihelion and Earth at her aphelion. Damon finds himself in the most impossible of conditions alone in an inhabitable environment and uses his own intelligence and knowledge to do the best he can to survive the odds. However he soon figures out that in order to survive beyond his pitiful rations from the first farm in Mars, he needs to get space-lifted back home. He realizes that the only way he could possibly pull that off is to figure out a way to establish some sort of communication with Earth and engage the experts back home and his spaceshipmates on their Earthbound journey, to initiate a rescue mission.

In the most carefully crafted storyline based on real science, physics, precise mathematics astronomy, dynamics of space travel and space communications, Damon’s character succeeds to engage remote experts from Earth. These experts in a control room at least 34 million miles away, use remote communications to guide him through the necessary steps based on real facts, real conditions, real expertise from around the world, considering real parameters at play, in a real life scenario to initiate a rescue mission to successfully bring Damon back home. This makes The Martian the most accurate rendition of the concept and technology of Remote Assisted Reality.

Defining True Reality – As Intelligent Humans

There aren’t any simulations, any pre-programmed set of algorithms, any preset computer generated pattern recognitions or Artificial Intelligence in Remote Assisted Reality. Its all about real life events occurring in real time, with dynamic and constantly changing situations and circumstances that occur in the field – even the unexpected. Only human expertise with a wider range of intelligence can engage, collaborate from anywhere in the world that is connected to the Internet and constructively resolve true and real situations, not only in this planet but even in deep space. When considering these three options for your business it is important for you to contemplate which of the three business models more accurately depict your day to day operations when it comes to serving your customers and generating revenue, before making your choice of technology.

Surely the machines have risen, but they have been, are and always be leagues behind the capabilities of the ever expanding human intelligence.

Virtual Reality, Augmented Reality, Remote Assisted Reality – Foes or Allies In The Enterprise

The seeds for Remote Assisted Reality were sown about three years ago to fill a void in the enterprise applications space and has made rapid strides ever since to gain enterprise grade maturity. For decades, organizations have been clamoring for real time situational awareness in real life business scenarios. Virtual Reality related technologies provided a ray of hope to fill this void. However the very nature of operation of VR under a simulated environment of pre-programmed scenarios and outcomes, defeated the purpose of being able to address the true reality happening in real time in real life business scenarios. Augmented Reality (AR) based technologies stepped up to the plate as the next step in the evolution of technologies to fill this void in the enterprise applications space. AR made giant strides in augmenting true and real life business scenarios with supplementary information overlaid on Augmented Reality glasses. However until today, AR technology is still heavily reliant on a pre-programmed set of computer generated information in the form of graphics and text being proposed as an overlay on the AR glasses. The proposal is based on pattern recognition algorithms overlaying data on the field of vision of the glasses.

Real Life Business Scenarios In Real Time

In the world of real life businesses in several vertical markets events do not typically follow a pre-set pattern or routine. Events and incidents just happen unannounced, unplanned and un-anticipated. These situations may well be outside the realm of any pre-programmed pattern recognition based algorithms. Under such circumstances , VR or AR based technologies fall way short of expectations, leaving that void in Enterprise Applications and business needs, wide open. The concept of Remote Assisted Reality was formed to bridge this gap between real life situations occurring in real life and the engagement of qualified human intellectual capital in the organization being brought to bear in the spirit of collaborative resolution, leading to an expedited well-informed conclusion.

Remote Assisted Reality Steps Up To The Plate

Despite the Rise Of The Machines in the Internet Of Things, the potency of qualified human expertise cannot be undermined. Dynamic analysis based on real time situational intelligence originating from remote operations, intelligent judgment and adjudication often through multi-party collaboration and educated resolution lies at the core of the concept and philosophy of Remote Assisted Reality. RAR Technology enables live HD audio/video content from remote locations to be streamed over broadband wired, wireless or cellular internet and consolidated in specialized Enterprise Applications. Such a synergy of live content with business data opens up an entirely new dimension to the traditional static and after-the-fact nature of enterprise transaction processing. The live HD audio/video streams elevate the true intelligence of the situation. Instead of relying on personal perception of the remote witness, RAR enables experts collaborating in real time decisive adjudication literally from anywhere in the world.

The ability to integrate such RAR Enterprise applications into backend Enterprise Application for bi-directional transactional data in real time over web services, further extends the reach of Enterprise data from the confines of a data center, into remote locations where real life business events are occurring. The Enterprise of the future is nimble, efficient, lean, has low operational overheads and waste, enjoys high cashflow, is profitable and grows rapidly with increased customer satisfaction.

VR, AR and RAR – Rivals or Collaborators

The question for the enterprise decision maker is which one is better? AR, VR or RAR. The answer lies in the business use case of reference. While VR has significant applications in gaming environments, at the enterprise level VR is best suited for training using simulated environments. The trainee can be immersed in a virtual world or pre-programmed scenarios and navigate through several procedural steps to get a good understanding of what they can expect in the field. Most of these are happy path scenarios and with some VR programming the trainee can be allowed to venture outside the norm to try things out – hopefully the programmed algorithms can meet those scenarios.

AR is best suited for business scenarios where a repeatable pattern with little variability exists. In such applications, the pre-programmed AR code would consult a library of pre-captured patterns as images. Behind each of these images a series of if-then-else constructs would be programmed in advance. When the AR glasses pick up one of those patterns the pre-programmed logic would be triggered and the response from those if-then-else constructs would be conveyed to the operator as visual overlays on their AR headset or through audible files that would play through their headset. Examples of such applications would be conveyor belt operations, pick-pack-ship operations, routine maintenance operations that still match the pre-programmed patterns  in the AR framework.

RAR is applicable for business scenarios that are outside the norm. There are no pre-set patterns that could have been captured and programmed in an AR framework. Machine intelligence fails to deliver solutions, since a pre-programmed solution does not exist and only human expertise needs to be leveraged to resolve the problem. The human brain is infinitely more intelligent in analysis and decision making than pre-programmed artificial intelligence. The live streaming functionality of RAR, where the real time situational content can be experienced in real time in high definition audio/video by remote experts located anywhere in the world , simply by using a browser on any Internet-connected device takes RAR into the world of true reality where the norm is broken and only human expertise can save the day.

So the true answer to the choice between VR, AR and RAR entirely depends on the use case for the business. Under no circumstances are they conflicting with each other if used optimally and do not cross each other’s paths with the intent of intrusion. These technologies are complimentary and collectively collaborate to cover a significant gamut of use cases in the modern day enterprise operations.