Using HoloLens with color measurement devices

There is the HoloLens contest organized by Unity and Microsoft. I decide to join it with my idea and quick prototype I have already made. It is about using HoloLens with color measurement devices to streamline user experience. In particular, implemented scenario could be useful for automotive sector and body shops. This post describes the idea.

Context

Painted surfaces could be characterized by many parameters such as color, gloss, fluorescence, etc. They all together define total appearance for the human eye. Ability to measure these parameters is very useful. For example, car manufacturers using it to ensure that parts from different suppliers have exact the same appearance and fit together when assembled. Also, body shops using measurement devices to order matching color mix for repairing. There are a lot of other application areas such as cosmetic, packaging industry and even in ISS space station. Most of the modern color measurement devices are handheld, equipped with WiFi and have rather small display.
spectroguide_on_wheel.jpg

Problem

It is hard to develop rich user interface for small displays. In addition, using touch functionality is complicated if the user should wear gloves which is typically the case by painting. Additional PC is required to further process measurement data and order color mix. In body shops, places to repair (such as scratches, etc.) should be documented while talking to customer. It is not always the same person who marks the places and performs actual repairing so it is important to avoid mistakes. Training materials and user manuals are not instantly available for the users.

Possible Solution

Develop HoloLens application initially targeted for body shops but extendable for other application domains in the future. The following functionality could be offered:
  1. Provide extensive training materials instantly whenever they are required.
  2. Create so-called work-list which is a set of cars (or other objects) which should be handled. The work-list contains photos of the objects to handle (taken with HoloLens), additional information such as manufacturer, license plates, etc., and labeled set of places where measurement should be taken. Labeling is made by sticking virtual measurement device on the place of interest.
  3. The person who should process work-list items can retrieve the labeled places (HoloLens app will place virtual measurement devices on the stored positions) and perform measurements.
  4. HoloLens app will retrieve measurements from device using WiFi connectivity, visualize it and store the data in the cloud for further processing. For example, to send the order for color mix and provide corresponding information to the bookkeeping department.
Such application could overcome small screen constraints by offering reach AR interface, eliminate the need for additional computer, improve user experience in painting facilities and simplify further processing of measurement data. Potentially, if widely accepted, it could eliminate the need for display on measurement devices at all which in turn reduce device costs, weight, size and can considerably increase battery operation time.

Screen-cast from HoloLens prototype implementation

It illustrates the very early prototype of the application described above.
In particular, it shows three main objects - measure, order mix and academy. It is a kind of main menu.
Activating the academy object shows disassembled measurement device, provides instructional video and offers virtual car which could be used for training.

Activating the measure object shows the work-list populated with three items. In addition, the user can mark places for measurement by sticking virtual measurement devices to the gazed spot.
Selecting order mix object retrieves measurement from the real measurement device, displays the results (spectrum and Lab values) on the virtual panels and uploads the data to the cloud (not implemented yet) to place the order and further processing.

Comments