Touché is a novel capacitive touch sensing technology that provides rich touch and gesture sensitivity to a variety of analogue and digital objects. The technology is scalable,i.e., the same sensor is equally effective for a pencil, a doorknob,a mobile phone or a table. Gesture recognition also scales with objects: a Touché enhanced doorknob can capture the configuration of fingers touching it, while a table can track the posture of the entire user (Figures 1.1b, 5.2and 5.3).Sensing with Touché is not limited to inanimate objects –the user’s body can also be made touch and gesture sensitive(Figures 1.1a and 5.4). In general, Touché makes it very easy to add touch and gesture interactivity to unusual, nonsolid objects and materials, such as a body of water. Using Touché we can recognize when users touch the water’s surface or dip their fingers into it (Figures 1.1c and 5.5).Notably, instrumenting objects, humans and liquids with Touché is trivial: a single electrode embedded into an object and attached to our sensor controller is sufficient to computationally enhance an object with rich touch and gesture
interactivity. Furthermore, in the case of conductive objects,e.g., doorknobs or a body of water, the object itself acts as an intrinsic electrode – no additional instrumentation is necessary.Finally, Touché is inexpensive, safe, low power and compact; it can be easily embedded or temporarily attached anywhere touch and gesture sensitivity is desired. Touché proposes a novel form of capacitive touch sensing that we call Swept Frequency Capacitive Sensing (SFCS).In a typical capacitive touch sensor, a conductive object isexcited by an electrical signal at a fixed frequency. The sensing circuit monitors the return signal and determines touch events by identifying changes in this signal caused bythe electrical properties of the human hand touching the object. In SFCS, on the other hand, we monitor the response to capacitive human touch over a range of frequencies.Objects excited by an electrical signal respond differently at
Figure 1.1: Touché applications: (a) on-body gesture sensing;
(b) a smart doorknob with a “gesture password”; (c) interacting
with water; (d) hand postures in touch screen interaction.
different frequencies, therefore, the changes in the return signal will also be frequency dependent. Thus,instead of measuring a single data point for each touch event, it measures a multitude of data points at different frequencies. It then use machine learning and classification techniques to demonstrate that it can reliably extract rich interaction context, such as hand or body postures, from this data. Not only can it determine that a touch event occurred,it can also determine how it occurred. Importantly,this contextual touch information is captured through a single electrode, which could be simply the object itself.Although electromagnetic signal frequency sweeps have been used for decades in wireless communication and industrial proximity sensors, Scientists are not aware of any previous attempt to explore this technique for touch interaction.Their contributions, therefore, are multifold:
1) They propose and develop a novel capacitive touch sensing technology, called Swept Frequency Capacitive Sensing. Itallows for minimally instrumented objects to capture a wealth of information about the context of touch interaction.It also permits novel mediums for capacitive touch and gesture sensing, such as water and the human body.
2) They report a number of innovative applications that demonstrate the utility of their technology including a) smart touch interaction on everyday objects, b) tracking human body postures with a table, c) enhancing touchscreen interaction,d) making the human body touch sensitive, and e)recognizing hand gestures in liquids.
3) They conduct controlled experimental evaluations for each of the above applications. Results demonstrate recognition rates approaching 100%. This suggests Touché is immediately feasible in a variety of real-world applications.
2. LITERATURE REVIEW
The importance of touch and gestures has been long appreciated in the research and practice of human-computer interaction(HCI). There is a tremendous body of previous work related to touch, including the development of touch sensors and tactile displays, hand gesture tracking and recognition, designing interaction techniques and applications for touch, building multi touch, tangible and flexible devices.The foundation for all touch interaction is touch sensing,i.e., technologies that capture human touch and gestures.This includes sensing touch using cameras or arrays of optical elements, laser rangefinders, resistance and pressure sensors and acoustics– to name a few. The most relevant technology is capacitive touch sensing,a family of sensing techniques based on same physical phenomenon – capacitive coupling.The basic principles of operation in most common capacitive sensing techniques are quite similar: A periodic electrical signal is injected into an electrode forming an oscillating electrical field. As the user’s hand approaches the electrode,a weak capacitive link is formed between the electrode and conductive physiological fluids inside the human hand, altering the signal supplied by the electrode. This happens because the user body introduces an additional path for flow of charges, acting as a charge “sink”. By measuring the degree of this signal change, touch events can be detected.There is a wide variety of capacitive touch sensing techniques.One important design variable is the choice of signal property that is used to detect touch events, e.g., changes in signal phase or signal amplitude can be used for touch detection. The signal excitation technique is another important design variable. For example, the earliest capacitive proximity sensors in the 1970s were oscillating at resonant frequency and measured signal dumping as additional capacitance that would affect the resonant frequency of the sensing circuit. The choice of topology of electrode layouts, the materials used for electrodes and substrates and the specifics of signal measurement resulted in amultitude of capacitive techniques, including charge transfer,surface and projective capacitive, among others.Capacitive sensing is a malleable and inexpensive technology– all it requires is a simple conductive element that is easy to manufacture and integrate into devices or environments.Consequently, today we find capacitive touch in millions of consumer device controls and touch screens. It has,however, a number of limitations. One important limitation is that capacitive sensing is not particularly expressive – it can only detect when a finger is touching the device and sometimes infer finger proximity. To increase the expressiveness,matrices of electrodes are scanned to create a 2Dcapacitive image. Such space multiplexingallows the device to capture spatial gestures, hand profiles or even rough 3D shapes. However, this comes at the cost of increased engineering complexity, limiting its applications and precluding ad hoc instrumentation of our living and working spaces. Current capacitive sensors are also limited in materials they can be used with. Typically they cannot be used on the human body or liquids.In this, they advocate a different approach to enhancing the expressivity of capacitive sensing – by using frequency multiplexing. Instead of using a single, pre-determined frequency,we sense touch by sweeping through a range of frequencies. We refer to the resulting curve as a capacitive profile and demonstrate its ability to expand the vocabulary of interactive touch without increasing the number of electrodes or the complexity of the sensor itself.Importantly, our technology is not limited to a single electrode.Sensor matrices can be easily constructed and would bring many of the unique sensing dimensions described in this paper. However, in the current work, we focus on asingle electrode solution, as that is the simplest – and yetallows for surprisingly rich interactions.
3. SWEPT FREQUENCY CAPACITIVE SENSING
Figure 3.1: Touché architecture
The human body is conductive, e.g., the average internal resistance of a human trunk is ~100 Ω. Skin, on the other hand, is highly resistive, ~1M Ω for dry undamaged skin. This would block any weak constant electrical(DC) signal applied to the body. Alternating current (AC)signal, however, passes through the skin, which forms a capacitive interface between the electrode and ionic physiologic fluids inside the body. The body forms a charge“sink” with the signal flowing though tissues and bones to ground, which is also connected to the body through a capacitive link.The resistive and capacitive properties of the human body oppose the applied AC signal. This opposition, or electrical impedance, changes the phase and amplitude of the AC signal. Thus, by measuring changes in the applied AC signal we can 1) detect the presence of a human body and also 2)learn about the internal composition of the body itself. This phenomenon, in its many variations, has been used since the 1960s in medical practice to measure the fluid composition of the human body, in electro-impedance tomography imaging and even to detect the ripeness of nectarine fruits. More recently, it has been used in a broad variety of capacitive touch buttons, sliders and touchscreens inhuman-computer interaction.The amount of signal change depends on a variety of factors.It is affected by how a person touches the electrode,e.g., the surface area of skin touching the electrode. It is affected by the body’s connection to ground, e.g., wearing or not wearing shoes or having one or both feet on the ground. Finally, it strongly depends on signal frequency.This is because at different frequencies, the AC signal will flow through different paths inside of the body. Indeed,just as DC signal flows through the path of least resistance,the AC signal will always flow through the path of least impedance. The human body is anatomically complex and different tissues, e.g., muscle, fat and bones, have different resistive and capacitive properties. As the frequency of the AC signal changes, some of the tissues become more opposed to the flow of charges, while others less, thus changing the path of the signal flow.Therefore, by sweeping through a range of frequencies in capacitive sensing applications, we obtain a wealth of information about 1) how the user is touching the object, 2)how the user is connected to the ground and 3) the current configuration of the human body and individual body properties.The challenge here is to reliably capture the data and then find across-user commonalities – static and temporal patterns that allow an interactive system to infer user interaction with the object, the environment, as well as the context of interaction itself.SFCS presents an exciting opportunity to significantly expandthe richness of capacitive sensing. Scientists are not aware of previous attempts to design SFCS touch and gesture interfaces,investigate their interactive properties, identify possible application domains, or rigorously evaluate their feasibility for supporting interactive applications. All relevant capacitive touch sensing techniques use a single frequency.
4. TOUCHÉ IMPLEMENTATION
The overall architecture of Touché is presented on Figure3.1. The user interacts with an object that is attached to a Touché sensor board via a single wire. If the object itself is conductive, the wire can be attached directly to it. Otherwise,a single electrode has to be embedded into the object and the wire attached to this electrode.
Figure 4.1: Touché sensing board: 36x36x5.5 mm, 13.8 grams.
Touché implements SFCS on a compact custom-built boardpowered by an ARM Cortex-M3 microprocessor (Figure4.1).The on-board signal generator excites an electrode withsinusoid sweeps and measures returned signal at each frequency.The resulting sampled signal is a capacitive profileof the touch interaction. Scientists stress that in the current versionof Touché They do not measure phase changes of the signal inresponse to user interaction. They leave this for future work.Finally, the capacitive profile is sent to a conventional computerover Bluetooth for classification. Recognized gesturescan then be used to trigger different interactive functions.While it is possible to implement classification directly onthe sensor board, a conventional computer provided moreflexibility in fine-tuning and allowed for rapid prototyping.
4.1 Sensor board design
An ARM microprocessor, NXP LPC1759 running at 120MHz, controls an AD 5932 programmable wave generator to syn the size variable frequency sinusoidal signal sweeps from 1 KHz to 3.5 MHz in 17.5 KHz steps (i.e., 200 steps in each sweep, see Figure 3.1). The signal is filtered to remove environmental noise and undesirable high frequency components and is also amplified to 6.6 Vpp (Figure 4.2), which is then used to excite the attached conductive object. In the current design we tune Touché to sense very small variations of capacitance at lower excitation frequencies by adding a large bias inductor Lb(~100 mH), a technique used in impedance measurement. By replacing it with a bias capacitor,we can make Touché sensitive to very small inductive variations, e.g., copper coil stretching.The return signal from the object is measured by adding a small sensing resistor, which converts alternating current into an alternating voltage signal (Figure 4.2). This signal is then fed into a buffer to isolate sensing and excitation sections;an envelope detector then converts the AC signal into a time-varying DC signal (Figure 4.2). The microprocessor samples the signal at a maximum of 200 KHz using a 12-bit analog-digital converter (ADC). A single sweep takes~33 ms, translating to a 33 Hz update rate.Currently, the sampling rate of ADC is a main limiting factor for speed: a dedicated ADC with a higher sampling rate would significantly increase the speed of Touché. Sampling is much slower at low frequencies, as it takes longer for the analogue circuitry to respond to a slowly varying signal. In applications where an object does not respond to low frequencies,they swept only in the high frequency range, tripling the sensor update rate to ~100 Hz.
4.2 Touché Sensing Configurations
There are two basic sensor configurations. First, the user issimply touching on object or an electrode. This is the classic capacitive sensor configuration that assumes that both the sensor and the user are sharing common ground, even through different impedances. For example,if the sensor were powered from an electrical outlet, it would be connected to the ground line of a building. The user would be naturally coupled to the same ground via a capacitive link to the floor or building structure. Although this link may be weak, it is sufficient for Touché.In the second case, the sensor is touching two different locations of the user body with its ground and signal electrodes. In this configuration Touché measures the impedance between two body locations.
4.3 Communication and Recognition
For classification, we use a Support Vector Machine (SVM)implementation provided by the Weka Toolkit (SMO,C=2.0, polynomial kernel, e=1.0) that runs on the aforementioned conventional computer. Each transmission from the sensor contains a 200-point capacitive profile, from which we extract a series of features for classification.The raw impedance values from the frequency sweep have anatural high-order quality. As can be seen in Figure 5.1-5.2,the impendence profiles are highly continuous, distinctive and temporally stable. Therefore, it use all 200 values as features without any additional processing. Additionally, it compute the derivative of the impedance profile at three different levels of aliasing, by down-sampling profiles into arrays of 10, 20, 40 and using [-1, 1] kernel, yielding another 70 features. This helps to capture shape features of the profile, independent of amplitude, e.g., it is easy to see the peaks minima in Figures 5.1 to 5.5– more difficult is to see the visually subtle, but highly discriminative peak slopes.Moreover, using the derivative increases robustness to global variations in impendence, e.g., an offset of signal amplitude across all frequencies due to temperature variations. As a final feature, we include the capacitive profile minima,which was found to be highly characteristic in pilot studies(see Figures 5.1-5.5). Once the SVM has been trained, classification can proceed in a real-time fashion.
5. EXAMPLE TOUCHÉ APPLICATIONS
The application space of Touché is broad, therefore at least some categorization is pertinent to guide the development of the interfaces based on this technology. It identified five application areas where we felt that Touché could have the largest impact – either as a useful enhancement to an established application or a novel application, uniquely enabled by our approach:
• makingeveryday objects touch gesture sensitive
• sensing human bimanual hand gestures
• sensing human body configuration (e.g., pose)
• enhancing traditional touch interfaces
• sensing interaction with unusual materials (e.g., liquids)
In the rest of this section they propose a single exemplary application for each category, highlighting the utility and scope of our sensing approach.
Figure 5.1: Capacitive profiles for making objects touch and grasp sensitive (doorknob example).
If analogue or digital objects can be made aware of how they are being touched, held or manipulated, they could configure themselves in meaningful and productive ways. The canonical example is a mobile phone which, when held like a phone, operates as a phone.However, when held like a camera, the mode could switch to picture-taking automatically.Touché offers a lightweight, non-invasive sensing approach that makes it very easy to add touch and gesture sensitivity to everyday objects. Doorknobs provide an illustrative example:they lie in our usual paths and already require touch to operate. Yet, in general, doorknobs have not been infused with computational abilities. A smart doorknob that can sense how a user is touching it could have many useful features.For example, closing a door with a tight grasp could lock it, while closing it with a pinch might set a user’s away message, e.g., “back in five minutes”. A sequence of grasps could constitute a “grasp password” that would allow an authorized user to unlock the door (Figure 1.1b).Objects such as doorknobs can be easily instrumented with Touché. More importantly, existing conductive structures can be used as sensing electrodes, for example,the brass surface of a doorknob. Our Touché sensor could be connected to these elements with a single wire, requiring no further instrumentation (Figure5.1). Contrast this to previous techniques that generally require a matrix of sensors.
5.2 Body Configuration Sensing
Touché can be used to sense the configuration of the entire human body. For example, a door could sense if a person is simply standing next to it, if they have raised their arm to knock on it, are pushing the door, or are leaning against it.Alternatively, a chair or a table could sense the posture of a seated person – reclined or leaning forward, arms on the armrests or not, one or two arms operating on the surface, as well as their configuration (Figure 5.3). More importantly,this can occur without instrumenting the user. Similar to everyday objects, conductive tables can be used as is, just by connecting a Touché sensor. Non-conductive tables would require a single flat electrode added to their surface or could simply be painted with conductive paint.Sensing the pose of the human body without instrumenting the user has numerous compelling applications. Posture sensing technologies are an active area of research, with applications in gaming, adaptive environments, smart offices,in-vehicle interaction, rehabilitation and many others. Itview Touché as one such enabling technology, with many exciting applications. To this end, an evaluation report of body posture sensing with a Touché-enhanced table in the following Touché Evaluation section.
5.3 Enhancing Touchscreen Interaction
Figure 5.3: Capacitive profiles for enhancing touchscreen interaction with a hand posture sensing.
Touché brings new and rich interaction dimensions to conventional touch surfaces by enhancing touch with sensed hand posture (Figure 1.1d) For example, Touché could sense the configuration of fingers holding a device, e.g., if they are closed into a fist or held open, whether a single finger is touching, all five fingers, or the entire palm (Figure 5.3). The part of the hand touching could be also possibly be inferred,e.g., fingertips or knuckles, a valuable extra dimension of natural touch input.These rich input dimensions are generally invisible to traditional capacitive sensing. Diffuse infrared (IR) illumination can capture touch dimensions such as finger orientation and hand shape. However, sensing above the surface is challenging as image quality and sensing distance is severely degraded by traditional diffuse projection surfaces(offers an expensive alternative). An external tracking infrastructure can also be used. This, however, prohibits the use of mobile devices and introduces additional cost and complexity. Touché provides a lightweight, yet powerful, solution to bring hand posture sensing into touchscreen interaction.There are many possible implementations – one is presented in Figure 5.3. At the very minimum, this would enable a touch gesture similar to a mouse “right click”. Right click is a standard and useful feature in desktop GUI interfaces.However, it has proved to be elusive in contemporary touch interfaces, where it is typically implemented as a tap and-hold. Additionally, combining hand pose and touch could lead to many more sophisticated interactions, such as gesture-based 3D manipulation and navigation (Figure 1.1d),advanced 3D sculpting and drawing, music composition and performance, among others.In general, Touché could prove particularly useful for mobile touchscreen interaction, where input is constrained due to small device size. In this context, a few extra bits of input bandwidth would be a welcomed improvement. Detailed controlled experiments evaluating gesture sensing on a simulated mobile device are reported subsequently.
5.4 On-Body Gesture Sensing
Unlike inanimate physical objects, the human body is highly variable and uncontrolled, making it a particularly challenging“input device”. Compounding this problem is that users are highly sensitive to instrumentation and augmentation of their bodies. For a sensing technique to be successful, it has to be minimally invasive. Research has attempted to overcome these challenges by exploring remote sensing approaches,including bio-acoustics and computer vision, each of which has its own distinct set of advantages and drawbacks. Touché is able to sidestep much of this complexity by taking advantage of the conductive properties of the body and appropriate the skin as a touch sensitive surface while being minimally invasive.Because humans are inherently mobile, it is advantageous to define an on-body signal source and charge sink for Touché. As our hands serve as our primarily means of manipulating the world, they are the most logical location to augment with Touché. In this case, the source or sink is placed near the hands, for example, worn like a wristwatch. The other electrode can be placed in many possible locations, including the opposite wrist (Figure 5.4), the waist, collar area, or lower back. As a user touches different parts of their body the impedance between the electrodes varies as the signal flows through slightly different paths on and in the user’s body. The resulting capacitive profile is different for each gesture, which allows us to detect a range of hand-to-hand gestures and touch locations (Figure 5.4).It is worth noting the remarkable kinesthetic awareness of a human being, which has important implications in the design of on-body interfaces. As the colloquialism“like the back of your hand” suggests, we are intimately familiar with our bodies. This can be readily demonstrated by closing one’s eyes and touching our noses or clapping our hands together. In addition to our powerful kinesthetic senses, we have finely-tuned on-body touch sensations and hand-eye coordination, all of which can be leveraged for digital tasks.A wide array of applications can be built on top of the body.One example is controlling a mobile phone using a set of on-body gestures(Figure 1.1a).
Figure 5.4: Capacitive profiles for on-body Sensing with wrists-mounted Touché sensors.
For example, making a “shh”gesture with index finger touching to the lips, could put thephone into silent mode. Putting the hands together, forminga book-like gesture, could replay voicemails.
5.5 Sensing Gestures in Liquids
The real world does not consist only of hard and flat surfacesthat can be easily enhanced with touch sensitivity. Liquid,viscous, soft and stretchable materials are importantelements of everyday life. Enhancing these materials withtouch sensitivity, however, is challenging. Although there is
a growing body of research sensing touch for textile, paperand silicon materials, enhancing a body of liquid with rich touch sensing has mostly remained out of reach,and is a good example of Touché’s application range.By interacting with water, we do not mean using touchscreens under water, but touching the water itself. In particular,approach can distinguish between a user touching the water’s surface and dipping their finger into it (Figure5.5), which is difficult to accomplish with current capacitive touch sensing techniques. Resistive e.g. touch pad sold work under water, but require users to physically press on the surface, which is not truly interaction with the liquid, but rather with a submerged touchpad. Mechanical or optical techniques introduce large external sensing apparatus, prohibiting ad-hoc and mobile interactions.Furthermore, optical sensing generally requires controlled lighting and clear liquids. Water-activated electrical switches can be used to detect the presence of water, but not the user playing with water. These are just a few of the challenges of user-liquid interaction.
Figure 5.5: Capacitive profiles for interacting with water.
Touché can easily add touch sensitivity to various amounts of liquid held in any container. Simply by placing the electrode on the bottom of the water vessel, we can detect a user touching the surface, dipping their fingers in the water, and so on (Figures 5.5). The container can be made of any material, and the electrode can be affixed to the outside – although putting it inside increases sensitivity.Applications of water sensing are mostly experiential, such as games, art and theme park installations and interactive aquariums. We can also track indirect interactions, i.e.,when users are touching water via a conductive object. In this way children’s water toys and eating utensils could be easily enhanced with sounds and lights (Figure 11.c).
The five studies followed the same basic structure describedbelow. Each study was run independently; the entire experimenttook approximately 60 minutes to complete.TrainingParticipants were shown pictographically a small set of gesturesand asked to perform each sequentially. While performinggestures, the participants were told to adjust theirgestures slightly, e.g., tighten their grip. This helped to captureadditional variety that would be acquired naturally withextended use, but impossible in a 60-minute experiment.While the participants performed each gesture, the experimenterrecorded 10 gesture instances by hitting the spacebarand then advanced the participant to the next gesture untilall gestures were performed. This procedure was repeatedthree times providing 30 instances per gesture per user. Inaddition to providing three periods of training data useful inpost-hoc analysis, this procedure allowed us to capture variability
in participant gesture performance, obtaining moregesture variety and improving classification.TestingFollowing the training phase, collected data were used toinitialize the system for a real-time classification evaluation.Participants were requested to perform one of the gesturesfrom the training set randomly selected and presented on adisplay monitor. The system – invisible to both the experimenterand participants – made a classification when participantsperformed each gesture. A true positive result wasobtained when the requested gesture matched the classifier’sguess. The experimenter used the spacebar to advance to thenext trial, with five trials for each gesture.
7. ACCURACY MEASURES
Figure 7.1. Real-time, per-user classification accuracy
for five example applications.
Procedure follows a per-user classifier paradigm where each participant had a custom classifier trained using only his or her training data. This produces robust classification since it captures the peculiarities of the user. Per-user classifiers are often ideal for personal objects used by a single user, as would be the case with, e.g., a mobile phone, desktop computer, or car steering wheel.To assess performance dimensions that were not available in a real-time accuracy assessment, they ran two additional experiments post-hoc. Their first post-hoc analysis simulated the live classification experiment with one fewer gestures per set. The removed gesture was the one found to have the lowest accuracy in the full gesture set. For example, in the case of the grasp-sensing doorknob study, the circle gesture was removed, leaving no touch, one finger, pinch and grasp(Figure 5.1) Accuracy typically improves as the gesture set contracts. In general, they sought to identify gesture sets that exceeded the 95% accuracy threshold.their second post-hoc analysis estimated performance with“walk up” users – that is, classification without any training data from that user, a general classifier. To assess this, they trained their classifier using data from eleven participants,and tested using data from a twelfth participant (all combinations,i.e., 12-fold cross validation). This was the most challenging evaluation because of the natural variability of how people perform gestures, anatomical differences, as well as variability in clothes and shoes worn by the participants.However, this accuracy measure provides the best insight into potential real-world performance when per-user training is not feasible, e.g., a museum exhibit or the me park attraction. Moreover, it serves as an ideal contrast to their per-user classifier experimental results.
Figure 7.2: “Walk Up” classification accuracy for
five example applications.
8. EVALUATION RESULTS
Figures 5.1 though 5.2 illustrate the physical setup and accompanying touch gesture sets for each of the five application domains they tested. Real-time accuracy results for all five studies are summarized in Figure 7.1 “Walk -up” accuracies with different-sized gesture sets are shown in Figure 7.2.
8.1: Making Objects Touch and Grasp Sensitive
A doorknob was an obvious and interesting choice for our touch and grasp sensing study setup (Figure 5.1). They used a brass fixture that came with a high-gloss coating, providing a beneficial layer of insulation. A single wire was soldered to the interior metallic part of the knob, and connected to their sensor. As doors are fixed infrastructure, they grounded their sensor in this configuration. This is a minimally invasive configuration that allows for existing doors to be easily retrofitted with additional touch sensitivity.A set of five gestures was evaluated as seen in Figure 5.1: no touch, one finger, pinch, circle, and grasp. This setup performed well in the real-time per–user classifier experiment,at 96.7% accuracy. Dropping the circle gesture increased accuracy to 98.6%.Walk-up accuracy was significantly worse for five gestures– 76.8% , where the circle gesture was responsible for 95.0% of the errors. Once the circle gesture was removed,walk-up accuracy improves to 95.8%.
8.2: Body Configuration Sensing
To evaluate performance of Touché in body posture recognition scenarios, they constructed a sensing table. This consisted of a conventional table with a thin copper plate on topof it, covered with a 1.6 mm glass fiber and resin composite board (Figure 5.2). A single wire connected copperplate to the Touché sensor board. The static nature of a table meant that they could ground the sensor to the environment in this configuration.A set of seven gestures was evaluated: not present, present,one hand, two hands, one elbow, two elbows, arms (Figure5.2). Average real-time classification performance with seven gestures was 92.6%. Eliminating the two elbows gesture boosted accuracy to 96.0%.Walk-up accuracy at seven gestures stands at 81.2%. As seen in Figure 7.2, accuracy surpasses 90% with five gestures(not present, present, one hand, two hands, two elbow;91.6%. With only three gestures (presence, two hands, two elbow), accuracy is 100% for every participant.
8.3: Enhancing Touchscreen Interaction
The application possibilities of Touché to touchscreen interaction are significant and diverse. For both experimental and prototyping purposes they chose mobile device form factor(Figure 5.3). Mobility implies the inability to ground the sensor, making this setup particularly difficult.As a proof of concept, they created a pinch-centric gesture set which could be used for, e.g., a “right click”, zoom in/out,copy/paste, or similar function. Their mobile device mock up has two electrodes: the front touch surface, simulating a touch panel, and the backside of the device. A Touché sensor is configured to measure the impedance between these two surfaces through the participant’s hand connecting them.Figure 5.4 depicts a set of five gestures that were evaluated:no touch, thumb, one finger pinch, two finger pinch and all finger pinch. Per-user classifier accuracy with all gestures is 93.3%. Removing the two finger pinch brings accuracy up to 97.7%. Walk-up accuracy at five gestures is 76.1%, too low for practical use.However, by reducing the gesture set to no touch, thumb and one finger pinch, accuracy is 100% for all participants,showing the immediate feasibility for mobile applications.
8.4: On-Body Gesture Sensing
Unlike the previous three studies, human-gesture sensing has a predefined device – the human body. This two design variables: sensor placement and gestures.For this study, they chose to place an electrode on each wrist,worn like a watch. The Touché sensor measured impedance between wrist electrodes through the body of participants.Due to the highly variable and uncontrolled nature of the human body, this experimental condition was the most challenging of the five studies.Gesture set consisted of five gestures: no touch, one finger, five fingers, grasp, and cover ears (Figure 5.4). Real time,per-user classification accuracy was 84.0% with five gestures. Removing a single gesture – one finger – improved accuracy to a use able 94.0%.In contrast, walk- up accuracy with a general classifier does significantly worse, with all five gestures yielding 52.9%accuracy. Reducing the gesture set to three (no touch, five fingers, grasp) only draws accuracy up to 87.1% – stronger, but still too low for robust use.This divergence in accuracy performance between per-user and general classifiers is important. The results suggest that for on-body gestures where the user is both the “device” and input, per-user training is most appropriate. This should not be particularly surprising – unlike doorknobs, the individual differences between participants are very significant, not only in gesture performance, but also in their bodies’ composition.A per-user classifier captures and accounts for these per-user differences, making it robust.
8.5: Touching Liquids
They attached a single electrode under a 250 mm-wide and 500 mm-long fish tank, and filled it to a depth of 35 mm of water. The electrode was separated from the liquid by a pane of 3 mm-thick glass and attached to the Touché sensor board via a single wire.Our test liquid gesture set consisted of no touch, one fingertip, three finger tips, one finger bottom, and hand submerged(Figure 5.5). This experimental condition performed the best of the five. Real-time, per-user classification accuracy with the full gesture set was 99.8%.“Walk-up” classification performance was equally strong with all five gestures: 99.3%. Removing three finger tips improves accuracy up to 99.9% (Figure 7.2).
8.6 Anatomical Factors
Touché is sensitive to variations in users’ anatomy. To testif anatomical variations have a systematic effect on classification accuracy, they ran several post hoc tests. They found no correlation between accuracy and height (1.6 ~ 1.9m),weight (52 ~ 111kg), BMI (19.6 ~ 32.3), or gender. This suggests the sensing is robust across a range of users.
• Enhances the touch that recognizes the touch of all human body parts
• Provides stronger security
• Touché is inexpensive.
• Low power.
• It can be easily embedded or temporarily attachedanywhere touch and gesture sensitivity is desired.
• One of the reasons why SFCS techniques have not been investigated before could be due to computational expense
• Instead of sampling a single data point at a single frequency,SFCS requires a frequency sweep
• Analysis of hundreds of data points.
• SFCS requires high-frequency signals, e.g., ~3 Mhz. Designing conditioning circuitry for high-frequency signals is a complex problem.
More importantly, Touché allowed us to capture details of shapes of apacitive profiles, which was important in some of the Touché applications, e.g., in hand-to hand gestures. Therefore, decreasing the sweep resolution would improve performance, but also reduces the robustness of gesture recognition in some of applications. Touché – without any modification– enables a rich swath of interactions from humans, to doorknobs, to water. This would be impossible if we limited the range of frequencies. However, in practical applications the sensing can be limited to a range of frequencies that are most appropriate for a particular product, reducing cost and improving robustness.