Why it matters
Sample less. Damp more.
The same perceptual result, ⅒ the update budget.
Feeling and Seeing: Issues in Force Display
Proceedings of the 1990 ACM Symposium on Interactive 3D Graphics (SI3D '90), pp. 235–243
Haptics Force display Control theory HCI Virtual environments
Read the full paper (PDF) TL;DR. Thirty-five years ago, the authors built a two-axis joystick that let you
feel virtual sandpaper. Part I introduces a trick — compute forces from the slope of a
height map — that made it work. Part II then asks why it sometimes didn't work, and uses
control theory to derive a simple stability rule: the product of stiffness and sampling
delay must stay below about twice the viscosity. Along the way they explain why 100 Hz
can feel identical to 1000 Hz, why your arm is stable going forward but not sideways, and why
none of this was obvious until the hardware started buzzing.
Why force display?
In 1990, research in virtual environments was almost entirely about what you could see and hear. This paper makes the case that the next honest dimension of presence is what you can feel — not just contact events, but the rich signature of a surface's texture, stiffness, and give.
Force display is especially useful for communicating surface texture and bulk properties of objects and environments as well as dynamics of objects. In this way force display technologies augment the strengths of computer-generated graphics and sound in creating convincingly realistic environments. — p. 235
The authors built Sandpaper: an on-screen palette of textured patches you could stroke with a two-axis force-feedback joystick (a retrofitted arcade controller from Atari). The pitch was simple — computer graphics had spent a decade learning to synthesize convincing visual texture; could the same be done for touch?
Part I · The gradient trick
The central idea of Sandpaper is almost embarrassingly simple. Represent the surface as a
one- or two-dimensional height map h(x). When the user's joystick is at
position x, compute the local slope Δh/Δx and push back with a
horizontal force proportional to it:
Fig. 1 · Interactive gradient technique
Drag on the surface below. The red arrow shows the force the motor produces at your current position — pointing down the slope, stronger where the bump is steeper. Adjust the bump height and spacing to feel (well, see) how "roughness" changes.
Recreated after Figure 2 of the paper. The motor never tells your hand "there's a bump here" — it only pushes left or right with the right strength, and your brain synthesizes the bump.
This is the cleanest insight in the paper. You don't need to physically stop the joystick at the peak of a bump; you only need to oppose uphill motion and release downhill motion. A sufficiently fast servo loop does the rest, and the user's sensorimotor system fills in a remarkably convincing mental model of a surface it never actually touched.
Pilot studies
Subjects were given several virtual patches, masked from view, and asked to order them by roughness. They could — consistently. Varying amplitude and groove-spacing produced orderings that matched the authors' intuitions about what the parameters "ought" to feel like. Suggestive rather than conclusive, but enough to justify the architecture.
Part II · When illusions break
The unpleasant surprise of building a force display is that it doesn't always behave. Push against a simulated hard wall and the joystick may start to buzz — a high-frequency oscillation that destroys the illusion and, occasionally, your wrist. Part II of the paper develops a control-theoretic explanation, and it generalizes a simple rule.
Fig. 2 · The stability threshold
Stable operation requires that the sampling period T stay below roughly
2B/K — where K is the simulated stiffness and B is
the combined viscosity of hand plus system. Set K and B; the band
shows the max safe T. Then pick your sampling rate and see whether you're in
the clear.
The paper's exact values for the "hard surface" condition: K = 2773, B(radial) = 10, B(tangential) = 1.1. Try both — and notice that pushing B from 1.1 to 10 buys you roughly 10× more allowed delay.
The counter-intuitive sampling result
Because T* ∝ B/K, you can trade sampling rate for viscosity. The paper reports
that 100 Hz with added damping felt indistinguishable from 1000 Hz — while
250 Hz without damping was flatly unstable. Faster is not always better; better-balanced
is better.
Fig. 3 · The three experimental conditions
1000 Hz
stable
Baseline: delay is small enough that any reasonable viscosity suffices.
250 Hz
unstable
4× longer delay; the wall buzzes. Users feel — and hear — the oscillation.
100 Hz + viscosity
stable
10× longer delay, damped. Subjects cannot tell it from the 1000 Hz condition.
Radial vs. tangential: why your arm has opinions
The same hardware, the same software, the same wall — and yet every one of the 13 test
subjects was stable when pushing the joystick forward and back, and every one was
unstable when moving side to side. The reason is mechanical: your arm's viscosity is
about 9× higher along its length than across it. The threshold T* = 2B/K
cuts differently in each direction.
Fig. 4 · Arm dynamics along two axes
From the paper's Table (p. 242): B(radial) ≈ 10, B(tangential) ≈ 1.1 N·s/m. At K = 2773, the stable period drops from about 7 ms (radial) to about 0.8 ms (tangential). 13 of 13 subjects match the prediction exactly.
Try the task of letting your hand behave like a piece of paper encountering a striking stick. Although one can see the coming stick by its trajectory, and feel its contact with the skin, it is impossible for one to make one's hand act like a piece of paper. — p. 242
Two puzzles about the human arm
The analysis raises two immediate questions. First, if the neural feedback loop through your brain takes ~200 ms, how does your arm ever stabilize contact at all? The answer, from Hogan, is that a relaxed arm behaves as a passive mechanical object with roughly fixed impedance over windows as long as 1.2 seconds — the nervous system is slow, but the muscles-and-tendons are a fast analog low-pass filter that does the stabilizing for free.
Second, if 1000 Hz is well past the bandwidth of kinesthetic perception, why do users feel a difference between 500 Hz and 1000 Hz updates? Because the skin's cutaneous receptors respond well past 400 Hz — and at lower sampling rates the tiny unstable oscillations, even when subthreshold for the arm, can be heard through the fingertip as a buzz.
What this paper got right
- Gradient-based texture — the canonical technique for force display of surfaces, still used in modern haptic APIs.
- T ≈ 2B/K — a back-of-envelope stability test you can run on any force-feedback system.
- Exploratory motion matters — texture perception is not passive contact but active stroking; interface design has to allow and reward it.
- Multi-sensory reinforcement — the illusion holds even under imperfect force rendering if vision and sound agree.
- From parameters to percepts — the call to move from stiffness and bump-height to roughness, softness, and stickiness as first-class descriptors.
We did not use all the theories in designing our first systems. When problems came one by one, we realized that an analysis would be useful. — p. 242
Read the paper
Citation
Minsky, M., Ouh-young, M., Steele, O., Brooks, F. P., & Behensky, M. (1990). Feeling and seeing: issues in force display. In Proceedings of the 1990 Symposium on Interactive 3D Graphics (SI3D '90) (pp. 235–243). ACM.
Key references
- Hogan, N. (1989). Controlling Impedance at the Man/Machine Interface. IEEE Int. Conf. on Robotics & Automation. — Source of the 1.2-second arm-impedance timescale.
- Lederman, S., & Klatzky, R. (1987). Hand movements: A window into haptic object recognition. Cognitive Psychology, 19(3). — The "lateral motion → texture" exploratory procedure.
- Colgate, J. E., & Hogan, N. (1989). An analysis of contact instability in terms of passive physical equivalents. IEEE Int. Conf. on Robotics & Automation.
- Ouh-young, M., et al. (1988). Using a manipulator for force display in molecular docking. IEEE Int. Conf. on Robotics & Automation. — The sibling application at UNC using the ARM remote manipulator.
- Sutherland, I. E. (1965). The ultimate display. Proc. IFIP Congress. — The founding vision this paper is working toward.