2018CHI Best Paper Award Agile 3D Sketching with Air Scaffolding

Kim Y, An S G, Lee J H, et al. Agile 3D sketching with air scaffolding[C]//Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 2018: 238.

 

ABSTRACT

 

Hand motion and pen drawing can be intuitive and expressive inputs for professional digital 3D authoring. However, their inherent limitations have hampered wider adoption. 3D sketching using hand motion is rapid but rough, and 3D sketching using pen drawing is delicate but tedious. Our new 3D sketching workflow combines these two in a complementary manner. The user makes quick hand motions in the air to generate approximate 3D shapes, and uses them as scaffolds on which to add details via pen-based 3D sketching on a tablet device. Our air scaffolding technique and corresponding algorithm extract only the intended shapes from unconstrained hand motions. Then, the user sketches 3D ideas by defining sketching planes on these scaffolds while appending new scaffolds, as needed. A user study shows that our progressive and iterative workflow enables more agile 3D sketching compared to ones using either hand motion or pen drawing alone. 

INTRODUCTION
 

Hand motion的特点
 

Hand motion is an intuitive and useful mode of expression [30]. People adroitly make hand motions freely in the air to communicate the scale and features of 3D shapes [10]. Such expressiveness can be leveraged to help designers in the early stages of rapid and rough idea exploration, for which conventional 3D CAD modeling systems are unsuitable [8, 22, 28]. While 3D sketching using hand motion can help form the approximate shape and scale of a product, it may not meet the level of fidelity required in conceptualization [22]. As such, designers cannot solely rely on their hand motions to develop a fully-fledged design concept 

Pen操作的特点

The pen is another intuitive and expressive tool [20]. Designers trained in traditional drawing techniques can effectively describe 3D shapes with a relatively small number of curves. Thus, there have been attempts to transfer these skills to a digital medium for authoring 3D shapes [6, 20]. However, these techniques introduce varying degrees of additional complexity and indirection in extracting the intended 3D information from the user’s 2D input. Maintaining a sense of proportionality and scale is another inherent difficulty while drawing 3D shapes on a 2D medium, whether analog or digital [24]. 

 

融合两者的特点

We introduce a new 3D sketching workflow where the user employs both hand motion and pen drawing (Figure 1). The user creates air scaffolds—intermediate shapes containing meaningful spatial information—using unconstrained hand motion in the air and defines sketch planes on them. Then, finer details are added using pen-based 3D sketching techniques. Rather than creating strokes or surfaces directly with raw hand motions, which are too imprecise for product design, we found a sweet spot in which rough but quick hand motions create intermediate 3D references for more precise 3D sketching. 

论文贡献

 

Propose a 3D sketching workflow combining the strengths of hand and pen input.
• Devise an algorithm to identify descriptive hand motions from transitory and extract air scaffolds from the identified
motions.
• Integrate the air scaffolding technique with pen-based 3D sketching into a practical system.
• Evaluate our system with users to verify that air scaffolding facilitates idea exploration through agile 3D sketching.
 

 

RELATED WORK
 

手势和触觉反馈的优缺点

In addition, elaborate 3D authoring requires fine motor control that may benefit from haptic feedback [17]. However, many current haptic technologies still severely restrict degrees of freedom and range of movement, undermining the expressiveness of hand motions in space. Without haptic feedback, users may be unable to create their desired 3D shapes solely based on hand movements in space. Thus, some techniques provide the means to modify motiongenerated shapes post hoc [8, 22]. However, these can add complexity and require more effort from the user. 

没有触觉反馈的问题

 

While hand motion may be unsuitable for creating highfidelity 3D shapes without the assistance of appropriate feedback technologies, it may hold important cues regarding the shapes the user had in mind. In the study by Holz and Wilson [10], when people were asked to describe a given 3D shape using bare hand gestures, they made gestures describing the overarching geometry, scale, and proportion of the shape, through various postures and motions of the hands, e.g., with spread palms swept tangentially to depict flat surfaces 

传统手在2D平面绘制的问题

Designers have traditionally used perspective drawings with pen and paper to depict 3D shapes on 2D surfaces. There are ongoing efforts to transfer perspective drawing techniques to create 3D shapes composed of 3D curves through pen input on 2D digital screens.

采用pen绘制的优点

Many pen-based 3D sketching systems assist the user with discovering and configuring appropriate sketch surfaces through preset sketch planes [7], manual control of the position and orientation of sketch planes using buttons [13] or dedicated 3D input devices [21], pen strokes on existing surfaces to invoke perpendicular sketch planes [31], offsetting and erecting sketch surfaces along a given 3D mesh [5, 23], and bounding boxes often used in architectural drawings [25]. These show that the ease with which the user can define and change the position and orientation of sketch surfaces affects the agility and expressiveness of 3D sketching. 

手势和Pen-based的三维设计的总结

We see that pen-based 3D authoring techniques using sketch surfaces offer simplicity and familiarity, and that the ability to quickly define desired sketch planes is crucial. Various techniques show that defining sketch planes in reference to existing 3D entities can be quick and accurate, but a chickenor-the-egg problem arises at the initial stage, when nothing exists to use as the reference. In fact, help is needed whenever the user wishes to make a significant departure from what already exists. 

两种交互界面的融合

Our approach differs in that we use shapes generated using hand motion only as intermediate scaffolds to assist with more precise pen-based 3D sketching, rather than as the final outcome, in consideration of the limited fidelity of the shapes that hand motion can produce. 

 

 

Design

Conclusion

 

Various computer graphics and 3D interaction techniques strive for more intuitive and expressive 3D modeling in design processes. In this study, we focused on the strengths and weaknesses of two salient input modalities and sought to develop a workflow in which the strengths of each are enforced and the weaknesses are complemented. Our contribution is in recognizing that 3D sketching has no technique analogous to 2D scaffolding in 2D sketching. 3D authoring utilizing hand motion was used to roughly and quickly generate the desired scale and volume, and penbased 3D sketching was used to delicately specify finer features. From the extensive user study, we found that it is possible to design an interaction technique that coherently integrates different input modalities to enable rapid and highfidelity 3D conceptualization in a progressive and iterative workflow that the designers could satisfactorily use in practice