emm2004.pdf

(1648 KB) Pobierz
Personalised Real-time Idle Motion Synthesis
Arjan Egges
MIRALab - University of Geneva
Switzerland
egges@miralab.unige.ch
Tom Molet
MIRALab - University of Geneva
Switzerland
molet@miralab.unige.ch
Nadia Magnenat-Thalmann
MIRALab - University of Geneva
Switzerland
thalmann@miralab.unige.ch
Abstract
In this paper, we propose a novel animation approach
based on Principal Component Analysis that allows gener-
ating two layers of subtle motions: small posture variations
and personalised change of balance. Such a motion gener-
ator is needed in many cases when one attempts to create
an animation sequence out of a set of existing clips. In na-
ture there exists no motionless character, while in computer
animation we often encounter cases where no planned ac-
tions, such as waiting for another actor finishing his/her
part, is implemented as a stop/frozen animation. We iden-
tify many situations where a flexible idle motion generator
can help: from synchronisation of speech/body animation
duration, to dynamic creation of stand still variations in be-
tween two active plays. Our approach overcomes the limi-
tations of using a small set of existing clips as a basis for
synthesizing idle motions, such as unnatural repetition of
movements and difficulties to insert idle motions into an an-
imation without breaking its continuity. A realistic anima-
tion is obtained by blending small posture variations with
personalised balance shifting animations.
tor control system is required to transfer this property to Vir-
tual Human motions.
From now on, we will call the motions that occur in wait-
ing/idle states between animation clips
idle motions.
These
motions include changing balance because of fatigue, small
variations in body posture caused by small muscle contrac-
tions or eye blinking. There is very little work done to realis-
tically simulate idle motions, with an exception to the work
by Perlin [18], that describes a motion generator (based on
a noise function) for a few joints. However, human idle mo-
tions affect
all joints,
which cannot easily be solved by us-
ing noise functions on all joints, because of dependencies
between joints. Apart from generating random movements
for separate joints, another possibility is to take captured an-
imation files as a basis for generating idle motions. This re-
sults in idle motions that affect all joints, but unfortunately,
they are very repetitive and inflexible. Secondly, this ap-
proach generates transition problems between the anima-
tion clips.
In this paper, we propose a system for real-time virtual
human idle motion synthesis for the body. The generated
idle motions are based on statistical data obtained from a
large set of recorded animations. As a basis for the mo-
tion generation, we use a Principal Component Analysis
(PCA) to determine dependencies between joints in the var-
ious postures. This results in animations that affect all main
joints in a coherent way. Furthermore, the Principal Compo-
nents allow performing many other operations such as easy
blending and fitting of motions, while working with virtu-
ally independent variables in linear space, resulting in flex-
ible and realistic animations. With the proposed technique,
we can also emulate idle behaviour of a specific individual
by using animation segments of the recordings of this per-
son, and thus make a Virtual Human move like this person
would.
1. Introduction
Although virtual models of humans continue to improve
both in real-time and non-real-time applications, control-
ling and animating them realistically still remains a diffi-
cult task. Human beings are moving
all the time
and in a
unique way.
When animating a Virtual Human, one first has
to solve the problem of the lack of animation between dif-
ferent animation clips, to avoid an unnatural-looking frozen
posture between motions. Additionally, as every person has
a unique way of moving and standing, an individualised mo-
In the next section, we will discuss some related work
in this area. After that, we discuss the PCA of animations.
In Section 4, we present the main architecture of our idle
motion synthesizer. Section 5 and Section 6 discuss the dif-
ferent parts of the system and the techniques behind them
in more detail. In Section 7 we discuss the integration of
the idle motion engine in a real-time animation framework.
Section 8 shows some results of the idle motion synthesis
and Section 9 presents our conclusions and indications of
future work.
ments (such as gestures or other arm and head movements
during the balance shift), clear constraints have to be de-
fined during the annotation process.
3. Principal Components of Body Animations
There exist many techniques for animating virtual char-
acters. Two very commonly used techniques are:
•
Key-frames:
an animation is constructed from a set
of key-frames (manually designed by an animator or
generated automatically) by using interpolation tech-
niques. Although this method results in very flexible
animations, the realism of the animations is low, un-
less a lot of time is invested.
•
Pre-recorded animations:
an animation is recorded
using a motion capture/tracking system (such as Vi-
con or MotionStar). The animation realism is high, but
the resulting animation is usually not very flexible, al-
though methods have been developed to overcome part
of this problem [6].
A method like PCA can determine dependencies be-
tween variables in a data set. The result of PCA is a ma-
trix (constructed of a set of eigenvectors) that converts a
set of partially dependent variables into another set of vari-
ables that have a maximum independency. The PC variables
are ordered corresponding to their occurrence in the dataset.
Low PC indices indicate a high occurrence in the dataset;
higher PC indices indicate a lower occurrence in the dataset.
As such, PCA is also used to reduce the dimension of a set
of variables, by removing the higher PC indices from the
variable set.
A PCA on animation data has already been successfully
attempted by Alexa and M¨ ller [3]. In their work, they per-
u
form a PCA on animations represented by relative defor-
mations of the model, thus creating a ‘shape space’. As we
are focusing on humanoid animations and we want to be as
independent as possible from the geometry itself, we per-
form the PCA on the underlying joint skeleton, according
to the H-Anim standard [10]. For our analysis, we use a
subset of the H-Anim joints. This is done not only to im-
prove the real-time performance, but also because we be-
lieve that some joint sets can better be treated separately
to increase flexibility. For example, finger animations need
to be very detailed for gesture animations, and they would
probably not benefit much from an integrated PC analysis
on both body and finger joints.
2. Related Work
There is a broad number of algorithms for motion syn-
thesis from motion data, although they are seldom directed
to addressing the idle-motion concerns. Kovar et al. [13]
proposed a technique called motion graphs for generating
animations and transitions based on a motion database. It
could possibly be applied to other specific problems than
the demonstrated locomotion; however this technique re-
lies on a closed database of motions, going through a pre-
processing stage, whereas our technique smoothly adapts to
any animation schemes, being motion captured or computed
on the fly. This is a key aspect since the idle-motion is rarely
the end goal of animators, but rather a convenient way for
building a transition between different active movements. It
is therefore desirable to have activate/deactivate functional-
ities that can be triggered at AI or script level.
Li et al. [15] divided the motion into textons modelled
using linear dynamic system. Additionally, Kim et al. [12]
propose a method that analyses sound and that can gener-
ate rhythmic motions based on the beats that are recognised
in the audio. These techniques are best suited for motions
consisting of rhythmic patterns such as disco dance. The
method might eventually be applicable for idle motions, but
this paper proposes a much simpler solution.
Pullen and Bregler [19] proposed to help the process of
building key-frame animation by an automatic generation
of the overall motion of the character based on a subset of
joints animated by the user. There the motion capture data
is used to synthesize motion for joints not animated by the
user and to extract texture (similar to noise) that can be ap-
plied on the user controlled joints. Our method tries to min-
imise user intervention, and it is specifically oriented to-
wards real-time applications.
Lee et al. [14] propose a motion synthesis method based
on example motions that can be obtained through motion
capture. Also relevant work has been done by Arikan et
al. [5, 4] that define motion graphs based on an anno-
tated motion database. They present an automatic annota-
tion method, which could be applied also to the balance
shifting motions as an extension. However, since balance
shifting motion clips often contain unwanted joint move-
3.1. Motion Capture
The clips that we use to produce new motions, are
recorded using the Vicon motion capture system [1]. The
marker positions are converted into an H-Anim skeleton an-
imation represented by joint angle rotations and a global
(root) translation. Clips recorded by motion capture are usu-
ally bound to specific coordinate frame of the tracking sys-
tem and/or start each at different global position and orien-
tation. In order to remove those irrelevant initial conditions,
we apply a data alignment procedure on the root joint. De-
pending on the clip, we align position and orientation (2 +
3 degrees of freedom DOF), position and turn (2 + 1 DOF),
or only position (2 DOF). In all of these procedures the ver-
tical displacement is kept unchanged because it is valuable
information. The first solution is the most severe since it
resets the initial global posture back to the world’s origin.
This is the best choice solution as long as no relevant ini-
tial orientation must be preserved. However, in cases where
the clips start with meaningful tilt, roll orientation, such as
in a laying down posture, the second correction—align po-
sition and turn—is preferred. These two alignment proce-
dures cover most of the situations because the initial head-
ing posture is rarely relevant unless being part of the de-
sign of the motion clip, in which circumstance we can ei-
ther align only the position or use no correction at all.
Figure 1. Overview of the steps involved for
converting transformation matrices into Prin-
cipal Components and back. First we apply
the matrix logarithm to go to linearised trans-
formation matrices, and then convert the data
to PC space by using the PC matrix.
where:
3.2. Conversion from and to Principal Compo-
nents
In order to perform a PCA, we need to convert each
frame of the animation sequences in the data set into an
N
-
dimensional vector. In our case, one posture/key-frame is
represented by 25 joint rotations (including one
root
joint)
and one root joint translation.
Directly applying the PCA on the rotation matrix values
does not give a desirable result, since the rotation space has
a non-Euclidian geometry. However, a ‘linearised’ version
of the (orthogonal) rotation matrix can be constructed, as is
explained in the following paragraph.
For representing rotations, we use a skew-symmetric ma-
trix. For every real skew-symmetric matrix, its exponential
map is always a rotation matrix (see for example Cheval-
ley [7]). Conversely, given a rotation matrix
R,
there ex-
ists some skew-symmetric matrix
B
such that
R
=
e
B
. The
skew-symmetric matrix representation of a rotation is very
useful for motion interpolation [16, 2], because it allows to
perform linear operations on rotations. A three-dimensional
real skew-symmetric matrix has the following form:
0
c
B
=
−b
−c
b
0
−a
a
0
 
a
b
r
=
c
(2)
√
Given that
θ
=
a
2
+
b
2
+
c
2
, the exponential of a matrix
B
is defined by Rodrigues’ formula:
e
B
=
I
3
+
sin
θ
(1
−
cos
θ)
2
B
+
B
θ
θ
2
(3)
Similarly, methods exist that define how to calculate a de-
termination of the (multivalued) matrix logarithm. For ex-
ample, Gallier and Xu [9] present methods to calculate ex-
ponentials and logarithms, also for matrices with higher di-
mensions.
Using the vector representation shown in Equation 2 for
a joint rotation, a posture consisting of
m
joint rotations
and a global root translation can be represented by a vec-
tor
v
∈
R
3m+3
. In our case, we use 25 joints, so a posture
is defined by a vector of dimension 78. We apply a Princi-
pal Component Analysis on these posture vectors, resulting
in a PC space of equal dimension (see Figure 1).
4. Multi-level Idle Motions
We have separately recorded the motions of ten people
of both genders while they were in a conversation. This pro-
vides us with motion data of both gestures and idle motions.
In the recorded data, we have observed three common types
of idle behaviour:
(1)
Such an element can also be represented as a vector
r
∈
R
3
Posture shifts
This kind of idle behaviour concerns the
shifting from one resting posture to another one. For
example, shifting balance while standing, or go to a
different lying or sitting position.
Continuous small posture variations
Because of breath-
ing, maintaining equilibrium, and so on, the human
body constantly makes small movements. When such
movements are lacking in virtual characters, they look
significantly less lively.
Supplemental idle motions
These kinds of motions gen-
erally concern interacting with ones own body, for ex-
ample touching of the face or hair, or putting a hand in
a pocket.
In this paper, we focus on the first two types of varia-
tions. The supplemental idle motions often involve an inter-
action with one’s own body. This makes it difficult to sim-
ulate them in a generic way without using complex retar-
geting techniques [21, 22], that are difficult to integrate into
a real-time framework. In our system, we use pre-recorded
sequences for these kinds of motions, but this is not a very
flexible solution and it is part of our future work to extend
the system so that it can also generate these kinds of mo-
tions automatically. In the following sections, we will dis-
cuss generating both generic posture shifts and small pos-
ture variations for the case of idle motions while standing.
Additionally, in Section 7, we will show how the idle mo-
tions can be integrated within a real-time animation system.
Figure 2. This figure illustrates what the esti-
mated translation offset is in the
(x,
z)-plane
(frontal-lateral plane) for an animation se-
quence.
form the transitions between each of these categories. These
animation segments together form a
database
that is used to
synthesize balancing animations. In order for the database
to be usable, at least one animation is needed for every pos-
sible category transition. However, more than one animation
for each transition is better, since this creates more varia-
tion in the motions later on. The database of animation seg-
ments is called
A
from now on. In order to generate new an-
imations, recorded clips from the database are blended and
modified to ensure a smooth transition.
5.2. Removing the Translation Offset
Since we will later on use the animations from the
database to create balance-shifting sequences, it is neces-
sary to apply a normalisation step so that the relative transla-
tions between the animations are minimised. Therefore we
estimate the
translation offset
for each of the animation
sequences to the origin
(0, 0, 0)
by the mean value of the
first and last frames of the animation (see Figure 2). In or-
der to remove this offset from the animation, for each frame
we subtract the translation offset from the root translation.
There are other possibilities to estimate the translation off-
set, for example by searching the most central key-posture
in the animation, or by using the timing information to de-
termine the middle of the posture shift.
5. Posture Changing
Every person has a distinctive way of standing, sitting,
lying and moving. In fact, from only the animations, in most
of the cases people (family, friends or colleagues) can de-
termine which person they belong to. Our goal is to cap-
ture (at least some of) this personalised way of moving. In
this section we will propose an approach that allows creat-
ing an animation database from recorded clips. From this
database, we can generate fully automatically new realis-
tic animations. This results in a Virtual Human that mim-
ics the postures and the way of moving of the person that
was recorded. As an example to illustrate our techniques,
we use recordings of people standing.
5.3. Transition between Categories
5.1. The Basic Animation Structure
When a human is standing, he/she needs to change pos-
ture once in a while due to factors such as fatigue [20].
Between these posture changes, he/she is in a resting pos-
ture. We can identify different categories of resting pos-
tures, such as: balance on the left foot, balance on the right
foot or rest on both feet. As such, given a recording of some-
one standing, we can extract the animation segments that
We denote the set of categories of resting postures as
C.
As each animation depicts a transition from one category
to another, we can assign the first and last frame of each
a
∈
A
to the category in
C
that they belong to. There-
fore a category
c
∈
C
corresponds to a set of postures
Q
c
=
q
1
, . . . , q
m
. Apart from these postures, we add stand-
alone postures (that are not part of any animation in the
database) in order to increase the variety of resting postures
Figure 3. A simple example of a category
transition graph. In this case, there are two
categories: balance on the left foot and bal-
ance on the right foot. For both the cate-
gories, a minimum and maximum time is de-
termined from the data (written in millisec-
onds).
5.4.3. Retrieving the Best-fitting Animation
The earlier
constructed database of animations
A
may not contain all
possible animations for all the postures. Therefore, we have
developed a technique that allows to map an animation onto
a set of key-postures. In order for this technique to be as re-
alistic as possible, we have to determine which animation
in the database would fit best with the start posture
p
and
end posture
q.
In order to determine this, the system must
choose the animation that has its first frame and last frame
as close as possible to respectively
p
and
q.
Suppose that
the postures are represented by an
N
-dimensional vector of
PC values. We then define the distance between two pos-
tures
p
and
q
as the weighted Euclidean distance between
the two vectors (w
i
is the weight of variable
i):
N
d
p,q
=
i=1
w
i
·
(p
i
−
q
i
)
2
(4)
in the category. However, these postures should be within a
limited translation interval to avoid unnatural shifts during
the transition. If necessary, such postures are normalised by
setting the translation to the mean translation of the other
postures that are already in the posture set for the corre-
sponding category.
From the original animation data, we also determine
what the probability is that a transition occurs from one cat-
egory to another by counting the occurrences in the data.
The probability for a transition between category
c
1
and
c
2
is denoted by
P
(c
1
⇒
c
2
).
Obviously, the sum of the prob-
abilities of one category to others should be 1.
Furthermore, for each category
c,
we extract from the
data the minimum and maximum amount of time (denoted
by
T
c,min
and
T
c,max
) that a person stays in one of the cat-
egories, before performing a transition to another category.
Figure 3 shows a simple example of a probability distribu-
tion for two categories and their
T
c,min
and
T
c,max
.
Using these weights allows us to assign more importance
to parts of the posture that occur often in the data. As a re-
sult, less relevant parts of the posture are contributing less
to the distance between postures. The problem of estimat-
ing the distance between postures has also been addressed
by Kovar et al. [13], who use point clouds that are driven
by the skeletons. As values for the weights, we use the nor-
malised
1
values of the eigenvector of the PC matrix, since
these values correspond to the occurrence in the data of each
PC value.
In order to select the best fitting animation between
p
and
q,
we calculate for each animation
a
∈
A
the maxi-
mum
M
a
of the distances
d
a
0
,p
and
d
a
e
,q
. The animation
that we will use, is the one that has a minimal
M
a
.
5.4.4. Fitting the Selected Animation to the Begin and
End Posture
The animation
a
(=(a
0
, a
1
, . . . , a
e
))
starts at
frame 0 and ends at frame
e.
If we want to fit the animation
so that it starts at posture
p
and ends at posture
q,
we have to
transform
a
so that
a
0
=
p
and
a
e
=
q
(see Figure 4). This
problem can be solved as a specific case of a more general
problem:
Given an animation
a
and a list
L
of tuples
(f,
p)
where
f
is a frame number,
p
is a posture repre-
sented by a vector of PC values, and
L
i
=
(f
i
, p
i
)
is the
i-th
element in the list
L.
Furthermore,
∀L
i
∈
L
:
f
i
< f
i+1
(i.e. the elements in the list
are ordered according to increasing frame num-
ber). Finally,
∀L
i
∈
L
: 0
<= f
i
<= e.
How
can we modify the animation
a
so that is passes
through all the postures in the list
L
at their spec-
ified frame numbers?
1
The eigenvector
e
of a PC matrix has the property that
e
i
≥
e
i+1
for
0
≤
i < N
and also
e
i
≥
0
for
0
≤
i
≤
N
. The vector is normalised
by applying a multiplication factor so that
e
0
= 1.
5.4. Synthesis of Balance Changes
5.4.1. First step
As a first step, we choose a random cate-
gory
c
and then a random posture
p
∈
Q
c
. Furthermore, we
choose a random time value
t,
where
T
c,min
≤
t
≤
T
c,max
.
This already allows to create an animation consisting of two
key-frames containing posture
p
at time indices 0 and
t.
5.4.2. Choosing a New Category
After the posture at
time index
t,
we need to construct a transition to another cat-
egory. In order to define the next category, we again choose
it randomly, but according to the probability distribution
as defined in the category transition graph. This gives the
successor category
s.
Within this category, we choose ran-
domly a posture
q
∈
Q
s
. Now we have to retrieve the ani-
mation from posture
p
to posture
q
and add the key-frames
of the animation to the animation that we are constructing.

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin