Fidelity costs in look-alike avatars: hair, eye color, and familiarity in mixed reality

1 Introduction

Mixed Reality (MR) increasingly places life-sized representations of real people into shared physical space, and advances in scanning and modeling have made it feasible to build avatars that resemble specific individuals (Hollensen et al., 2020; Ma et al., 2021; Yu et al., 2025). These “look-alike” avatars, or digital doubles, are intended to stand in for a particular person. In workplace and collaborative settings, the people viewing such an avatar will frequently be colleagues, collaborators, or friends of the person depicted, which makes the observer’s judgment rather than the owner’s the one that governs interaction outcomes.

Avatar research has nonetheless been weighted heavily toward self-perception, focusing on how users experience and are affected by their own representations. Yee and Bailenson (2007) explicitly identified the mismatch between self-representation and others’ perception as an unresolved problem, and it has attracted comparatively little direct empirical attention since. Yet resemblance demonstrably shapes how observers respond, influencing social attraction, perceived competence, and recognition accuracy (Nowak and Fox, 2018; Nowak et al., 2009; Fysh et al., 2022). The present study addresses this asymmetry by placing other-perception at the center.

The question matters because avatars are seldom left as faithful reconstructions. Users routinely modify their representations toward preferred or idealized characteristics (Messinger et al., 2019; Freeman and Maloney, 2021), and hair in particular is among the most frequently altered attributes in avatar creation systems (Ducheneaut et al., 2009). Deviation from a person’s actual appearance is therefore the normal case rather than an artificial manipulation. What is unclear is what such deviation costs in the eyes of someone who knows the depicted person. This is a trade-off, where the literature documents the benefit to the user of self-expression, whereas the cost is borne by the observer, and it is the cost side that we measure in this paper.

Expectancy Violations Theory offers a principled account of why that cost should depend on the observer (Burgoon et al., 2016). A deviation can only constitute a violation to the extent that the observer holds an expectation precise enough to be violated. That precision varies with acquaintance rather than being simply present or absent. An observer who has seen only a photograph holds a thin, single-view record, and matching an unfamiliar face from a single image is error-prone even when that image is available for direct comparison (Jenkins et al., 2011; Megreya et al., 2013). An observer who knows the person holds a representation accumulated across many views, expressions and lighting conditions, against which a discrepancy is far easier to detect. On this account familiarity is not merely a moderating nuisance variable but the factor that determines whether an alteration is read as preference or as error. Supporting evidence comes from work showing that negative responses to near-human characters track specific violations of expectation rather than overall realism (Burleigh et al., 2013; De Borst and de Gelder, 2015), that familiar faces are represented with sufficient precision to make distortions detectable (Diel and Lewis, 2022), and that observers judge others more accurately than people judge themselves (Vazire and Carlson, 2011). Together these predict that observers who know the depicted person will evaluate a look-alike avatar with deviated features more harshly than observers who do not.

Prior research has shown that not all features carry equal weight. Takano and Taka (2022) report that avatar attributes which can be altered in real life are more readily accepted as part of a person’s identity. Hair color is routinely changed in everyday life, so a lightened variant remains within the plausible range of the same individual. Eye color is not, so altering it departs from an attribute observers treat as stable. This predicts an asymmetry in which eye deviation is costlier than hair deviation. Users appear already to behave as though this were true, preserving their own eye color when building avatars (Wu et al., 2023) while altering hair freely (Ducheneaut et al., 2009).

This framing also reconciles an apparent conflict with prior work. Schwind et al. (2015) found that neither hair nor eye color significantly affected likability or attractiveness judgments of avatars. Those judgments concerned generic faces, for which a color choice is simply an aesthetic preference with no referent to contradict. When the avatar is a look-alike of a specific known person, the same change becomes a discrepancy against a stored representation. The distinction is between customization-as-preference and customization-as-error, and it predicts precisely where color should and should not matter.

Finally, familiarity presents an apparent paradox. It improves recognition of avatars (Fysh et al., 2022; Gonzalez-Franco et al., 2016; Park et al., 2021), yet by furnishing a precise expectation it should also increase the penalty when that expectation is violated. Whether both hold, and whether they apply to the same perceptual judgments, has not been established.

We therefore examine how deviations in hair and eye color on look-alike avatars, viewed as life-sized holograms through a HoloLens 2, affect observer judgments of acceptance, humanlikeness, and eeriness, and how prior familiarity with the depicted person moderates those judgments. MR is the deployment context that motivates the question, and the high perceptual standards that a life-sized co-located representation invites are part of why expectancy is high in this setting. We did not compare MR against other presentation formats, so we treat display context as a boundary condition on interpretation rather than as a tested factor.

1.1 Research questions

This study addresses the following research questions:

• RQ1: How do deviations in hair color and eye color independently affect perceived acceptance, humanlikeness, and eeriness of look-alike avatars, and does eye deviation carry a greater cost than hair deviation?
• RQ2: Do hair and eye deviations combine additively, or does deviating both features simultaneously produce an effect greater than the sum of the individual deviations?
• RQ3: Does prior familiarity with the depicted person change observers’ evaluations, and does it alter the magnitude of the penalty attached to a given feature deviation?
• RQ4: Do these effects differ between two look-alike avatar reconstructions, and can such differences be attributed to any single attribute of the depicted individuals?

2 Related work

Avatar research has concentrated on self-avatars, first-person representations of the user’s own body (Steed et al., 2016). This work shows that self-avatars shape interaction dynamics and behavioral responses (Pan and Steed, 2017), contribute to presence and realism (Slater et al., 2009), and that users are more comfortable when their avatar resembles them (Bailenson et al., 2006). Resemblance to the owner has also been linked to greater social attraction, credibility, and competence (Nowak and Fox, 2018; Nowak et al., 2009; Nowak and Rauh, 2005).

Look-alike avatars raise a different question. A digital double is built to represent a specific real person, so the informative judgment is frequently made by someone other than its owner (Pakanen et al., 2022; McDonnell et al., 2012; Zibrek et al., 2019; Suk and Laine, 2023; Oyekoya and Baffour, 2025). Methods for producing such avatars now include single-image reconstruction, multi-camera photogrammetry, volumetric capture (Feng et al., 2014; Zollhöfer et al., 2011), and 3D-aware generative models (Ma et al., 2021; Yu et al., 2025), all of which optimize objective resemblance. Far less is known about what happens when that resemblance is deliberately relaxed, and how observers respond when it is.

Digital characters that approach human appearance can nonetheless provoke discomfort, classically described as the uncanny valley (Mori, 1970; Mori et al., 2012). Subsequent work found that negative responses appear to be driven less by realism per se than by identifiable violations of expectation, such as anomalous facial features or incongruent behavior (Burleigh et al., 2013; de Borst and de Gelder, 2015). Kätsyri et al. (2015), reviewing the empirical literature, similarly concludes that perceptual mismatch provides better support than global realism as an explanation of these effects. On this reading, higher fidelity does not itself cause discomfort, rather it raises the precision with which discrepancies are detected.

Evidence from dynamic features is consistent with this. Frampton-Clerk and Oyekoya (2022) found that lip animation without accompanying facial expressivity provoked uncanny responses, whereas full expression with synchronized audio did not, indicating that partial rather than low realism is the problem. Seyama and Nagayama (2007) showed that realistic faces evoke negative impressions when abnormal features are present, with effects concentrated on distortions of the eyes, and Weibel et al. (2010) report that ocular attributes in particular shape impression formation. These studies concern dynamic and structural anomalies; whether the same logic extends to color, a purely surface attribute that leaves geometry and motion untouched, is one of the questions this study addresses.

Avatar customization has been studied mainly as a route to identification and self-expression. Users modify avatars toward preferred or idealized characteristics (Messinger et al., 2019), present themselves consistently with offline identity while also exploring alternatives (Freeman and Maloney, 2021), and want more flexibility than current systems provide, particularly for hair (Hube et al., 2024; Jones et al., 2021). Ducheneaut et al. (2009) identified hair style and color as high-impact, frequently altered attributes, describing hair as a salient and malleable identity cue. Liu et al. (2025) report the first-person counterpart of the effect examined here, that users dislike their digital doubles when customization departs too far from their self-concept.

This literature is almost entirely first-person, concerning what users choose and how they feel about their own representations. It establishes that deviation is common and motivated, but not what it costs when a third party who knows the depicted person evaluates the result. That asymmetry is the gap addressed here.

Findings on color are mixed in an instructive way. Schwind et al. (2015) found that hairstyle influenced avatar likability while hair color did not affect attractiveness judgments, and separately that eye color did not significantly influence likability, whereas eye shape and size did. These studies evaluated generic faces on aesthetic criteria, where a color change is a matter of preference and there is no referent against which it could be wrong.

Takano and Taka (2022) provide a principle that predicts when color should matter. Avatar attributes that can be altered in real life are more readily accepted as belonging to the person while those that cannot are treated as fixed identity markers. Hair color is routinely changed in everyday life, so a lightened variant remains a plausible version of the same individual. Eye color is not, so altering it departs from an attribute observers treat as stable. Consistent with this, Wu et al. (2023) report that users typically select eye colors resembling their own when building avatars, while hair is altered freely (Ducheneaut et al., 2009). First-person choice patterns and third-person perceptual cost may therefore align, a convergence this study tests directly.

Familiarity determines whether an expectation exists at all. Expectancy Violations Theory holds that departures from an established interpersonal expectation attract heightened attention and evaluative response (Burgoon et al., 2016), which requires that the observer possess a sufficiently precise expectation to begin with. Diel and Lewis (2022) show that familiarity sharpens sensitivity to facial distortion by strengthening the observer’s norm for a known individual, and Fysh et al. (2022) that recognition of avatars improves when the depicted person is already known. Neurocognitive work indicates that avatars of oneself or a known other engage stronger representation processes than unfamiliar faces (Gonzalez-Franco et al., 2016; Park et al., 2021), and Park et al. (2021) further report that avatars reproducing habitual facial characteristics are more recognizable.

Familiarity therefore appears to cut both ways. It aids recognition, yet it also supplies the precise expectation whose violation should be penalized, and Vazire and Carlson (2011) note that observers judge others more accurately than people judge themselves, implying that familiar collaborators may be harsher critics of a digital double than its owner. Suh et al. (2011) emphasize self-congruity in shaping attitudes toward avatars, while Poletaykina et al. (2025) link familiarity to trust and social acceptance rather than to affective discomfort, and Guo et al. (2024) find that familiarity can bridge uncanny responses when behavior matches expectation. Whether familiarity amplifies all evaluative responses uniformly, or only those that depend on a person-specific referent, remains open. That question is the focus of RQ3.

Three gaps follow from this literature. First, the evidence on avatar modification is overwhelmingly first-person, describing what users choose for themselves rather than what observers make of the result, despite other-perception being the judgment that governs collaborative outcomes. Second, work on expectancy violation in virtual characters has concentrated on structural and dynamic anomalies, leaving open whether purely surface deviations such as color, which alter no geometry and no behavior, provoke comparable responses. Third, familiarity is established as an influence on recognition, but its role in setting the expectation against which a deviation is judged has not been separated from its role in aiding identification. The present study addresses all three by manipulating two localized color attributes on look-alike avatars of specific individuals and comparing observers who know those individuals against observers who do not.

3 Methodology

3.1 Participants

An a priori power analysis was conducted in G*Power 3.1 for a repeated-measures ANOVA testing a within-between interaction. Assuming a medium effect size ( f = 0.25 ), α = . 05, power = 0.95, two groups, eight repeated measurements per participant (2 actors × 4 deviation conditions), a correlation among repeated measures of 0.5, and no nonsphericity correction ( ϵ = 1 ), the analysis returned a required total sample size of 24 (noncentrality parameter λ = 24.00, critical F = 2.07, numerator d f = 7, denominator d f = 154, achieved power = 0.96). We note that this analysis was specified for the omnibus within-between interaction and not for the between-subjects familiarity contrast, which is the effect most constrained by the realized group sizes; this limitation is addressed in Section 5.6.

A total of 30 participants (18 male, 12 female) were recruited. All participants were aged between 18 and 24 years. Participants were assigned to a Familiar group ( n = 9 ) or an Unfamiliar group ( n = 21 ) according to their prior relationship with the two actors. Familiarity was established by recruitment design rather than post hoc classification. The nine familiar participants were known in advance to have been mentored by both actors during a high-intensity, eight-week summer program, and were recruited on that basis. The end-of-session question described in Section 3.4 served as a confirmation check and returned no reclassifications. All nine familiar participants were acquainted with both actors, so familiarity is not confounded with actor identity. The size of the familiar group was constrained by the number of program participants available. To maximize the strength of the familiarity manipulation, the study was run during the final week of the program. Over this period, participants engaged in frequent one-to-one interactions with the actors across a variety of contexts, including formal educational seminars, ice-breaker events, and informal social outings (e.g., a boat cruise). This multi-contextual exposure ensured that participants were well-acquainted with the actors’ facial features, expressions, and 3D physical presence prior to evaluating their digital avatars. Twenty-one of the participants received extra course credit for participation. Students who chose not (or were unable) to participate were given alternative extra credit opportunities. Since the familiar participants were recruited from a program in which two of the authors were their mentors, we note the steps taken to protect voluntariness. Participation was not solicited by either actor, the mentoring relationship carried no assessment component, an alternative extra-credit activity of equal value was available to anyone who declined or withdrew, and participation or non-participation was not reported back to the actors. All participants reported normal or corrected-to-normal vision. This study was reviewed by Institutional Review Board (IRB) at Hunter College, City University of New York under protocol number 2024-0419-Hunter, dated 20 June 2024. The study was granted exempt status under Category 3 (benign behavioral interventions with adult participants), and under that exemption the Board waived the requirement for written informed consent. Participation was voluntary, and informed consent was obtained verbally from every participant before taking part.

3.2 Materials

The stimuli consisted of two look-alike avatars, one based on a male actor (Actor A) and one on a female actor (Actor B). Both actors are authors of this paper and provided explicit, written consent for the use and publication of their facial likenesses. Neither actor administered the study. The experimenter was a third party who did not appear in any stimulus.

The avatars were created with Reallusion Character Creator v4.23 and the Headshot plugin v1.11. This is a semi-automated pipeline, where the plugin generates an initial head mesh from a single frontal photograph, and the operator then refines it manually. We describe it as image-guided manual modeling throughout, and note that it is not a photogrammetric or multi-view reconstruction method. Generation relies on a single frontal view, so the resulting head shape is an approximation of the actor’s rather than a metrically accurate reconstruction. The plugin was run in Auto mode, which transfers facial features from the photograph to a pre-rigged base model and derives skin tone and texture from the same image, after which manual adjustments were applied. Sculpting tools were then used to refine head shape and to fit a hair asset, and Character Creator assembled the head, hair, and body into a single rigged character. The same operator produced both avatars using the same base model, workflow, and render settings. The two actors differ in gender and race, and separate hair assets were therefore used but the specific assets were not recorded, which we note as a limitation on exact reproducibility.

For each actor, four distinct conditions were rendered: (i) Baseline/Unaltered (natural hair and eye color, Figures 1b, 2b); (ii) Hair Deviation (modified hair color with natural eyes, Figures 1c, 2c); (iii) Eye Deviation (modified eye color with natural hair, Figures 1d, 2d); and (iv) Dual Deviation (modification of both hair and eyes simultaneously, Figures 1e, 2e). Deviation was implemented by changing the color of the hair and/or eyes away from the actor’s own coloring, using the color and texture tools in Character Creator. Both actors have dark brown hair and brown eyes. Table 1 reports the color specification. Eye values are the albedo values set in the authoring tool; hair values are the mean rendered color of the hair region measured directly from the stimulus images, obtained by differencing the baseline and deviated renders, and sampling the pixels that changed most. Rendered values are reported for hair because the deviated albedo was not retained, and because rendered color is in any case the closer proxy for what participants saw. All values are converted to CIELAB, and deviation magnitude is expressed as Δ E a b * (CIE76) from the corresponding baseline.

TABLE 1. Color specification for the deviation manipulations. Eye values are authoring-tool albedo while hair values are mean rendered color of the hair region measured from the stimulus images. Δ E a b * is computed against the corresponding baseline. Note that the two manipulations differ in kind. The eye deviation is predominantly a hue reversal at near-constant lightness, whereas the hair deviation is predominantly a lightness increase at near-constant hue.

Two properties of this specification warrant comment. First, the deviations were substantial in colorimetric terms, with Δ E a b * between 20 and 34 against baseline, well above any plausible discrimination threshold under ideal viewing. Second, and more importantly for interpretation, the two manipulations were not equivalent in kind. The eye deviation was principally a change of hue, rotating from a warm brown (19°) to a cool slate-teal (193°) at almost unchanged lightness. The hair deviation was principally a change of lightness, increasing L * by 18–31 points while leaving hue essentially fixed within the warm brown-orange family. The customized eye color is therefore more accurately described as a dark, moderately desaturated slate-teal than as “blue”, and the customized hair as a substantially lightened brown rather than a categorically distinct blond. We return to the consequences of this asymmetry in Section 5.6.

Each avatar was presented with a neutral standing animation and scaled to be lifelike relative to the actors’ heights. The same nominal hair and eye targets were applied to two actors with different baseline coloring and skin tone, so the perceived magnitude of deviation was not equated across identities, as the differing Δ E a b * values in Table 1 show.

3.3 Apparatus

The avatars were imported into Unity 2019.4.40f1 and deployed to a Microsoft HoloLens 2 Mixed Reality headset. The device has a transparent holographic display with a 52° diagonal field of view and 2k 3:2 light engines per eye. This apparatus was selected to approximate a social Mixed Reality setting, in which a digital double appears as a life-sized hologram co-located in the participant’s physical space rather than on a two-dimensional screen. Optical see-through displays have known constraints in color reproduction, brightness, and resolution, so all conditions were presented using identical device settings and environmental lighting, and any such limitation therefore applied equally across all stimuli.

3.4 Procedures

Upon arrival, participants were briefed on the HoloLens 2 and underwent a standard eye-calibration sequence. They were then shown shoulders-up photographs of both actors (Figures 1a, 2a) and could ask to see these images again at any point. The eight avatars were presented one at a time, as shown in Figure 3, in an order randomized independently for each participant to distribute any order effects across conditions. For each avatar, participants were encouraged to walk around and observe the hologram from multiple angles. After each exposure, the experimenter read the five questions aloud and recorded the participant’s spoken responses before proceeding to the next condition.

Participants were permitted, but not prompted, to return to a previously viewed avatar and revise an earlier rating. In practice this facility was rarely used. Its frequency was not systematically logged, so we cannot report how often revisits occurred or how many ratings changed, and we are unable to conduct the first-pass sensitivity analysis that this design would otherwise permit. Presentation order was likewise not recorded and cannot be entered as a covariate. Both are acknowledged as limitations in Section 5.6, and the facility is removed in our ongoing work. At the end of the session, participants were asked whether they knew either actor, which served as a confirmation check on the familiarity assignment established at recruitment.

3.5 Measures

Participants evaluated each avatar using a structured five-item questionnaire assessing perceived resemblance, willingness to deploy the avatar in a formal setting, recognizability, perceived humanlikeness, and perceived eeriness. The first three items were adapted from Blom et al. (2014) and rated from 1 (not at all) to 10 (completely). Item 1 asked how closely the avatar represents the actor (resemblance); Item 2 asked whether the actor should use the avatar in a formal social setting such as a work teleconference (usage); Item 3 asked how likely the participant could pick the actor’s avatar out of a crowd (recognition). Item 1 served as a manipulation check on perceived fidelity following deviation. Item 3 indexes whether the subject’s identity remained discernible after deviation. Item 2 indexes the threshold for social acceptance of a deviated look-alike avatar. We note one adaptation. In Blom et al. (2014), participants evaluated avatars of themselves, so the items carried components of ownership and self-congruity. Here participants evaluate a third party’s avatar, and those components do not apply. The items are therefore used as measures of observer-judged fidelity and social acceptability rather than of self-identification, which is the construct relevant to the present question. The remaining two items were adapted from Burleigh et al. (2013) and rated from 1 (not at all) to 7 (completely). Item 4 asked the extent to which the avatar is humanlike; Item 5 asked how eerie the avatar is. Burleigh et al. (2013) report that negative responses to near-human characters track specific violations of expectation rather than overall realism, and these two items index the two components of that response. Both measures consist of a single item, and validated multi-item instruments exist for these constructs, so the implications of not using them are considered in Section 5.6.

4 Results

The four avatar conditions form a fully crossed 2 × 2 within-subjects factorial, hair (unaltered or deviated) × eye (unaltered or deviated), rather than four arbitrary levels of a single factor. We therefore analyzed the design as a 2 × 2 × 2 mixed factorial, with hair deviation, eye deviation, and actor identity (Actor A, male; Actor B, female) as within-subjects factors and familiarity (familiar or unfamiliar) as a between-subjects factor. This parameterization is what permits a direct test of whether deviating both features simultaneously produces an effect greater than the sum of the two individual deviations, which is the question posed by RQ2. A compounding effect is a claim of super-additivity, and super-additivity is tested by the hair × eye interaction term. Each effect was computed as a single degree-of-freedom contrast on each participant’s own ratings and tested directly, which is the standard contrast-based analysis of a within-subjects factorial. Effects are reported collapsed across actor identity unless stated otherwise. Where a family of related contrasts was tested, p values were adjusted using the Holm procedure and both unadjusted and adjusted values are reported. Table 2 provides a consolidated summary of the key inferential statistics for all three outcomes. Full detail, including omnibus tests, pairwise comparisons, and GEE parameter estimates, is provided in the subsections below. Each analysis addresses a specific research question. The 2 × 2 factorial decomposition of hair and eye deviation addresses RQ1, which concerns their independent effects and relative size, and RQ2, which concerns whether they combine additively; it is reported for all three outcomes in Panel A. The familiarity effects address RQ3 and the actor identity effects address RQ4, both reported in Panel B. The acceptance composite is a summed score across three items and is analyzed with repeated-measures ANOVA and within-subjects contrasts. Humanlikeness and eeriness are single ordinal items and are analyzed with cumulative logit GEE, within which the same 2 × 2 decomposition is fitted so that the treatment of the factorial is consistent across all three outcomes.

TABLE 2. Summary of key inferential statistics.

Panel A reports the 2 × 2 decomposition of feature deviation (hair × eye) for each outcome. For acceptance, a summed composite, estimates are mean within-subjects contrasts with Holm-adjusted p values across the three effects. For humanlikeness and eeriness, both single ordinal items, the same decomposition is reported from cumulative logit GEE models. The χ 2 and p columns are Type III Wald tests and are unadjusted. Odds ratios are given only for the hair × eye terms, where the term is of highest order and the Type III test therefore coincides with the corresponding parameter test, so the interval and the χ 2 describe the same quantity. For the main effects the Type III test averages over the other factor and has no matching single parameter, so no odds ratio is reported. The final column gives the within-subjects contrast on the original rating scale for comparison of direction and magnitude. Panel B reports the familiarity and actor identity effects; these are Type III tests, which average over the levels of any interacting factor and therefore do not correspond to the reference-level parameters in Table 3 (see Section 4.2.1). ns = not significant.

TABLE 3. GEE parameter estimates for humanlikeness and eeriness from the four-level parameterization of feature deviation (cumulative logit, exchangeable working correlation, robust covariance; both models converged).

These are separate fits from the 2 × 2 models summarized in Table 2, Panel A, whose working correlation structures are stated per model at the start of this subsection. Reference categories are the unaltered baseline avatar, Actor A, and unfamiliar. Odds ratios below 1 indicate a lower probability of higher ratings on the outcome. Threshold parameters and the parameters of all interaction terms are omitted for brevity; the interaction terms retained in the model are those listed in Table 2, Panel B. The familiarity and actor identity rows below are simple effects at the reference cell and are not the omnibus tests for those factors. Where they differ from the Type III tests in Table 2, Panel B, the Type III test is the one on which the corresponding claim rests (Section 4.2.1).

4.1 Avatar acceptance (resemblance, usage, and recognition)

The first three items measured perceived resemblance, usage, and recognition, and were summed to give a composite acceptance score (range 3–30) for each participant in each condition. Computed by pooling all 240 observations, Cronbach’s alpha was 0.721. That estimate treats repeated observations from the same participant as independent and is therefore inflated. Computed within condition, where the independence assumption holds, alpha ranged from 0.469 to 0.740 (M = 0.64 across the 8 cells). The composite is accordingly retained on conceptual rather than psychometric grounds, since resemblance, usage, and recognition are distinct facets of whether a look-alike avatar is accepted as standing in for its referent, and they are analyzed together for that reason. The three items are also reported separately in Figure 4B, in Tables 4, 5, and as a robustness check in Table 6. Normality was assessed by visual inspection of stem-and-leaf plots and by Shapiro-Wilk tests. The hypothesis of normality was not rejected for any condition ( p > . 05 ). Mauchly’s test indicated that sphericity was violated for feature deviation, χ 2 ( 5 ) = 16.98 , p = . 005, and degrees of freedom for all effects involving that factor are therefore Greenhouse-Geisser corrected (ϵ = 0.722 for feature deviation; ϵ = 0.958 for the actor × deviation term). Levene’s test was non-significant for all 8 cells (p ranging from 0.115 to 0.906), so the assumption of homogeneity of variances was met despite the unequal familiarity group sizes. Descriptively, acceptance was highest at Baseline ( M = 20.78 , S D = 4.65 ) and declined through Hair Deviation ( M = 18.90 , S D = 4.66 ) and Eye Deviation ( M = 16.97 , S D = 4.91 ) to Dual Deviation ( M = 15.38 , S D = 5.27 ); see Figures 4a, 5a. The omnibus effect of feature deviation was significant, F ( 2.17 , 60.62 ) = 35.89 , p < . 001 , η p 2 = . 562, as was the effect of actor identity, F ( 1,28 ) = 10.64 , p = . 003 , η p 2 = . 275, with higher acceptance for Actor A ( M = 18.60 ) than Actor B ( M = 16.16 ). These are estimated marginal means, which weight the two familiarity groups equally and therefore differ slightly from the unweighted condition means recoverable from Tables 4, 5.

TABLE 4. Median/mean (SD) ratings for Actor A (male) across the four feature deviation conditions (n = 30 per cell). Resemblance, usage, and recognition were rated 1–10; humanlikeness and eeriness were rated 1–7.

TABLE 5. Median/mean (SD) ratings for Actor B (female) across the four feature deviation conditions (n = 30 per cell). Resemblance, usage, and recognition were rated 1–10; humanlikeness and eeriness were rated 1–7.

TABLE 6. Robustness check. Each effect computed separately on each component item of the acceptance composite. The three deviation rows per item are the 2 × 2 decomposition collapsed across actor identity; every item independently reproduces the composite pattern, namely reliable hair and eye main effects, a larger eye effect, and a null hair × eye interaction. The actor identity and familiarity rows are reported per item because the three items index different facets of acceptance. Deviation and actor identity are within-subjects contrasts, t ( 29 ); familiarity is a between-subjects comparison, t ( 28 ). These are contrasts computed identically to the rows above them, so they are not numerically identical to the mixed-model F tests for the composite in Table 2, Panel B, which estimate actor identity and familiarity jointly.

4.1.1 Hair and eye deviation act additively (RQ2)

Decomposing feature deviation into its crossed components (Table 2, Panel A) showed significant main effects of both hair deviation, M = − 1.73, 95% CI [ − 2.42 , − 1.05 ], t ( 29 ) = − 5.15, p < . 001, d z = 0.94, and eye deviation, M = − 3.67, 95% CI [ − 4.59 , − 2.74 ], t ( 29 ) = − 8.10, p < . 001, d z = 1.48. Critically, the hair × eye interaction was not significant, M = + 0.30, 95% CI [ − 0.61 , + 1.21 ], t ( 29 ) = 0.68, p = . 503, η p 2 = . 016. The confidence interval for the interaction comfortably includes zero, and its point estimate is small relative to either main effect. Deviating both features simultaneously therefore produced a decrement that is not distinguishable from the sum of the two individual decrements (− 5.40 observed against − 5.69 predicted under strict additivity). We find no evidence of a compounding effect. The lowest acceptance scores in the dual condition are fully accounted for by two independent, additive contributions.

4.1.2 Eye deviation is costlier than hair deviation (RQ1)

The two main effects differed in magnitude, with eye deviation producing approximately 2.1 times the decrement of hair deviation. A direct contrast confirmed that this difference is reliable for acceptance, M = − 1.93, 95% CI [ − 2.92 , − 0.95 ], t ( 29 ) = − 4.02, p < . 001, p Holm = . 001, d z = 0.73. All six pairwise comparisons among the four conditions were significant after Holm correction (Table 7), confirming that each successive level of deviation was distinguishable from every other. In a balanced 2 × 2 the hair-versus-eye main-effect contrast reported above is algebraically identical to the hair-only versus eye-only comparison in that table; the two are the same test, and the adjusted p values differ only because the correction families differ. The value reported here belongs to the three-outcome family.

TABLE 7. Pairwise comparisons among the four feature deviation conditions for composite acceptance, collapsed across actor identity. Positive differences indicate higher acceptance for the first-named condition. p values are Holm-adjusted across the six comparisons.

4.1.3 Familiarity as a criterion shift rather than a change in penalty (RQ3)

A significant main effect of familiarity was found, F ( 1,28 ) = 4.85 , p = . 036 , η p 2 = . 148, with participants familiar with the actors rating the avatars lower overall ( M = 15.81 , S E = 1.19 ) than unfamiliar participants ( M = 18.95 , S E = 0.78 ), as shown in Figure 5b. This is a main effect rather than a moderation. The familiarity × deviation interaction was not significant, F ( 2.17 , 60.62 ) = 0.21 , p = . 826 , η p 2 = . 008, and testing familiarity against each component of the 2 × 2 separately confirmed the same pattern, since the two groups did not differ in the size of the hair deviation effect (M = − 1.39 familiar against M = − 1.88 unfamiliar, t ( 28 ) = 0.66 , p = . 513), the eye deviation effect, where the two group means coincided at M = − 3.67 while differing in dispersion (S D = 3.35 and 2.10 respectively), t ( 28 ) = 0.00 , p = 1.000, or the hair × eye interaction ( t ( 28 ) = − 0.77 , p = . 449 ). We therefore found no evidence that the two groups differed in the size of any deviation effect, though with nine familiar participants these tests were underpowered and a difference of moderate size would not reliably have been detected. A non-significant interaction indicates that no difference was detected, not that the groups responded uniformly, and we do not interpret these nulls as evidence of equivalence. Because the three component items index different facets of acceptance, the familiarity and actor identity effects were also computed separately on each (Table 6). The familiarity effect is carried principally by the resemblance item, M = − 1.25, t ( 28 ) = − 2.61, p = . 014, with usage, M = − 1.20, t ( 28 ) = − 1.55, p = . 133, and recognition, M = − 0.69, t ( 28 ) = − 1.13, p = . 267, pointing in the same direction but not reaching significance individually. Actor identity is distributed more evenly across the three items. The familiarity effect on the composite is therefore strongest for judgments of likeness to the depicted person and weaker for the willingness and recognizability facets, which is consistent with an account in which knowing the person sharpens the comparison against them rather than altering the practical judgment of whether the avatar can be deployed.

4.1.4 Actor identity (RQ4)

A significant interaction was observed between actor identity and feature deviation, F ( 2.88 , 80.50 ) = 5.39 , p = . 002 , η p 2 = . 161. As shown in Figure 4a, Actor A was rated higher at baseline ( M = 22.80 ) than Actor B ( M = 18.77 ), but this gap narrowed under dual deviation (Actor A M = 15.73, Actor B M = 15.03). As set out in Section 5.4, the two actors differed in baseline reconstruction quality as well as in gender presentation and skin tone, so this interaction cannot be attributed to any one of those attributes and is reported as a difference between two specific reconstructions.

4.2 Humanlikeness and eeriness

The remaining two items measured perceived humanlikeness and eeriness. Responses to the negatively scored item (eeriness) were first inverted. Cronbach’s alpha across the two items was low ( α = 0.581 ), indicating that they do not index a single underlying construct. We therefore analyzed them separately rather than compositing them. We note that this value is a correlation between two conceptually distinct measures and is not a reliability estimate for either item individually. Both outcomes are ordinal single items, and were therefore analyzed with repeated-measures Generalized Estimating Equations using a multinomial distribution and a cumulative logit link (IBM SPSS Statistics, version 31.0.0.0; IBM Corp., Armonk, NY), with an exchangeable working correlation matrix and robust covariance estimation. Actor identity, familiarity, and feature deviation were entered as categorical factors. Reference categories were Actor A, unfamiliar, and the unaltered baseline avatar respectively. The baseline avatar is the appropriate reference because it is by construction the closest available approximation to each actor, as confirmed by the acceptance results above. To keep the analytic treatment consistent across outcomes, the 2 × 2 decomposition reported in Table 2 was fitted for these two items inside the GEE framework, entering hair deviation, eye deviation and their interaction as crossed factors alongside actor identity and familiarity. Working correlation structures differ across these models and are stated per model. The four-level models reported in Table 3 converged under an exchangeable working correlation for both outcomes. Among the 2 × 2 models, the humanlikeness model likewise converged under an exchangeable structure. The eeriness model did not. It reached the maximum iteration count without meeting the convergence criterion, and the output produced at the final iteration relied on a ridge adjustment to the working correlation matrix; raising the iteration limit terminated in an estimation failure instead. This is unsurprising given that an 8 × 8 exchangeable correlation matrix must be estimated from 30 subjects. We therefore fitted the 2 × 2 eeriness model with an independent working correlation and robust (sandwich) standard errors, under which the parameter estimates remain consistent for the population-averaged model and the standard errors continue to account for clustering, at some cost in efficiency. Because no admissible exchangeable fit was obtained for this model, we make no claim that the eeriness results are corroborated under that structure. For comparison, the corresponding within-subjects contrasts are reported alongside the GEE estimates in Table 2, and the two approaches agree on all six tests. Two kinds of quantity therefore appear for these two outcomes and should be read differently. The inferential tests are the GEE Type III Wald tests, which treat the ratings as ordinal and make no interval-scale assumption. The mean contrasts reported alongside them, including the direct eye-versus-hair comparisons below, treat the rating as interval-scaled and are offered as descriptive summaries on the original scale, since an odds ratio does not convey the size of a shift in rating units. No conclusion rests on the parametric quantities alone. Where the two agree, as they do on all six factorial tests, we take the ordinal model as the basis for the claim.

The GEE models reported in Table 3 retain the four-level parameterization of feature deviation and are given for the level-wise comparisons against baseline.

4.2.1 Type III tests and parameter estimates answer different questions

Two quantities appear in the tables that follow and should not be read as interchangeable. The Type III Wald tests in Table 2, Panel B assess a factor averaged over the levels of any factor it interacts with. The parameter estimates in Table 3 are simple effects evaluated at the reference level of those interacting factors, which here are Actor A and the unaltered baseline. Where a model contains interactions involving a two-level factor, the two therefore diverge, and the divergence is a property of the parameterization rather than a discrepancy. Where a model contains no such interaction the two coincide exactly, as they do in the 2 × 2 models reported above, in which the Type III and parameter-level Wald statistics for familiarity agree to four decimal places. We report Type III tests as the omnibus main-effect tests throughout and treat them as the basis for the claims that follow.

4.2.2 Humanlikeness (RQ1, RQ3, RQ4)

Feature deviation significantly affected perceived humanlikeness, Wald χ 2 ( 3 ) = 36.34 , p < . 001 (Figure 5c). At the parameter level (Table 3), only dual deviation differed reliably from baseline, B = − 1.00, S E = 0.42, p = . 018, OR = 0.37, 95% CI [ 0.16 , 0.84 ]. Neither eye deviation alone ( p = . 339 ) nor hair deviation alone ( p = . 520 ) significantly reduced humanlikeness relative to baseline. Fitting the 2 × 2 decomposition inside the GEE framework nevertheless showed reliable main effects of both hair deviation, Wald χ 2 ( 1 ) = 4.34, p = . 037, and eye deviation, Wald χ 2 ( 1 ) = 9.77, p = . 002, with no hair × eye interaction, Wald χ 2 ( 1 ) = 2.18, p = . 140, OR = 0.72, 95% CI [ 0.46 , 1.12 ]. Both main effects were in the direction of reduced humanlikeness under deviation, as the corresponding contrasts in Table 2 show. The within-subjects contrasts on the raw scale agree with all three tests (Table 2, Panel A). The apparent tension with the level-wise parameters above is the contrast definition rather than a conflict, since the main effect averages two conditions against two while the parameters compare a single condition against baseline. Although eye deviation was numerically the larger of the two effects (a ratio of approximately 1.6), the direct contrast between them was not significant, M = − 0.15, 95% CI [ − 0.47 , + 0.17 ], t ( 29 ) = − 0.95, p Holm = . 528. We therefore cannot conclude that eye deviation reduces humanlikeness more than hair deviation does. Familiarity had a significant effect, Wald χ 2 ( 1 ) = 6.96, p = . 008, with unfamiliar participants assigning higher humanlikeness ratings than familiar participants (M = 5.47 against M = 4.53). The gap was present in every deviation condition, at 0.67 scale points at baseline, 0.95 under hair deviation, 1.09 under eye deviation and 1.06 under dual deviation, and it is this consistency across cells that the Type III test summarizes. The corresponding parameter in Table 3 is not significant, B = − 0.07, p = . 891, because it estimates the familiarity difference within a single cell, that of Actor A at the unaltered baseline, where the gap is smallest and rests on nine familiar participants. The two values are not in conflict and are not alternatives between which a reader must choose; they answer different questions, as set out in Section 4.2.1. The effect of actor identity was not significant, Wald χ 2 ( 1 ) = 3.29, p = . 070, although actor identity interacted with feature deviation, Wald χ 2 ( 3 ) = 9.73 , p = . 021, indicating that the pattern across deviation conditions differed between the two reconstructions. The familiarity × deviation interaction was not significant, Wald χ 2 ( 3 ) = 4.18 , p = . 243.

4.2.3 Eeriness (RQ1, RQ3, RQ4)

Feature deviation significantly affected perceived eeriness, Wald χ 2 ( 3 ) = 29.85 , p < . 001 (Figure 5d). Relative to baseline, eye deviation raised the odds of higher eeriness ratings (OR = 2.70, 95% CI [ 1.28 , 5.72 ], p = . 009), as did dual deviation (OR = 3.95, 95% CI [ 1.41 , 11.01 ], p = . 009), while hair deviation alone was marginal (OR = 2.07, 95% CI [ 0.93 , 4.61 ], p = . 076). As with humanlikeness, the 2 × 2 GEE showed reliable main effects of hair deviation, Wald χ 2 ( 1 ) = 9.57, p = . 002, and eye deviation, Wald χ 2 ( 1 ) = 17.81, p < . 001, with no interaction, Wald χ 2 ( 1 ) = 1.12, p = . 289, OR = 0.75, 95% CI [ 0.44 , 1.27 ]. Both main effects were in the direction of increased eeriness under deviation. The within-subjects contrasts agree with all three tests (Table 2, Panel A). The direct eye-versus-hair contrast was again non-significant, M = + 0.20, 95% CI [ − 0.16 , + 0.56 ], t ( 29 ) = 1.14, p Holm = . 528, despite a numerical ratio of approximately 1.5 in the same direction as the other two outcomes. Familiarity had no significant effect on eeriness, Wald χ 2 ( 1 ) = 0.04 , p = . 833, nor did actor identity, Wald χ 2 ( 1 ) = 0.01 , p = . 941. Actor identity interacted with feature deviation, Wald χ 2 ( 3 ) = 12.96 , p = . 005; the familiarity × deviation interaction was not significant, Wald χ 2 ( 3 ) = 6.77 , p = . 080.

4.2.4 Component measures

The individual items comprising the acceptance composite, together with humanlikeness and eeriness, are shown in Figure 4b. Usage ratings decreased from baseline ( M = 7.40 , S D = 2.16 ) to dual deviation ( M = 5.80 , S D = 2.60 ). Humanlikeness followed the same downward trend, reaching its minimum under dual deviation ( M = 4.80 , S D = 1.63 ). Collapsed across actors, eeriness was highest under dual deviation ( M = 4.50 ), above eye deviation ( M = 4.27 ), hair deviation ( M = 4.07 ), and baseline ( M = 3.53 ). For Actor B considered alone, eye deviation and dual deviation produced indistinguishable eeriness ratings (M = 4.33 and M = 4.27 respectively; t ( 29 ) = − 0.26 , p = . 798).

5 Discussion

We examined how deviations in hair and eye color on look-alike avatars affect observer judgments of acceptance, humanlikeness, and eeriness, and how prior familiarity with the depicted person moderates those judgments. Below we take each research question in turn, then consider what the findings imply for design and where they stop.

5.1 Deviations combine additively (RQ2)

The clearest result concerns how deviations combine. Treating the four conditions as a crossed factorial rather than as four levels of a single factor permits a direct test of super-additivity, and that test was null on all three outcomes, with the interval for the interaction including the null value in each case, whether expressed as a contrast on the rating scale or as an odds ratio. The dual condition attracted the lowest acceptance and humanlikeness scores and the highest eeriness scores, but that ordering is exactly what two independent, same-signed effects produce, and is not in itself evidence of compounding. The observed dual-condition decrement in acceptance ( − 5.40 ) was not larger than strict additivity predicts ( − 5.69 ). This matters because the intuition that combined imperfections compound is easy to form from descriptive ordering alone. The additive result is the more useful finding for design purposes, because it means the cost of a deviation can be estimated feature by feature without needing to model interactions between features. It is also more tractable theoretically. Under an expectancy-violation account each altered attribute contributes its own discrepancy against the stored representation, and in this case those discrepancies summed rather than amplifying one another. We emphasize that this conclusion is established for two attributes of the same type, both surface color properties. Whether deviations of different kinds, for example a color change combined with a geometric or behavioral one, would also combine additively is untested here, and MacDorman and Chattopadhyay (2016) give reason to expect that inconsistency across levels of realism behaves differently from inconsistency within one.

5.2 Eye deviation is costlier, but only for acceptance (RQ1)

Both attributes mattered independently. Deviating either hair or eye color reduced acceptance and humanlikeness and raised eeriness. The two were not equal in magnitude. Eye deviation cost roughly twice what hair deviation cost on the acceptance composite, and this difference was statistically reliable. The same ordering appeared for humanlikeness and eeriness, at ratios of about 1.6 and 1.5 respectively, but neither direct contrast reached significance at this sample size. The asymmetry is therefore established for acceptance and suggestive but unconfirmed for the other two outcomes. Our design recommendations below rest principally on the acceptance result, which is also the outcome most directly tied to whether a digital double can serve as a usable stand-in for its referent.

Two explanations are available and the present design cannot separate them. The first is the account we set out in advance, following Takano and Taka (2022), in which attributes that can be altered in everyday life are accepted as belonging to the person, whereas attributes treated as fixed are not, so changing eye color departs from a stable identity marker in a way that lightening hair does not. This is consistent with users’ own behavior, preserving eye color while altering hair freely (Wu et al., 2023; Ducheneaut et al., 2009), and with the broader finding that focal facial features, particularly the ocular region, disproportionately shape perceptual evaluation (Seyama and Nagayama, 2007; Weibel et al., 2010; Kätsyri et al., 2015).

The second explanation arises from the stimuli themselves. As Table 1 shows, our two manipulations were not equivalent in kind. The eye deviation was principally a hue reversal, rotating from warm brown to cool slate-teal at near-constant lightness. The hair deviation was principally a lightness increase, with hue essentially unchanged within the warm brown-orange family. The two were comparable in overall colorimetric magnitude (Δ E a b * of 20–34), so eye deviation did not simply receive a larger change; it received a categorically different one, crossing a hue boundary that the hair change never crossed. The observed asymmetry may therefore reflect the type of color change rather than the identity status of the feature. These two accounts are difficult to separate, and we would argue that they are entangled in the world rather than only in our stimuli. Human hair color varies principally along a lightness axis within warm hues, and is altered in practice by lightening or darkening. Human iris color varies principally along a hue and saturation axis at near-constant lightness, and is altered in practice by shifting hue. A naturalistic deviation of each feature therefore takes the form we applied, and a design that crossed feature with type of change would have to introduce non-naturalistic stimuli, such as hue-shifted hair or lightness-shifted eyes, trading one confound for another. We nonetheless regard such a design as the most informative next step for identifying the mechanism. For the design guidance we offer below, the distinction matters less, since a practitioner altering eye color in any realistic way will be making a hue change, and our estimate of its cost applies whichever account is correct.

This also reconciles our results with Schwind et al. (2015), who found no significant effect of either hair or eye color on avatar likability or attractiveness. Those judgments concerned generic faces, where a color is an aesthetic choice and there is no referent it could contradict. Here the avatar is a look-alike of a specific person, and the same change becomes a discrepancy against a stored representation. The relevant distinction is not between colors but between customization-as-preference and customization-as-error, and it predicts that color should matter here and not there, which is what we observe.

A caveat on the display applies to all of the above. Optical see-through headsets have constrained color reproduction, brightness, and effective contrast against the physical background (Gabbard et al., 2010), and the HoloLens 2 renders dark, low-luminance regions weakly because black is displayed as transparency. The deviated eye color is dark and moderately desaturated, so it is plausible that its on-device appearance was closer to the brown baseline than the authored values imply, which would understate rather than overstate the eye effect. All conditions were presented under identical device and lighting settings, so this constraint applies equally across conditions. Participants also commented on hair and eye color spontaneously and unprompted, indicating that the manipulations were perceptible in situ. That is informal evidence and not a substitute for on-device photometric verification, which we did not perform.

5.3 Familiarity as a criterion shift rather than a change in slope (RQ3)

Familiarity affected acceptance and humanlikeness. Observers who knew the actors rated the avatars lower on both. Within the acceptance composite this effect is concentrated in the resemblance item rather than spread evenly across all three facets (Table 6), so it is best described as familiarity sharpening judgments of likeness specifically. This is what an expectancy-violation account predicts. Familiar observers hold a precise representation of the depicted person, accumulated over 8 weeks of multi-context interaction, against which a discrepancy is readily detected. Unfamiliar observers were not without a comparison, since every participant was shown photographs of both actors and could consult them at any point (Section 3.4), but a single photograph supports a much less precise expectation than personal acquaintance does (Jenkins et al., 2011). The contrast is therefore between a rich and a thin representation rather than between the presence and absence of one. This makes the comparison conservative, since unfamiliar observers were given a comparison aid and the familiarity difference emerged even so. It is consistent with Diel and Lewis (2022), who show that familiarity sharpens sensitivity to facial distortion by strengthening the observer’s norm for a known individual, and with Vazire and Carlson (2011), whose finding that observers judge others more accurately than people judge themselves predicts that familiar collaborators will be the harsher critics of a digital double.

The form this effect takes is more informative than its presence. Familiarity showed a main effect but no evidence of moderation. It did not interact with feature deviation overall, and testing it against each component of the factorial separately produced no differences in the size of the hair effect, the eye effect, or their interaction. The eye deviation effect was numerically identical in the two groups. We found no evidence that familiar observers became selectively more sensitive to particular deviations, though that test was underpowered and cannot exclude a difference. In the language of signal detection, the pattern is consistent with familiarity shifting the criterion rather than the slope, which we offer as the reading best supported by the data rather than as a demonstrated result. This distinction matters for design, since knowing the depicted person lowers the overall evaluation of a digital double, but it does not change which features are costly to alter. The ranking of features is stable across audiences even though the absolute standard is not. We note that with nine familiar participants these null interactions are underpowered, and we treat them as consistent with a criterion shift rather than as established evidence of one.

Familiarity had no detectable effect on eeriness. Familiar observers rated the avatars as less humanlike and less acceptable, yet no eerier. This dissociation is the most theoretically interesting result in the study, and it follows naturally from the expectancy account. Acceptance and humanlikeness are judgments of fidelity to a specific person, so they depend on how precisely the observer can represent that person, and familiarity sharpens that representation. Eeriness appears to track something else, plausibly generic artificiality arising from rendering, shading, or material properties, which is available to any observer regardless of whether they know the person depicted. On this reading familiarity does not make an imperfect likeness feel more unsettling; it makes the imperfection easier to see. The pattern is broadly consistent with Poletaykina et al. (2025), who tie familiarity to trust and social acceptance rather than to affective discomfort, and with Guo et al. (2024), who report that familiarity can bridge uncanny responses when behavior matches expectation.

Two qualifications are needed. First, eeriness was measured with a single item, so a null result is harder to interpret than a positive one, and we cannot exclude the possibility that a more sensitive multi-item instrument would detect an effect. Second, the dissociation is between two measures of differing psychometric quality, which weakens the inference that they index genuinely separate processes. Both points are taken up in Section 5.6.

These findings also raise important questions about the depth and nature of familiarity itself. In this study, familiar participants had intensive, multi-contextual exposure to the actors over an eight-week period, likely producing a high-resolution facial norm for those individuals. However, familiarity is not a monolithic construct. The quality of knowing a public figure from media differs substantially from knowing a close colleague or family member. Familiarity acquired through brief interactions may not produce the same degree of face-specific sensitivity as familiarity built through sustained personal contact. Future studies should therefore employ finer-grained measures of familiarity, ranging from mere recognition to close acquaintance, to better disentangle how different depths of face knowledge moderate avatar perception.

In practical terms, these findings suggest that while an unfamiliar participant pool can provide useful baseline insight into avatar design quality, familiar evaluators are better positioned to assess whether an avatar faithfully represents its subject. Both perspectives therefore serve distinct and complementary roles in avatar evaluation and validation workflows.

5.4 Differences between the two reconstructions (RQ4)

Acceptance was higher for Actor A than Actor B, and the decline across deviation conditions was steeper for Actor A, whose dual-deviation avatar also attracted the highest eeriness ratings. We report these differences but we do not attribute them to any property of the actors, and we want to be explicit about why.

With one exemplar per level, actor identity is fully confounded with everything that distinguishes the two specific reconstructions. Three such differences are documented in this study. The first is reconstruction quality, since 93.3% of participants rated Actor A’s baseline avatar at 6 or above for resemblance, against 66.7% for Actor B, and participants reported a visible artifact at Actor B’s hairline. The second is phenotype, since both actors have dark hair, dark eyes, and differing skin tone, and applying the same nominal light-blond and slate-teal targets to both produced deviations of unequal colorimetric magnitude, as Table 1 shows. Applying light hair and cool eyes to a darker-skinned actor also crosses a phenotypic boundary that the same manipulation does not cross for a lighter-skinned one, so the perceived incongruity of an identical nominal change is not equated across identities. The third difference is gender presentation. All three are perfectly correlated in our design and cannot be separated from one another.

We therefore treat RQ4 as answered in the negative, in that the differences we observe are differences between two particular reconstructions, and the design does not license attributing them to gender, phenotype, or fidelity individually. Zell et al. (2015) show that shading and material stylization interact with perceived realism in ways that can dominate other manipulations, which is a further reason to treat a two-exemplar comparison cautiously. Establishing whether phenotypic incongruity drives these effects requires a design with multiple identities per level and deviation magnitudes equated in perceptual rather than nominal terms, which we regard as a prerequisite for any claim in this area.

5.5 Implications for look-alike avatar design

These recommendations apply to a specific case that avatars intended to represent an identifiable person to observers who may know them. Where an avatar is not meant to depict anyone in particular, deviation is not error, and the considerations below do not apply. Prior work supports this scoping in several respects. Social VR users present themselves consistently with offline identity while also using avatars for exploration (Freeman and Maloney, 2021), users prefer avatars that resemble their remote collaborators (Pakanen et al., 2022), particularly in instructional contexts (Oyekoya and Baffour, 2025), and Ratan et al. (2022) recommend that students be encouraged toward actual rather than idealized self-representation. The common thread is that fidelity matters when a referent exists and is known to the audience.

Within that scope, three implications follow. First, the cost of deviation is separable by feature. Hair and eye deviations combined additively, so a creation tool can reasonably present their consequences independently rather than modeling interactions between them. Second, eye color is the more expensive attribute to alter, at roughly twice the cost of hair on acceptance. Systems might therefore offer wide latitude for hair color, which Ducheneaut et al. (2009) note users already want, while treating eye color as a fidelity-critical attribute, either by constraining the available range or by signaling to users that departures will be noticed by people who know them. We attach a caveat here because our eye manipulation was a hue rotation and our hair manipulation was a lightness shift, so this recommendation may generalize to the type of change rather than to the feature. Until that is resolved, the safer formulation is that departures crossing a hue boundary on a focal feature are costly.

Third, audience composition matters. Look-alike avatars in workplace and collaborative settings will predominantly be seen by people who know the person depicted, and those observers apply a stricter standard across the board. Evaluation practice should reflect this asymmetry, since an unfamiliar panel is adequate for assessing generic rendering quality, but only familiar evaluators can assess whether an avatar succeeds as a likeness, and our results indicate they will rate it lower than an unfamiliar panel would. The two panels answer different questions and are not substitutes.

Finally, a consideration that follows from the study’s premise rather than its results. Look-alike avatars depict identifiable people, and the same pipeline that produces a sanctioned digital double can produce an unsanctioned one. The capacity for impersonation and social engineering is a recognized concern in avatar systems (Bailenson, 2018), and our findings bear on it in an uncomfortable way. Unfamiliar observers rated deviated avatars more favorably than familiar observers did, which implies that an imperfect likeness is most persuasive precisely to those least equipped to detect the imposture. Creation tools for look-alike avatars should therefore treat consent and provenance as first-order design requirements rather than downstream policy questions.

5.6 Limitations and future work

The two manipulations differed in kind as well as in location, the eye deviation being predominantly a hue rotation and the hair deviation predominantly a lightness shift (Table 1). Feature and type of color change are therefore confounded, and the observed asymmetry may be attributable to either. Resolving this requires crossing feature with type of change, and we regard it as the most informative extension of this work. Relatedly, the same nominal targets applied to two actors of differing phenotype produced deviations of unequal colorimetric magnitude, so manipulation strength was not equated across identities. No perceptual pretest was conducted to confirm that the deviations were detected as intended, and no on-device photometric measurement was taken; the values we report are authoring-tool albedo and rendered image color, neither of which captures the appearance of the hologram against the participant’s physical background. The specific hair assets used were not recorded, which limits exact reproducibility.

The HoloLens 2 is an optical see-through display with constrained color reproduction, brightness, and effective contrast against the real-world background (Gabbard et al., 2010). Black is rendered as transparency, so dark low-luminance regions such as the deviated iris are displayed weakly and blend with the background. This most plausibly attenuates the eye manipulation, meaning our estimate of the eye effect may be conservative. All conditions were presented under identical device and lighting settings, so the constraint applies equally across conditions, but conclusions about color-based deviation should be replicated on higher-fidelity hardware. We also did not compare MR against other presentation formats, so we cannot claim that any of these effects are specific to mixed reality rather than general to viewing a look-alike avatar.

Humanlikeness and eeriness were each measured with a single item. Validated multi-item instruments exist for precisely these constructs (Ho and MacDorman, 2017), and their use would place the familiarity dissociation on considerably firmer ground, since that argument depends on a null result for one single-item measure alongside significant effects on others. The acceptance composite is defensible conceptually but not psychometrically. Its three items are distinct facets rather than interchangeable indicators, within-condition reliability was modest, and we report the items separately alongside the composite for this reason. No pre-study manipulation check confirmed participants’ familiarity, and familiarity itself was treated as binary, which does not capture depth of acquaintance. Knowing a colleague of 8 weeks, a family member, and a public figure are different states, and finer-grained measures would sharpen the account.

Participants could return to a previously viewed avatar and revise a rating. In practice this was rare, but it was not logged, so we cannot report its frequency, cannot analyze first-pass responses separately, and cannot include presentation order as a covariate. Where comparison did occur, ratings become relative rather than independent judgments, which would tend to widen differences between conditions. This bears on the magnitude of the main effects rather than on the additivity conclusion, since exaggerated spread does not produce a null interaction. Ratings were also given aloud to an experimenter rather than recorded privately. Both actors are authors of this paper and were the familiar participants’ program mentors, though neither administered the study and the experimenter did not appear in any stimulus. Familiar participants were therefore evaluating the likeness of people known to them within a research context, which could bias responses toward either leniency or criticality. Since familiarity is a between-subjects factor, this potential bias aligns with the effect of interest and cannot be separated from it. Private response collection is advisable in any replication.

Two actors is one exemplar per level of actor identity, which prevents attributing the differences between them to any single attribute (Section 5.4). Both had dark hair and dark eyes, so the findings do not generalize to other baseline colorings. The familiar group comprised nine participants, so all interactions involving familiarity are underpowered and the null moderation results should be read as uninformative rather than as evidence of absence; the a priori power analysis targeted the omnibus within-between interaction and not this contrast. Participants were 18–24 and predominantly students receiving course credit, and face-processing sensitivity, avatar experience, and prior MR exposure all plausibly vary with age and experience. Color vision deficiency was not screened, which is a material omission in this study, given a prevalence of roughly 8% for red-green deficiency among males and 18 male participants in the sample. Prior MR experience and participants’ own hair and eye color were not collected.

Avatars were static, with a neutral idle animation and no expression, speech, or interactivity, so ratings reflect appearance alone. Prior work indicates that behavioral realism interacts strongly with appearance, and that partial realism can be worse than none (Frampton-Clerk and Oyekoya, 2022; MacDorman and Chattopadhyay, 2016). Whether color deviations retain the additive structure observed here once animation, gaze, and speech are present is an open question, and one where the interaction we did not find might well appear.

6 Conclusion

We examined what it costs, in the eyes of an observer, to alter the hair and eye color of a look-alike avatar viewed as a life-sized hologram in mixed reality, and how that cost depends on whether the observer knows the person depicted. Deviations in hair and eye color combined additively. Altering both features produced the lowest acceptance and humanlikeness and the highest eeriness, but the effect was no greater than the sum of the two individual deviations, and the interaction was null on all three outcomes. We find no evidence that simultaneous deviations compound. For design, this is a convenient result, since the cost of altering a feature can be estimated independently of what else has been altered. The two features were not equally expensive. Eye deviation cost roughly twice what hair deviation cost on acceptance, a difference that was statistically reliable; the same ordering held for humanlikeness and eeriness at a smaller ratio that did not reach significance. We cannot yet say whether this reflects the identity status of the eye region or the fact that our eye manipulation rotated hue while our hair manipulation shifted lightness, and separating those two accounts is the clearest next step.

Familiarity lowered acceptance and humanlikeness but left eeriness unchanged, and we found no evidence that it altered the size of the penalty attached to any particular deviation, though with nine familiar participants that comparison was underpowered. The pattern is consistent with observers who knew the actors applying a uniformly stricter standard rather than a selectively sharper one, which we treat as an interpretation the data support rather than one they establish. We read this dissociation as indicating that acceptance and humanlikeness are judgments of fidelity to a specific person, and therefore depend on how precisely the observer can represent that person, whereas eeriness tracks generic artificiality available to any viewer. Familiarity does not make an imperfect likeness feel more unsettling; it makes the imperfection easier to see.

These conclusions are bounded, in that they rest on two identities, one of each gender and differing in phenotype and reconstruction quality, so the differences we observed between them cannot be attributed to any single attribute. They rest on single-item measures of humanlikeness and eeriness, on a familiar group of nine, and on an optical see-through display whose color reproduction most likely understates rather than overstates the eye effect. Within those bounds, the practical implication for anyone building digital doubles of real people is that fidelity in eye color earns more than fidelity in hair color on acceptance, where the direct contrast between the two is reliable, though not demonstrably so for humanlikeness or eeriness, where the same ordering appears numerically but does not reach significance. A second implication is that the audience most likely to encounter such an avatar in professional use, namely people who already know its subject, is also the audience that will judge it most strictly.

Generative AI statement

The author(s) declared that generative AI was used in the creation of this manuscript. The authors used Claude (Anthropic) during the preparation and revision of this manuscript. It was used for checking, spelling, grammar, and terminological consistency. It was also used to conduct the re-analysis required by the review, comprising the 2 × 2 factorial decomposition, the Holm-adjusted contrasts and pairwise comparisons, the within-condition reliability estimates, the item-level robustness checks, and the colorimetric measurements in Table 1. It also extracted and tabulated parameter estimates from the authors’ existing SPSS output and regenerated Figures 4, 5 from the raw data. All decisions regarding study design, data collection, analysis, theoretical framing, and interpretation were made by the authors. The authors verified every reported statistic against the source data and the original analysis output, checked all citations and quotations, and take full responsibility for the accuracy and integrity of the manuscript.

Any alternative text (alt text) provided alongside figures in this article has been generated by Frontiers with the support of artificial intelligence and reasonable efforts have been made to ensure accuracy, including review by the authors wherever possible. If you identify any issues, please contact us.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Footnotes

Edited by: Michael Wiesing, University of Barcelona, Spain

Reviewed by: Alan Douglas Fraser, Edith Cowan University, Australia

Sankar B, Indian Institute of Technology Delhi, India