The Customization500 study (see http://www.mc-500.com) provided us also a very detailed view into the current practice of choice navigation and online configuration. When crunching the numbers, we found a large puzzle:
The reality of toolkits clearly falls behind the broad body of academic research on design parameters of successful toolkits.
You find lots of additional information about the Customization 500 (short: MC500) study in a special section in my blog, www.mc-500.com
We used more than 40 parameters to evaluate the state and design features of each toolkit in our study (Table 3).
This objective data was then matched with the performance outcome of the toolkit, i.e. the perception of our trained experts of the configurator.
For this evaluation, we used five criteria to measure the performance of a configuration toolkit in our sample:
(1) visual realism, (2) usability, (3) creativity, (4) enjoyment and (5) uniqueness.
All impressions were measured by our panel of trained expert judges on a 1 to 5 rating scale (1=very low value / 5=very high value).
- Multiple items: For most of these criteria, we used multiple items to gather the scope of these criteria. Items were grouped and tested with the help of factor and reliability analysis.
- Inter-rater reliability of these factors was checked with the Intra- Class-Correlation-Coefficient.
- Performance: Finally, the single factors were combined to one overall performance factor.
In more detail, the five evaluation criteria were constructed as follows:
- Visual Realism was measured by means of one question “How realistic do you assess the visualization of the configuration process?”
- To measure usability, the evaluators were asked to rate: "The configurator is (1) intuitively usable, (2) user-friendly as well as (3) clearly and (4) logically structured".
- The creativity factor consisted of two items: "(1) The website gave me a lot of freedom", and "(2) I could give my creativity free rein while designing the product".
- To determine enjoyment, the experts had to rate: "The configuration was (1) fun, (2) delight, (3) pleasure, (4) entertaining and (5) interesting".
- To assess uniqueness following statements had to be evaluated: "My created product (1) is unique, (2) is different, (3) helps me to differentiate; and (4) no one else has such a product."
In addition, we had multiple other scales for performance like the satisfaction with the final product, the willingness to purchase, the likelihood to recommend the vendor to a friend, etc.
All 500 companies that are included in the Customization500 are presented with a profile picture like the following Figure 4.
Figure 4: Sample profile picture (purchase the full report for all 500 evaluations)
Our analysis showed that on the one hand side, “preference fit” and the meeting of a customer’s “need for uniqueness” are strong drivers of satisfaction with a particular mass customization offering.
However, process satisfaction, resulting from the enjoyment and creative involvement during a user-friendly configuration process, has an even higher impact in many cases, according to our data.
This confirms the early findings of researchers like Nikolaus Franke or Martin Schreier who have stated that in B2C mass customization about 50 percent of the additional willingness to pay can be explained by the process experience and a feeling of achievement and co-design success – and not by the higher functionality of fit of a custom product.
Our study clearly supports this claim. We urge managers to look beyond the sheer technology and back office integration of configuration toolkits and also focus on delivering a great configuration experience.
Academic research often has stressed the importance of realistic visualization as a core element of a good toolkit. But many companies in practice still have very simple visualization features, and sometimes no illustration of the outcome at all. But there also can be too much of a good thing:
Evaluators often highlighted not those sites with the most advanced 3D visualization as best in class, But those with visualization features that matter and come to the point.
For many products, a realistic, fast, plug-in free and well-described visual of an individual configuration is better than a complicated 3D model wearing, for example, my custom T-shirt – that takes many seconds to load and almost crushed my computer when playing with it. Sounds obvious? Well, it is not. We still see many sites where technology is used as a point of differentiation – but not as a source of customer value!
When looking at the data which features of a configurator drive most the perceived usability and use experience, we found that navigation- and orientation-help features, such as a progress bar or an activity list, play a key role. Co-design toolkits with a higher level of company- and/or customer-help features, such as design inspirations, deeper product information or recommendations by other consumers, in general performed better in terms of satisfaction.
However, about 50% of the toolkits in the "Customization 500" do not offer any or only a low level of these features. Here, we find many untapped opportunities for practice to enhance the gross utility of customers.
At the other side, we’ve found quite a few offerings without or with only a low level of these features which were performing excellently nonetheless. In some cases, the simple product design (solution space) did not require special help features. In other cases, customer satisfaction with the offerings was excessively influenced by the particular value provided by the customizable product itself.
There is not one best way. Companies should “customize their mass customization strategy” based on the requirements of their customer stock. But having an understanding of the perception of customers or a firm's toolkit is crucial to make such a decision.
The largest gap between practice and recommendations of academic research can be found in the area of parameter- versus need-based toolkits. We found that in today's mass customization reality, basic parameter (option) based toolkits still rule. Customers have to make their own decisions from a list of predefined options. This often demands a large number of decisions and also knowledge of the user about the product. While this may be perfect in the business-to-business context, it is not always the best option in consumer markets.
Here, need-based configuration has been shown to provide better results. In such a need-based system, users share something about their preferences, requirements, or expected outcomes. This input then is transferred by an algorithm into a product configuration. A need-based configurator hence mimics the behavior of a good sales person who also may recommend you exactly the right product (configuration) after asking just a few but insightful questions. In our study of the best 500 toolkits, less than 3% of companies had such a need-based configurator in place. While we acknowledge that it is more costly to develop a good need-based configurator, these systems seem to offer a great opportunity for differentiation and larger customer satisfaction.
Concluding, we can state that mass customization still is an area in the making. While there has been much progress, and there are some really great toolkits in the market, the majority of systems still are in an early stage. But as our data shows, from the customer perspective it often is just a small step between a good to a great toolkit.
All companies that have been included in the Customization500 received this vignette to illustrate that they are part of the leading companies in the field of mass customization and personalization. So when search for customization on the web the next time, watch out for this sign.
But we expect that the Customization500 is a very dynamic field. Even during the time of our research we found many developments, improvements, and failures. This is why studying the field of mass customization remains a continuous endeavor … but a fun one, too!
Part 1: of this series: Data on the MC industry structure
www.mc-500.com: More information on the Customization500 study and a list of the 500 companies included in the evaluation.
http://bit.ly/mcpc-proceedings: The Proceedings of the last MCPC conference cover many dozens of case studies, latest research, 2500+ slides, and 15+ hours of video of the plenary presentations. As part of the proceedings, you also find three detailed PPT presentations using the Customization500 data in larger detail.
www.mc2012.org: Speaking German? Then participate at the next meeting of the German-speaking mass customization community (Salzburg, 29 June 2012) – and learn from some of the German champions of the Customization500.