Open Research Issues


Summary of Poll on various topics in Decision Based Design

From September 2003 to June 2004 participants were asked to give their opinion on various "hot" topics in the DBD community. There was a solid response to all the topics and some interesting comments from the community. The topic results are shown graphically below and a link to the comments from each topic is provided.

The first topic dealt with the role of optimization in decision making. Of the 35 responses, the majority (51%) feel that optimization may be helpful on formulating and solving some decisions, but most decisions do not fit the mold of a standard optimization formulation. Many (29%) however, believe that any decision in a normative design process can be formulated as an optimization problem and solved, resulting in an optimal decision solution. Only a few (6%) feel that decision making, if performed correctly using the axioms of utility theory and principles of decision theory, does not need any kind of formal optimization formulation or solution. The remaining respondents (14%) have other views.

The second topic sought opinion on the way in which uncertainty should be handled in DBD. Here the majority (42%) believe that decision making under uncertainty involves the study of the impact of various types of uncertainty on system performance (i.e., uncertainty propagation) and as such, methods need to be developed to integrate various types of uncertainty in uncertainty propagation. Others (29%) feel that the probablistic measure of uncertainty is the only quantification that is consistent with utility theory. It is the only representation that should be used in a rigorous dicision-based design framework. Some (13%) have the opinion that the various types of uncertainty cannot be integrated since the different methods for uncertainty quantification follow very different decision making frameworks. While a few (6%) believe that due to the different nature of uncertainties in engineering design, different methods (such as fuzzy sets, probablistic distributions, and interval methods) should be used to model and quantify each different type of uncertainty. The remaining respondents (10%) have other views.

The final topic was an open ended question for the research community to respond to. Comments from all topics can be viewed here.


Summary of Poll on Meeting Customer Needs in Engineering Design

Compared to the previous questions for debate posted at our workshop website, we received fewer responses (11 on average) to our recent poll of the views on meeting customers' needs in engineering design. The first question deals with whether the primal goal of design is to meet the customers' needs or to make profit. The responses are split (40% to 40%) between the view that "Meeting customer satisfaction is the primary goal of design decision making. A product will eventually become profitable if it is of good quality and customers like it" and the view that "The primary goal of design decision making is not to meet customer satisfaction, instead it is to make profit. Costs associated with improving a quality feature that customers desire must be considered. Designers may decide not to improve a quality feature if it does not lead to profit". The rest 20% choose other views.

The second question is on how to capture the preference of a group of customers. Only 8% of the respondents think that "A group of customers' preferences can be captured by a value function that represents the aggregated preference as a function of multiple product attributes". The rest disagree with the use of multiattribute value function. Among them, 58% think that "A group of customers' preferences cannot be aggregated and captured by a value function. Market share (or demand) is the most appropriate measure of how much a group of customers like the product". The rest 33% hold other views.

The last question is on how to meet both the needs of producer and customers in engineering design. We received divided views on this topic. 29% of the respondents think that "Engineering design is a tradeoff between meeting the needs of customers and that of the producer. This tradeoff can be made through multiattribute utility analysis and modeling the customers' preference and the producer's preference by two separate utility measures"; 43% hold the view that "The design utility is solely determined based on the producer's preference in engineering design. There is no such "tradeoff" between the customers' preference and the producer's preference in the utility function. Customers' interests (or desires) are captured through other means such as the product demand"; the rest 29% chose "other views".

In summary, the overall consensus gained from this survey is that meeting customers' needs is important in engineering design. A large percentage of respondents support the view that multiattribute value function cannot be directly used to capture the preference of a group of customers. However, views are divided on how to simultaneously model the customers' needs and the producer's preference. Close to a third of the respondents seem to hold views that are different from those we listed.


Summary of Poll on Role of Game Theory in Design

From July 2001 to March 2002, the DBD workshop home page posed a series of views on the applicability of game theory for decision making in engineering design. We received on average 95 responses to each of the four views posed. A summary of polls is provided in the figure.
In summary, the overall consensus gained from this survey is that the game theory can be applied to decision making in engineering design. A larger percentage of respondents support the view that game theory is applicable for design situations that involve different companies compared to those supporting the view that it is applicable to any design situation whenever multiple designers are involved. The respondents expressed diverse views on whether engineering design should be profit-driven or should be performance and quality based.


Summary of Poll on Single vs. Multi-Criteria Decision Making Approaches

The Decision Based Design Workshop's home page posed a series of questions on multi-criteria decision making approaches versus single-criterion approach in engineering design. We received 115 responses to the statement that "Existing multi-criteria decision making approaches are flawed. Only single-criterion approaches (such as maximization of the profit) should be used for product design." About 87% of respondents disagreed with that statement, only 7% of respondents agreed with the statement, while the rest were neutral. Significantly fewer responses were submitted to the five supporting views that further survey a respondent's position. It appears that the respondents only picked one of the five views as the one that they support instead of expressing their positions on each of the support views. Over all the responses, 12 respondents expressed their support of the view that existing multi-criteria decision making approaches are flawed. 9 respondents supported the view that multi-criteria decision making approaches have their limitations but still can by used if they are exercised properly and the assumptions are satisfied. 7 respondents agreed with the view that the paradoxes associated with multi-criteria decision making approaches only happen in very limited situations. About 8 respondents agree that the use of a single-criterion approach in product design is not practical.

The overall consensus gained from this survey is that the multi-criteria decision making approaches should still play an important role in engineering design even though they have limitations. Due to the limited number of responses received for the supporting views, we are not able to draw any further conclusions.


Summary of Poll on Attitudes Toward QFD in Design

Recent discussion of the Open Workshop on Decision-Based Design has focused on the use of popular methods in design. During our most recent face-to-face meeting of the Open Work we began a discussion of the usefulness and appropriateness of methods for design. Quality Function Deployment (QFD) is a visual technique for structuring a design process. QFD places emphasis on an aggregate interpretation of the "voice of the customer." The focus on identifying aggregate preferences calls the method into question. To explore attitudes of our website visitors toward QFD, we posted a series of questions about the method.

We asked the question, "Is QFD a useful approach in Engineering design?" A summary of the current 54 responses is as follows: 59% agreed that QFD is useful in the design process; 19% disagreed with 4% strongly disagreeing; and, 22% were neutral to the statement. Five follow-up questions tried to determine each respondent's rationale for their opinion. What we found from the smaller sample of responses to these questions is that there is a core group of researchers who object to QFD on the grounds of the mathematical flaw in the method. Most respondents were neutral to QFD's use in industry.


Summary of Poll on Attitudes Toward The Role of Decision Analysis in Engineering Design

The Decision Based Design Workshop's home page posed a series of questions on The Role of Decision Analysis in Engineering Design. We received 74 responses to the statement that "Decision analysis is the most important aspect of design." Over 60% of respondents agreed with that statement. This is not surprising since we all agree that design can't move to completion without analysis and the making of choices. A minority of 21% of respondents view decision analysis as a much less important facet of design. The responses to the support views yield a little insight. (We must be mindful that many fewer responses are being submitted to these questions.) A clear majority of respondents are willing to accept that design is both art and science, implying that it's not all "analysis", and that decision analysis brings both benefits and limitations in application to the design process. There is also a sense that the community is not afraid of using quantitative methods in this hybrid (art and science) activity of design.

We could not gain any consensus on two other issues. Respondents were more neutral to claiming that all design activities must be constructed as decision-making. At the same time, no consensus was present in the reaction to the statement that some design process activities are not "decision-making."