MERWYN
Technology
Merwyn Technology is the decision support technology that powers the truth teller concept scores and sales forecasts. Parts of the technology is also used to power insight mining, idea coach, idea scan and rapid testing.
In it’s full deployment as part of truth teller the system forecasts the true odds of sales success for multiple sales, marketing and development scenarios. It does this by modeling both the “most likely” forecasting assumptions AND the uncertainty of each assumption at the current moment in time.
Development Team Pedigree: Merwyn Technology was created by the Merwyn Research team. at They have over 30 years of experience forecasting odds of success for over 20,000 innovations.
Merwyn Research clients include such corporate leaders as: Procter & Gamble, Nike, Walt Disney, Hewlett Packard, Schlumberger, American Express, Ford Motor Company and Johnson & Johnson and the US Department of Commerce.
The Merwyn Research team is lead by Doug Hall, a Chemical Engineer by education who spent ten years at Procter & Gamble where he rose to the rank of Master Marketing Inventor. At P&G Doug led a small team that shipped a record 9 innovations in 12 months. The original rapid test system was implemented at P&G compressing the time for fielding concept and use tests from 3 months to 7 days. Note, today it’s 24 hours. Doug was the founder and inventor of AcuPOLL precision research and the the Merwyn technology.
Forecasting Methodology: Merwyn technology is a specialized version of the Fourt-Woodlock system of sales forecasting. It has been modified to make it effective early in development where there is significant uncertainty and to make it effective for both industrial and consumer products as well as services. (For academic details see: Fourt L.A., Woodlock J.W., 1960. Early prediction of market success for new grocery products. Journal of Marketing 25: 31–38.) Fourt Woodlock is also the basis of AC Nielsen/BASES forecasting methodology that holds an estimated 60% share of all innovation sales forecasting by large companies.
Fourt-Woodlock forecasting is based on the class of forecasting known as Decomposition. The bible of forecasting Principles of Forecasting edited by J. Scott Armstrong of the Wharton School and published by Kluwer Academic Publishers defines Decomposition as: “Decomposition is a method for breaking down (decomposing) the estimation tasks into a set of components that can be more readily estimated, and then combining the component estimates to produce a target estimate.”
The book documents the effectiveness of Decomposition especially when there is high uncertainty as exists with innovations.It states, “The research evidence in support of this general principle is enormous and cannot be adequately covered here. Over four decades of research in human judgment and decision making show that decomposition improves judgmental performance over unaided or holistic judgment.”
In it’s full deployment as part of truth teller the system forecasts the true odds of sales success for multiple sales, marketing and development scenarios. It does this by modeling both the “most likely” forecasting assumptions AND the uncertainty of each assumption at the current moment in time.
Development Team Pedigree: Merwyn Technology was created by the Merwyn Research team. at They have over 30 years of experience forecasting odds of success for over 20,000 innovations.
Merwyn Research clients include such corporate leaders as: Procter & Gamble, Nike, Walt Disney, Hewlett Packard, Schlumberger, American Express, Ford Motor Company and Johnson & Johnson and the US Department of Commerce.
The Merwyn Research team is lead by Doug Hall, a Chemical Engineer by education who spent ten years at Procter & Gamble where he rose to the rank of Master Marketing Inventor. At P&G Doug led a small team that shipped a record 9 innovations in 12 months. The original rapid test system was implemented at P&G compressing the time for fielding concept and use tests from 3 months to 7 days. Note, today it’s 24 hours. Doug was the founder and inventor of AcuPOLL precision research and the the Merwyn technology.
Forecasting Methodology: Merwyn technology is a specialized version of the Fourt-Woodlock system of sales forecasting. It has been modified to make it effective early in development where there is significant uncertainty and to make it effective for both industrial and consumer products as well as services. (For academic details see: Fourt L.A., Woodlock J.W., 1960. Early prediction of market success for new grocery products. Journal of Marketing 25: 31–38.) Fourt Woodlock is also the basis of AC Nielsen/BASES forecasting methodology that holds an estimated 60% share of all innovation sales forecasting by large companies.
Fourt-Woodlock forecasting is based on the class of forecasting known as Decomposition. The bible of forecasting Principles of Forecasting edited by J. Scott Armstrong of the Wharton School and published by Kluwer Academic Publishers defines Decomposition as: “Decomposition is a method for breaking down (decomposing) the estimation tasks into a set of components that can be more readily estimated, and then combining the component estimates to produce a target estimate.”
The book documents the effectiveness of Decomposition especially when there is high uncertainty as exists with innovations.It states, “The research evidence in support of this general principle is enormous and cannot be adequately covered here. Over four decades of research in human judgment and decision making show that decomposition improves judgmental performance over unaided or holistic judgment.”
Six Factors that Drive The Intelligence of Merwyn Technology
#1. Improved reliability by segmenting data into 3 streams and processing each differently.
A cornerstone of Merwyn methodology is to separate data forecasting inputs into three categories and to handle inputting of the data and processing of data differently for each data stream.
In the case of sales forecasting the primary unknowable is Trial Rate. Specifically, the level of Marketing Support (awareness & distribution) as well as the concept persuasion or “pull”. Errors with these numbers from small company innovators and independent inventors are usually a result of ignorance in the quantification of marketing support and persuasion impact. Errors with Fortune 500 Clients are a result of over optimism especially early in the process when actual investment levels are highly variable due to lack of definition of actual marketing message impact, media plan design and efficiencies. This over optimism is generally accepted as the reason for the 25 to 80% error between sales forecasts at the moment of innovation development decisions and actual marketplace results. Obviously, all forecasts are highly accurate after the fact, when the true input data is known.
Instead of inputting marketing, advertising and promotion plans and then translating the plans into effective awareness and distribution impact levels - the Truth Teller system starts with industry standard stock assumptions for distribution and awareness. These are then optimized based on forensic analysis of past results.
Merwyn Concept Testing evaluates ideas by benchmarking versus 50 +/- success factors are evaluated. Each have been validated as predictive of marketplace results with direct links to marketplace success. A set of 5 raters as well as a “computer reading” evaluate each factor relative to predetermined benchmarks. Each rater scores only the presence or absence of each success factor. The Merwyn computer model assigns the weight and importance to each factor to generate an overall probability score. The Merwyn score is translated into a trial rate that is adjusted downward based on the assumed level of distribution and awareness. Research indicates that Merwyn Concept Testing is 7 times smarter than project leaders when evaluating concepts probability of success as it has no emotional connection or bias.
Research has validated Merwyn Concept scores versus AcuPOLL, BASES, Direct Mail Response Tests and other test systems in the USA and Internationally. It’s important to note, that Merwyn is an overall measure of the odds of sustained success - it is NOT a simple measure of purchase intent. The closest analogy to the Merwyn concept score is called “Meaningful Difference.” This is most closely replicated in classic concept research through a blended score that incorporates 60% Purchase Intent and 40% New and Different perception.
Comparison of truth teller Concept results to high precision Information Resources scanner data finds significant correlations with % trial, purchase frequency, percent repeat purchase and $/1000 customers. Impressively, these correlations are raw data to raw data i.e. they are not corrected for differences in marketing support and results.
To further improve the truthfulness of Merwyn Concept Testing both the “most likely” value and the variance (standard deviation) are used in the Merwyn Business Simulation to reflect the true dynamics associated with the innovation concepts persuasive power.
#2. Calibration of Input Estimates to Provide Marketplace Grounding.
Calibration data charts help the person providing the inputs assess, evaluate and determine the proper estimates. Calibration benchmarks include hundreds of listings of the number of final decision makers, number of additional repeats, EBIT percentages, repeat rates, proprietary protection and development status. In the case of custom corporate models calibration data is based on the company’s actual historical results.
#3. Estimates of most likely values and confidence ranges.
To model the true uncertainty of each data input three numbers area requested for each input.: 1) Most Likely Estimate, 2) Pessimistic Estimate and 3) Optimistic Estimate.
The three numbers are translated into a probability distribution. This natural mapping of the true uncertainty into dynamic forecasts is one of the reasons why Merwyn Business Simulation is significantly more truthful than point estimate forecasting systems. It’s important to note that Truth Teller is designed to be appropriately conservative. As such, the bias in the case of all modeling is towards protecting from “false positives” e.g. overly high estimates. Net, the Truth Teller system appropriately “rounds down” key dimensions.
#4. Detailing of Data Sources and Basis for Assumptions To Increase Transparency
To further enhance credibility of forecasts, innovators are required to document their data sources and basis for assumptions for each forecasting input. This complete transparency allows those viewing the report to quickly and easily asses the wisdom and integrity of the innovator’s assumptions
#5. Monte-Carlo Simulation of Inputs More Closely Simulates the Real World
All input data is processed using a Monte-Carlo simulation. In simple terms, Monte Carlo involves randomly selecting a value from a probability distribution for each variable in the sales forecasting equation. The values are randomly distributed across the range of possible values for each variable. The random values are then inserted into the volume equation and a sales forecast generated. The process is completed some 10,000 times for each marketing support and alternative development scenario (improved product, marketing concept or overall improvement of product and marketing concept).
The big benefit of Monte-Carlo simulation is that it takes into account the true individual and cumulative uncertainty associated with the forecast. In effect, Truth Teller translates innovation uncertainty into defined and quantified business probabilities.
6. Probability Matrix Report Format Aids Decision Making
To aid decision-making, the uncertainty of all inputs is translated into a report that details three estimates that together incorporate the variance of inputs and more closely model the real risk and uncertainty associated with the innovation.
Results from the 10,000 Monte-Carlo simulations for each level of marketing support are rank ordered and the 2,000th value from the bottom reported as the pessimistic estimate. This means there is an 80% chance that the actual sales for this innovation will be at least this amount. The median or 5,000th value is reported as the Most Likely (50%) value. The 8,000th value is reported as the optimistic forecast meaning there is a 20% chance that the true value will be higher than this.
Direct Questions to
Doug Hall 513 310-6374
or Doug@EurekaRanch.com
#1. Improved reliability by segmenting data into 3 streams and processing each differently.
A cornerstone of Merwyn methodology is to separate data forecasting inputs into three categories and to handle inputting of the data and processing of data differently for each data stream.
- Facts: Factual inputs are easy to input and use. Examples include Innovation Development Status and Proprietary Protection Level which are used when forecasting fair market royalty rates.
- Estimates: Estimates require four handling steps to aid accuracy and transparency: 1) Calibration, 2) Estimates of most likely values and confidence ranges, 3) Quantification of Uncertainty and 4) Documentation of Basis for Estimate. Each of these are the basis for the next four points of difference.
- Unknowables: Unknowable inputs are not likely to be correct even when the four special handling steps are applied by Fortune 500, small company innovators or independent inventors. To compensate for this, each unknowable is inputted into the Truth Teller model as a constant and reported as such.
In the case of sales forecasting the primary unknowable is Trial Rate. Specifically, the level of Marketing Support (awareness & distribution) as well as the concept persuasion or “pull”. Errors with these numbers from small company innovators and independent inventors are usually a result of ignorance in the quantification of marketing support and persuasion impact. Errors with Fortune 500 Clients are a result of over optimism especially early in the process when actual investment levels are highly variable due to lack of definition of actual marketing message impact, media plan design and efficiencies. This over optimism is generally accepted as the reason for the 25 to 80% error between sales forecasts at the moment of innovation development decisions and actual marketplace results. Obviously, all forecasts are highly accurate after the fact, when the true input data is known.
Instead of inputting marketing, advertising and promotion plans and then translating the plans into effective awareness and distribution impact levels - the Truth Teller system starts with industry standard stock assumptions for distribution and awareness. These are then optimized based on forensic analysis of past results.
Merwyn Concept Testing evaluates ideas by benchmarking versus 50 +/- success factors are evaluated. Each have been validated as predictive of marketplace results with direct links to marketplace success. A set of 5 raters as well as a “computer reading” evaluate each factor relative to predetermined benchmarks. Each rater scores only the presence or absence of each success factor. The Merwyn computer model assigns the weight and importance to each factor to generate an overall probability score. The Merwyn score is translated into a trial rate that is adjusted downward based on the assumed level of distribution and awareness. Research indicates that Merwyn Concept Testing is 7 times smarter than project leaders when evaluating concepts probability of success as it has no emotional connection or bias.
Research has validated Merwyn Concept scores versus AcuPOLL, BASES, Direct Mail Response Tests and other test systems in the USA and Internationally. It’s important to note, that Merwyn is an overall measure of the odds of sustained success - it is NOT a simple measure of purchase intent. The closest analogy to the Merwyn concept score is called “Meaningful Difference.” This is most closely replicated in classic concept research through a blended score that incorporates 60% Purchase Intent and 40% New and Different perception.
Comparison of truth teller Concept results to high precision Information Resources scanner data finds significant correlations with % trial, purchase frequency, percent repeat purchase and $/1000 customers. Impressively, these correlations are raw data to raw data i.e. they are not corrected for differences in marketing support and results.
To further improve the truthfulness of Merwyn Concept Testing both the “most likely” value and the variance (standard deviation) are used in the Merwyn Business Simulation to reflect the true dynamics associated with the innovation concepts persuasive power.
#2. Calibration of Input Estimates to Provide Marketplace Grounding.
Calibration data charts help the person providing the inputs assess, evaluate and determine the proper estimates. Calibration benchmarks include hundreds of listings of the number of final decision makers, number of additional repeats, EBIT percentages, repeat rates, proprietary protection and development status. In the case of custom corporate models calibration data is based on the company’s actual historical results.
#3. Estimates of most likely values and confidence ranges.
To model the true uncertainty of each data input three numbers area requested for each input.: 1) Most Likely Estimate, 2) Pessimistic Estimate and 3) Optimistic Estimate.
- Most Likely: This is the value that based on testing, experience and judgment is the most likely value the innovation will achieve.
- Pessimistic: This is the value where there is only a 1 in 5 chance (20%) that the true number is WORSE than this number. Note, this is not the “worst” possible value. Rather, it’s the mostly likely lowest value. In the case that an inventor has “measured data” on the question being asked they are instructed to provide the low end of one standard deviation.
- Optimistic: This is the value where there is only a 1 in 5 chance (20%) that the true number is BETTER than this number. Note, this is not the “best” possible value. Rather, it’s the mostly likely highest value. In the case that an inventor has “measured data” on the question being asked they are instructed to provide the high end of one standard deviation.
The three numbers are translated into a probability distribution. This natural mapping of the true uncertainty into dynamic forecasts is one of the reasons why Merwyn Business Simulation is significantly more truthful than point estimate forecasting systems. It’s important to note that Truth Teller is designed to be appropriately conservative. As such, the bias in the case of all modeling is towards protecting from “false positives” e.g. overly high estimates. Net, the Truth Teller system appropriately “rounds down” key dimensions.
#4. Detailing of Data Sources and Basis for Assumptions To Increase Transparency
To further enhance credibility of forecasts, innovators are required to document their data sources and basis for assumptions for each forecasting input. This complete transparency allows those viewing the report to quickly and easily asses the wisdom and integrity of the innovator’s assumptions
#5. Monte-Carlo Simulation of Inputs More Closely Simulates the Real World
All input data is processed using a Monte-Carlo simulation. In simple terms, Monte Carlo involves randomly selecting a value from a probability distribution for each variable in the sales forecasting equation. The values are randomly distributed across the range of possible values for each variable. The random values are then inserted into the volume equation and a sales forecast generated. The process is completed some 10,000 times for each marketing support and alternative development scenario (improved product, marketing concept or overall improvement of product and marketing concept).
The big benefit of Monte-Carlo simulation is that it takes into account the true individual and cumulative uncertainty associated with the forecast. In effect, Truth Teller translates innovation uncertainty into defined and quantified business probabilities.
6. Probability Matrix Report Format Aids Decision Making
To aid decision-making, the uncertainty of all inputs is translated into a report that details three estimates that together incorporate the variance of inputs and more closely model the real risk and uncertainty associated with the innovation.
Results from the 10,000 Monte-Carlo simulations for each level of marketing support are rank ordered and the 2,000th value from the bottom reported as the pessimistic estimate. This means there is an 80% chance that the actual sales for this innovation will be at least this amount. The median or 5,000th value is reported as the Most Likely (50%) value. The 8,000th value is reported as the optimistic forecast meaning there is a 20% chance that the true value will be higher than this.
Direct Questions to
Doug Hall 513 310-6374
or Doug@EurekaRanch.com
