The Power of Near-Real Time Food Trend Volatility Analysis

Is your food company busy addressing rising Vegan, Vegetarian, DASH, Anti-Inflammatory, ABS or Paleo diet trends today? If not, why not? These are among today’s top-10 diet trends driving consumer food choices.

Although Internet and social media analysis techniques are still in their infancy, these “big data” bases should not ignored: they best reflect what consumers think and say about their food choices. Think of the Internet as a very large consumer survey population that can be sampled at-will, at very low cost and on a near-real time basis. Internet chatter analysis also reveals trends not easily discernable using conventional analysis techniques, such as retail product scanning, new product placement tracking and consumer surveys…in near-real time! There is the problem of information clutter, however: the Internet has a very low signal-to-noise ratio and it can often be difficult to discern between what is important or relevant and what is not. Also, Internet search engines are fickle and just finding the most applicable search terms can be challenging.

This post references work undertaken at BEST VANTAGE Inc. ( to establish new tools for consumer trends analysis, drawing on techniques developed by the financial industry. Previous work undertaken on this challenge is referenced here and here. In this posting, we demonstrate how volatility analysis, using our VIC™ internet chatter volatility indices, can rapidly prioritize emergent trends not readily detectable using conventional market analysis tools. The earlier warned, the faster that companies can adapt to and capture the high ground of new consumer opportunities.

Why volatility? Volatility is a leading indicator of change. Whether in nature, societies, economies or financial markets, “volatility” marks rapid exchanges of material and information that signal impending change. Internet chatter surges or wanes as individuals adapt to new information and adjust their demands and expectations accordingly. Internet chatter volatility denotes activity and information exchange: it does not explain the underlying reasons for change, which requires a more forensic analysis of the Internet database. Thus, a surge in Internet chatter signals that change is pending and that a more in-depth analysis of the underlying reasons for volatility is likely warranted.

In the chart below, the VIC™ volatilities of the top 9 diet trends (out of 40 analyzed) are presented together. It is clear that, already in late-2009 (six years ago), interest in vegetarianism surged, followed by a surge in vegan diet-related chatter beginning in 2013. These are the markers that should have signaled to the processed food, foodservice and food ingredient companies to closely track these diet trends and adjust their product lines and strategic plans accordingly. This period (2009 – present) also exhibited very significant spikes in Internet chatter volatility pertaining to high-protein, low-carb Paleo and ABS diet-related Internet chatter.

Top-9 Trend Volatility

A look at annualized growth trends in Internet chatter suggests how rapidly these four trends will remake our industry. Between 2013 and 2014 alone, BEST VANTAGE observed the following growth rates (i.e., velocity) in Internet chatter, presented along with 5-year annualized growth rates as benchmarks.

  • Vegan Diet (1-yr: 590%; 5-yr Annualized Growth Rate: 89%)
  • ABS Diet (1-yr: 315%; 5-yr Annualized Growth Rate: 115%)
  • Paleo Diet (1-yr: 159%; 5-yr Annualized Growth Rate: 101%)
  • Vegetarian Diet (1-yr: 152%; 5-yr Annualized Growth Rate: 85%)

Internet volatility and velocity analysis should not be used in place of conventional market tracking techniques. They do offer powerful early indicators of emergent trends, helping companies to know where to look and how to respond to the most volatile index of all, consumer behavior. In a future posting, I will explain the value of using Internet chatter volatility and velocity analyses as strategic decision-making tools.

Daniel Best (BEST VANTAGE Inc.) to speak at April 2014 “Proteins Trends and Technologies” Seminar.

Daniel Best will speak at the Global Food Forum’s 2nd Annual “Protein Trends and Technologies Seminar” (April 8-9, in Arlington Heights, IL) on the topic of how to establish practical working criteria in choosing between different protein ingredient options.

The food industry will soon face a plethora of different protein ingredient selections, even as global demands for protein are soaring. Best’s presentation, “Sooo many proteins, so little time: how to choose” is designed to help food, beverage and nutritional industry professionals make clear-eyed choices based not just on protein functionality, but business and legal considerations as well.

The first day (April 8th) of the two-day Protein Trends and Technologies Seminar, “Strategic Insights for Business Growth”, will emphasize marketing, business and regulatory aspects of the surging protein ingredients industry. The second day, “Formulating with Proteins”, will focus on the technical and quality challenges of incorporating protein ingredients into finished food, beverage and nutritional products. For more information on this conference and how to register, please visit:

Note: This 2nd Annual Seminar continues 2013’s highly acclaimed “Proteins” seminar hosted by Global Food Forums, a complete Conference Report on which can be downloaded from this Link:

Pulses in foods…summarized.

BEST VANTAGE Inc.’s fourth and final “How To:” Pulse WEBINAR, sponsored by the USA Dry Pea & Lentil Council through a grant conferred by the Idaho State Department of Agriculture, was presented November 7th, with a record number of attendees. All four webinars in this series will be available for review in .pdf and/or presentation format in the near future at the link provided at the end of this post…so please stay posted.RandDTrends3

Although the focus of this webinar was on extruded products, ranging from pasta to puffed snacks, the presentation also reviewed previously addressed bakery, battered and fried products and beverage formulations as well the overall trend toward more pulse consumption in Western societies. Please note also that, whereas these webinars addressed peas, chickpeas and lentils, there have also been significant developments in the use of dry bean (Phaseolus sp.) – derived ingredients.

This most recent webinar #4 addressed the importance of amylose-to-amylopectin ratios in pulse starches and the effects of proteins, dietary fiber, moisture and barrel temperature conditions on the expansion properties of HTST (high-temperature, short-time) extruded snacks and breakfast cereals. The webinar also looked at low-shear extrusion…specifically, the positive contributions of pulse flours on the textural qualities, cooking properties and nutritional value of pasta. Much of this work has been undertaken at the North Dakota State University-affiliated Northern Crops Institute (NCI), which can be linked at

In our view pulses are perfectly positioned to exploit a unique convergence of consumer trends in Western countries, that include: economy-driven price concerns, increased dietary protein consumption; heightened food safety awareness; sensitivity to environmental concerns and growing interest in ethnic-fusion cuisines. These trends will continue to stoke interest in pulses and pulse ingredients for the foreseeable future.

Here is a top-line overview of our four-part series:

 Pulses are going mainstream

Pulses increasingly appear in center-plate entrees, soups, salads and side dishes. New product applications for pulse ingredients include breakfast cereals, nutritional products, sweet baked goods, breads, nutrition bars, extruded snacks, crackers, snack chips, battered and fried goods, beverages and nutritional fitness products. Representative formulations were provided.

Quality standards

Pulses constitute a relatively new growth category in foods and international quality benchmarks have yet to be standardized. North America’s agricultural environment, combined with its production, shipping, handling and technical support infrastructure, has transformed it into a world leader in the production and export of high-quality dry beans, lentils, peas and chickpeas.

Strong consumer drivers

These include: strong demand for high-protein foods; the search for overall improved nutritional value; increased awareness of the importance of Glycemic Index and an aversion to the presence of gluten and other allergenic ingredients in foods. Pulse ingredients do not require allergen warning statements on food packages and examples were provided of how they can replace egg, milk and soy ingredients in a range of food formulations…at (usually) a considerably lower cost. Replacing cereal flours with pulse flours will significantly improve the nutritional profile of products and renders possible nutritional content claims for protein and dietary fiber. Finally, North American-grown pulses are environmentally friendly, as they are used as rotational crops that rejuvenate soils and that require very low levels (if any) of agricultural chemical inputs.

Functional pulse ingredients

Pulses differ from legumes, such as soy or peanut (groundnut), in that they contain significant quantities of starch and virtually no oil. Pulses contain high levels (22-30%) of highly functional proteins (similar to soy proteins) and a significant portion of their starch fraction is in the form of resistant starch. Already, a plethora of highly functional pulse ingredients have been made available to product developers from a growing number of ingredient suppliers, including starches, proteins, dietary fibers, brans and starch-protein combinations. This is only the beginning.

Cost advantages

The relatively low cost of pulses as compared to other protein (milk, eggs, soy) or even starch sources (corn, tapioca) provides a low-cost basis for developing further-processed pulse ingredients. In addition, they exhibit relatively low price volatility, which protects processed foods against commodity price swings. That being said, there can still occur temporary price spikes for the more-highly processed ingredients such as pea protein isolate, which are reflective of rapidly increasing demand outstripping production capacity (as has happened in the last two years with pea protein isolate). Nonetheless, such spikes are temporary and limited to the more narrow pulse ingredient segments. The industry is still catching up to increasing consumer demands for food product attributes that only pulse-based ingredients are best positioned to satisfy. It will take time.

A review of this Webinar #4 was published by Food Navigator and can be found at:

The Link for the four archived webinars can be found at the US Dry Pea & Lentil Council’s website under the “Food Industry Tab” at:

Pulse Proteins in Value-Added Food and Beverage Product Development



SEPTEMBER 24, 2013


Legume foods represent the perfect protein food in this conflicted era of economic stress, high-value food expectations and environmental responsibility. Here is an opportunity to learn the how and why of incorporating  pulse and pulse ingredients into food product development

This addresses the third in a series of four WEBINARS that address how to use pulses and pulse ingredients in food product development.

Legume foods (soybeans excepted) fall under the category of “pulses”, which include peas and chickpeas and lentils. The distinguishing feature of pulses as foods is that 1) they are high in protein content (20-30%), high in dietary fiber (9-17%) and very low in fat. They are also highly sustainable crops. As legumes, they restore nitrogen to soils and, in most large pulse-producing regions, require only minimum if any applications of agricultural chemicals.

Because pulses are not listed as allergenic foods, they can easily replace other allergenic protein ingredients, such as egg, milk, soy, nutmeats and wheat gluten.  Their well-balanced primary ingredient components (protein, starch and dietary fiber) are also highly functional, contributing thickening, emulsification, shelf-life extension and other properties to foods.  Finally, pulse-based ingredients are low cost.

Today, “protein” is a very hot food commodity, as consumers actively seek to increase their dietary protein consumption. Why?

Pulse proteins are nutritionally beneficial

The U.S. “2012 Food & Health Survey on Consumer Attitudes Toward Food Safety, Nutrition & Health”, conducted by the International Food Information Council (IFIC), found that 56% of consumers actively look for protein content on a food ingredient label and 48% actively try to consume a specific amount of or as much as possible protein in their food choices. The reasons given for protein’s appeal varied:

  • 88% recognize that protein helps to build muscle.
  • 69% recognized that protein “helps people to feel full”
  • 60% recognized that “high-protein diets can help with weight loss.”

On that note, pulse proteins offer  high-nutritional value, as measured by nutritional “Protein Score”: pulses offer all of the essential amino acids and a very significant component of the branched chain amino acids associated with muscle growth and healing.

Unfortunately, 25% of respondents also disclosed that they believed “Foods that contain protein are too expensive to consume as much as I would like.” For young, lower-income or overweight consumers, this number well-exceeded 30%. Well, that may be true for meat, egg and dairy products, but it is not the case for pulse foods and pulse protein ingredients.

Pulses proteins are highly cost-effective.

Pulses and pulse ingredients can be used to very cost-effectively boost the protein value of foods: for example, an analysis of the cost-per-unit protein of pulses found that they generally cost between 1/10th to 1/20th the cost-per-unit of protein in egg and milk ingredients. In addition, they exhibit very low price volatility, which is helps food and beverage-company purchasing agents sleep better at night. This, too, renders them highly desirable in food and beverage formulations.

On Tuesday, September 24, 1:00 – 2:00 PM U.S. Central Standard Time, the USA Dry Pea & Lentil Council will host the third in its series of four FREE “Food R&D Trends” WEBINARS that offer practical “how-to” guidance on using pulse ingredients as value-added protein sources in food and beverage formulations. Pulses can play a very important role in cost-effectively satisfying growing consumer interest in the protein and amino acid value of their foods.

The WEBINAR will address the following:

• Pulses as Food Ingredients

• Why the Growing Consumer Interest in Proteins

• The Nutritional and Nutraceutical Value of Pulse Proteins

• How to Use Pulse Ingredients in High-Protein Food and Beverage Applications

• Food and Beverage Formulations

For more details on this FREE WEBINAR and how to register and participate, please follow this link:

Napa Valley (California) Culinary Tour (September 17-18, 2013)

If you can fit it into your schedule, please join the feast. A FREE event open to the food industry: Developed and underwritten by the USA Dry Pea & Lentil Council together with the Culinary Institute of America, this two-day educational seminar will take place at the Culinary Institute of America’s premier culinary college at the elegant Greystone in St. Helena, California (a 90-minute drive from San Francisco, Oakland or Sacramento airports). The event will explore and create new food, beverage and nutritional product concepts using pulse and pulse-based ingredients.

Please note: although attendance and lodging is underwritten by the USA Dry Pea & Lentil Council, you will still need to provide your own transportation.

Editorial comment: I attended a similar horizon-expanding culinary event in Chicago earlier this year, sponsored by the USA Dry Pea & Lentil Council in conjunction with Chicago-based Culinary Sales Support, Inc. and I can thus highly recommend attending this Napa Valley event if you can at all fit it into your schedule.

Not only will you gain a hands-on education about a new and exciting growth category and its many, many applications, you will also walk away with many new creative product ideas developed by America’s leading culinary institute. In BEST VANTAGE Inc.’s view, pulses represent a still early-stage product category for the American food and foodservice industries. The stunning rise of hummus as a ($0.5b -plus) retail and foodservice product category in just a decade should alert us all to the future possibilities for this category. The Chicago presentation was replete with ideas for pulse-derived baked goods, salad toppings, sauces, desserts, nutrition bars, soups, meats and beverages. I would expect even more creativity to find expression at this upcoming Culinary Institute of America event in September.

Many of today’s pulse-based product applications draw from recipes originating in South Asia or the Middle East. This is only the first stage. The next stage of development will see the fusion of pulse and pulse-based ingredients into new combinations tailored to American tastes and imaginations. (a wasabi-flavored hummus developed by chefs at the Chicago event offered very interesting retail possibilities, in my view). Be forewarned, however: you may inadvertently waaay overeat. — Daniel Best

To find out more about this event and to register, please follow this link:

2013 IFT-EXPO Trends – Encapsulated!

BEST VANTAGE Inc. associates have been devoted attendees of the international Institute of Food Technologists’ Annual Meetings & Expos (IFT EXPO), held this year in Chicago (July 14-16). This year as in others, our members diligently covered the expo floor, gathering information to discern the consumer and technological trends that shape our food industry. Attendance at this year’s meeting was at a record high, so we were told. Here is a quick summation of this year’s major trends observed:

Clean labels were a dominant theme at this meeting. Food companies are gravitating to ingredients that read well on package labels and avoid ingredients that sound complex and artificial. So, definitely, anything “natural” is “in”. As one of our researchers put it, this means that ingredients such as gluconate, for example, could be in trouble, due to the word’s phonetic resemblance to “gluten” or “glutamate”. Food ingredient companies may want to revisit their marketing materials and reposition their narratives. Incidentally, Global Food Forums will highlight an applied R&D-focused conference on “Clean Labels” October 29-30, in Oakbrook, Illinois. Don’t miss it.

Gluten-free is hot…very hot! We knew that, as we have been doing a fair amount of work in gluten-free R&D. The art and science of gluten-free product development are still new, so opportunities abound. As per earlier posts, BEST VANTAGE believes that the gluten-free market projections provided by product-scan market research firms are way-too low: we project the market value at $70b by 2020. If you doubt us, call us or email us.

Natural colors everywhere. This is definitely a big push, in line with the food industry’s segue into clean label, sustainable and other consumer-friendly narratives. Even flavor companies (such as FONA) are jumping on this trend. There’s a lot of money to be made in this category and, as far as the technical challenges involved…well, that just looks like the color of opportunity. We aren’t there yet, but there has been quite a bit of progress

Proteins and more proteins. Surging cost-and-demand pressures on animal protein ingredients (e.g., egg, dairy), largely from Asia, are stoking interest in new sources of plant proteins. Pulse proteins (peas, chickpeas) are just now coming onto their own, with companies such as Allied Grain Traders (AGT) of Bismarck, North Dakota and Harvest Innovations (Indianola, Iowa) introducing new ingredient offerings. Burcon Nutrasciences Corp. (Vancouver, British Columbia) offered a pea protein isolate technology for licensing, with good flavor. Burcon has also developed a canola protein technology. Up-and-coming in the food industry: oat and potato proteins. 

Pulses. Full disclosure – the USA Dry Pea and Lentil Council is one of our clients and pulse ingredients have figured prominently in some of our gluten-free work and webinar presentations. Once derisively dismissed as “poor man’s meat”, pulses are now quite culturally chi chi, offering compelling culinary narratives with respect to variety, nutrition, label friendliness, allergen aversion and sustainability. We remain great fans! Prior to the IFT-EXPO, we participated in “Culinary Tour” of the Chicago restaurant scene, sponsored by the USA Dry Pea & Lentil Council. The tour featured wonderfully innovative renditions of pulse foods and food ingredients. A wasabi-flavored hummus…fabulous! We have seen the future. Go forth in peas!

Blended natural sweeteners. Two years ago, at the IFT-EXPO in New Orleans, the U.S. food industry witnessed a full-court press (using American basketball terminology) of stevioside, rebaudioside, luo han guo fruit (mogrisides) and other natural sweeteners that promised to sweep the industry. The only problem was that, with the exception of agave, they generally tasted, well…awful! This year these same ingredients were back, blended with other natural sweeteners. They were also very, very credible. Blends of natural (caloric) sweeteners with low or non-caloric sweeteners likely represent the future of our industry.

Sodium reduction. This trend, in our estimation, has peaked and now flat-lined. Companies now have a good understanding on the opportunities and limitations of sodium-reduction alternatives.

For the past six years, BEST VANTAGE Inc. has formulated and tracked technical developments in this category. This year, we noticed a marked flip from sodium reduction to potassium awareness (i.e., potassium is good!), which makes much more nutritional sense. The U.S. population is not so much over-sated with sodium as it potassium deficient. It is also deficient in two other critical nutrients that affect hypertension: calcium (/ vitamin D) and magnesium. It appears as if the food industry and consumer have both caught-on, which should spell excellent opportunities for new food product development. A major wild card that was thrown into the mix was the recent (May, 2013) statement, by the U.S. National Academy of Science’s Institute of Medicine (IOM), that questioned about the skewed focus of U.S. public health authorities and nutrition nags on the overall public health benefits of dietary sodium-reduction. Let’s see what transpires.

Ethnic flavors. One of the appealing attributes of the U.S. consumer market is its incessant demand for innovation and variety. Combine: the competitive drive for innovation in the American food manufacturing and restaurant industries; heavy influxes of immigrants from the around world, and the peripatetic wanderings of American tourists, and what do you get? Not confusion, but cultural food fusion. This year, Caribbean and South American influences predominated (e.g., Puerto Rican, Peruvian), but we also saw a flowering of East Mediterranean (Turkish, Middle Eastern) flavors and hints and glimmers of African and South Asian cuisines on the horizon. In many of the foods and flavors offered on the IFT-EXPO floor, East met West and South merged with North in ethnic interminglings of exciting new eating sensations. Fusion!

This really is an industry worth celebrating.