Our dairy industry is under serious stress. For most dairy farmers this is the third consecutive year in which cash coming in is much less than cash going out. Last year the average cash deficit was $78,000. This year could be worse.
Debt to cover working capital is climbing. Average return on equity is negative. Highly geared farmers with high costs are facing foreclosure.
Across the board, farm spending is being slashed and regions with high exposure to dairying are being hit hard.
Why is this happening? In a nutshell, supply has been significantly greater than demand in overseas dairy markets and prices have plunged as a result.
Only about 6 per cent of world dairy production is traded. So very small changes in global supply and demand can have a really big impact on prices. This makes international dairy prices inherently volatile.
The last decade in particular has been a price rollercoaster with three sharp rises and falls. At the peak, prices reached their highest levels since the mid-1970s.
However, the plunge over the last two years dropped prices to levels lower than when Fonterra was formed 15 years ago.
Adjusted for inflation, last year's average payout was about the 4th lowest since 1950.
Under the co-operative system, most of this price risk is borne directly by dairy farmers. So dealing with volatility is a fundamental part of dairy farming.
"Right sizing" the farm so its break-even point can cope with this price volatility is critical.
From a wider perspective, dairy price risk has significant implications for our economy as a whole with around 30 per cent of our export earnings coming from the dairy sector.
As Chair of the Productivity Commission, Murray Shewin, highlighted in a 2012 address: "Successful economies try hard to understand the risks they are exposed to, and devise means to reduce their exposure and build resilience in the face of risks".
The thing that stands out in our dairy industry is its continuing concentration on a single strategy and structure - an approach not well suited to effective risk management or adaptation for wealth creation.
The plunge over the last two years dropped prices to levels lower than when Fonterra was formed 15 years ago.
Farmers and the country would be better served by a diversity of approaches.
Diversity is not an outcome but a process. In this context, it refers not just to spreading risk, but more broadly to a relentless dynamic in which people and firms - trying to achieve returns over time that properly reflect the risk they are prepared to take - continuously adapt the use their resources to changing risks and opportunities.
This generates an ever evolving array of ideas and strategies - reflecting the fact that innovation is not the domain of experts but ordinary people seeking to find better ways of doing things. As economist Tim Harford highlights, adapting to a complex changeable world is best achieved by a multiplicity of experiments from many different players.
This process of diversity is a cornerstone of continuous productivity improvement, not just in the dairy sector but the economy as a whole. As Nobel economist, Paul Krugman, famously put it: "Productivity isn't everything, but in the long run it is almost everything".
Graph: Dairy commodity price index:
In commerce, it's driven by constant competitive pressure, with firms and individuals competing to discover new opportunities and bidding to get resources to enable their ideas. The "strong and positive link" between competition and productivity is highlighted by our Productivity Commission.
Some people mistakenly think that competition means an absence of cooperation.
Not so - cooperation and collaboration are at the core of individuals or firms choosing to combine disparate resources in a new entity or joint venture. It also underpins the provision of true "industry goods".
Dairy's lack of diversity
The problem is, for the last 100 years, our dairy industry it has suffered from a serious dearth of diversity and experimentation beyond the farm gate.
Since the early days, dairy farmers have relied on a relatively small group of Government and industry insiders to deal with the value-chain between their farm gate and end-customers.
Governments promoted co-operatives over proprietary processors, banned competitive exporting, and enabled the industry to systematically eliminate competition between processors at the farm gate. Competition was (and, in many quarters, still is) viewed as "pernicious".
From the early days, protection from competition extended to heavily restricting the supply of non-dairy alternatives like margarine.
Even the rapid merging of dairy co-operatives in the lead up to Fonterra's formation was not so much a drive for efficiency but a race for control of the Dairy Board, with the winner hoping to secure the driver's seat in shaping the industry's future.
From the early days, protection was extended to heavily restricting the supply of non-dairy alternatives like margarine.
Rather than fostering competition and diversity, the industry's approach beyond the farm gate has been a relentless drive to homogeneity and centralisation.
So while our dairy industry has been highly successful in growing milk production, since monopoly exporting started during WWI, there has been minimal trialling by different parties of alternative approaches to aggregating capital, pricing, managing risk, using global value chains, understanding customers' preferences and, most importantly, using different strategies to create wealth from the many market opportunities that a handful of decision-makers in a near-monopoly seller simply can't see, or don't have the capacity to exploit.
This lack of diversity and experimentation stands out in a "down" market where the common strategy among many producers is simply to cut spending and pray for prices to pick up.
Fonterra v economy as a whole
But wasn't deregulation and the creation of Fonterra 15 years ago supposed to change all that? The answer is, not really. And here we hit the crux of the problem.
There is a fundamental conflict between Fonterra's demand for dominance in New Zealand and the interests of the wider economy.
To Fonterra's backers, its genetic purpose is to be a near-monopoly national dairy co-operative controlling the lion's share of New Zealand milk so it can be our "national champion" in global dairy markets. Dominance in New Zealand is encoded into its concept design - so it has scale to compete overseas and grow more value from New Zealand milk. Some competition at the margins is fine, but nothing should erode the size of its core New Zealand platform.
But the wider economy is best served by competition and the process of diversity.
Firms like Fonterra, with artificially high market power, typically stifle diversity. They are generally not good for an industry or the economy.
The simple reason is that, over time, the gains from diversity typically hugely outweigh any gains from scale. With highly dominant firms like Fonterra, we miss out on those diversity gains - it's a big cost to all of us.
Fonterra is based on an article of faith that the benefits of size and integration will outweigh those dynamic losses. Unfortunately, the opposite is almost invariably true.
Is the regulatory regime effective?
Like the dissonance between the Fonterra and the wider economy, the Government's regulatory regime for Fonterra is pulling in different directions.
One set of measures aims to grow dairy competition in and from New Zealand with a view to sustained competitive pressure becoming the main driver of efficiency, not just in reducing costs but, more importantly, in fostering the dynamic of diversity.
To this end, the regime provides various aids to help competitor processors enter and grow their market share. For example, the rules allow farmers to freely join and leave Fonterra, and Fonterra is required to make about 3 per cent of its milk available to competitors.
Another set of regulatory measures aims to promote efficiency by overseeing the way Fonterra sets its raw milk price. This approach is broadly equivalent to regulating prices of natural monopolies (like airports and electricity lines companies).
However, Fonterra is not a natural monopoly and the milk price rules seem to be delivering weaker pressure on Fonterra than intended while also narrowing the window for competitors to enter.
How much competition has the regime fostered?
Since 2001, about eight relatively niche dairy processors have entered the market taking between them about 4 per cent of milk supply in the North Island and about 20 per cent in the South Island.
Overall, Fonterra has lost only about 11 per cent of its market share in New Zealand over the past 15 years. It remains highly dominant, even in the face of huge growth in milk production.
Fonterra's expectation of some-but-not-too-much competition seems to be shared by the Government. Cabinet papers express concern not to over-stimulate competition "at Fonterra's expense".
To illustrate some of the problems in the regulatory regime, let's take a closer look at three key elements.
Wholesale milk price
In simple terms, the price Fonterra pays its farmers for their milk is the money left over after deducting operating expenses and capital costs from Fonterra's commodity revenues.
However, instead of using Fonterra's actual costs and product mix, the rules assume Fonterra can match the lower costs and better product mix of a very efficient hypothetical competitor. This has the effect of boosting Fonterra's milk price.
The size of this boost has been steadily increasing with Fonterra using ever more idealised models of its hypothetical competitor.
The problem is, to get farmers to leave Fonterra, a real competing processor has to match or better Fonterra's boosted milk price.
While in theory farmers should compare total returns, not just milk prices, in practice milk price is the headline driver. This means that independent processors more efficient than Fonterra, but less efficient than the hypothetical competitor used in the rules, tend to be boxed out. In effect, the rules narrow the window of competitive entry.
Setting a milk price as if Fonterra were a very efficient processor squeezes Fonterra's profits. Until it gets its costs down and improves its product mix, Fonterra can only pay the higher milk price by short-changing its profits and return on equity. This is supposed to induce Fonterra to improve its efficiency so it can deliver both the higher milk price and a proper return on equity.
However, the logic seems to have a weak link. Boosting the milk price by short-changing profits is not at all unusual for dairy co-operatives - indeed, it's what they do. Since it was formed, Fonterra has struggled to cover its cost of capital.
This is because return on a co-operative's equity is not a particularly important performance measure for its farmers. From their perspective, the co-operative is an extension of their farms - a club in which farmers jointly own assets to provide shared services (dairy processing, marketing and exporting).
The club is expected to cover its costs and pay out the rest of its earnings to club members. It's seen as a tolling operation, not a profit centre. Members measure performance from the perspective of their farms.
In theory, the 7 per cent of Fonterra's shares capital held by the Fonterra Shareholders Fund should strengthen its drive to deliver profits at proper rates.
Fonterra's boosted milk price creates three other problems: it reduces funds available for value-adding activities, it over-stimulates milk production and it causes unnecessary environmental effects.
However, investors in the fund have no votes and, in valuing their shares, the market recognises the reality that Fonterra's purpose first and foremost is to serve its member-farmers by maximising their milk price, and that the adequacy of Fonterra's profit is a second or third order consideration.
In short, the pressure created by the milk price rules on Fonterra to improve efficiency is likely to be weaker than assumed. If so, the regulatory regime is narrowing the window of competitive entry for limited gain, while also increasing reliance on milk price regulation rather than competition as the main efficiency discipline on Fonterra within New Zealand.
Fonterra's boosted milk price creates three other problems: it reduces funds available for value-adding activities, it over-stimulates milk production and (as a result) it causes unnecessary environmental effects.
The regulatory rules require Fonterra to make certain volumes of milk available to competitors. The Government is now proposing to phase-down this obligation.
One of the main parties adversely affected is Goodman Fielder, whose brands - Meadow Fresh, Naturalea, Puhoi and Chesdale - compete against Fonterra's brands - Anchor, Mainland, Tip Top and Kapiti - in our domestic retail markets.
In effect, the Government is saying to Goodman Fielder: there isn't enough competition at the farm gate so you need to start buying your milk directly from hundreds of individual farmers, not Fonterra as of right.
However, the problem is not with Goodman Fielder; it's due to the absence of a wholesale milk market.
In forming Fonterra, a key part of the Government's vision was that Goodman Fielder (and others) would be able to buy their milk on a wholesale milk market that Fonterra would want to create with new entrant dairy processors. It was to be like the New York Mercantile Exchange with a spot market for milk backed by contracts and derivatives trading. Until it was set up, however, Fonterra would have a regulatory obligation to make milk available to Goodman Fielder at a regulated price.
That wholesale milk market vision has not been realised. Nor is there sufficient competition for milk at the farm gate. But neither problem is the responsibility of Goodman Fielder, and nor is the solution.
The real problem rests with the Government's regulatory regime and Fonterra's artificially-created dominance. Rather than forcing Goodman Fielder to significantly change its business model, it would be more logical (and better policy) to create the intended wholesale milk market. Some of the early initiatives used to set up the wholesale electricity market may be helpful precedents.
Obligation to accept all milk from members
With a few exceptions, the regulatory rules allow any dairy farmer to become a member of Fonterra and to hold shares in proportion to his or her volume of supply.
Fonterra argues that this obligation drives it down the commodities path.
The Commerce Commission rejected this claim, finding that Fonterra's commodities push comes not from the rules, but from two drivers of Fonterra's own making: its co-operative structure, which requires it to process all milk from farmer-shareholders; and its new 10 year strategy, which calls for an additional 8 billion litres of New Zealand milk (about a 38 per cent increase).
What about value-added?
Of course, Fonterra's main plan to deal with price volatility and lift value for farmers is to earn a higher proportion of revenue from higher margin products. This is the perennial value-add strategy - long promised but scarcely delivered.
Forming Fonterra was supposed to enable it. According industry biographer Clive Lind, the idea of merging the whole New Zealand dairy industry into a single vertically integrated co-operative was actually conceived 35 years ago, driven by a belief from leaders like former Dairy Board chief, Warren Larsen, that "if the industry was split up, competing companies selling dairy product in the international marketplace would be forced to give up margins and revert to commodity trading", which would lead to lower prices for New Zealand farmers.
The sad irony is that this is exactly what the long-planned mega-merger has delivered - a huge increase in commodity-related sales leading to lower prices for farmers. Fonterra today is more overwhelmingly a commodities and ingredients business that it was 15 years ago.
Yet its plan for 2025 still calls for a major increase in its share of earnings from higher margin products. And like McKinsey's hype 15 years ago, advisers like KPMG still laud that it will "unleash the giant" on a "positive flight path" that will "systematically transition the business from the world's most competitive dairy supplier, to being a globally relevant food business".
Risk allocation is another domain where Fonterra's current tools are limited.
It would be a lot more helpful if Fonterra were to provide its farmers (and itself) with an objective critique of why the industry has so consistently missed its value-added mark over the last 35 years, and what needs to change.
Fonterra and KPMG say it boils down to "velocity and discipline". This is racy jargon, but not strongly connected to root problems, which have more to do with incentives, pricing structures, access to capital, and economic tools to fix the poor alignment between Fonterra's business model and the demands of consumer-driven markets in which it wants to compete.
Last year's article set out six key reasons why it hasn't worked. Three more should be added.
Pricing structures matter. They are like traffic lights on a multi-lane intersection steering the way resources flow, in what volumes and at what times. Fonterra's pricing signals are askew at several major "intersections", particularly in signalling milk flows.
Take two examples: the price farmers pay Fonterra for the right to supply a unit of milk, and the price Fonterra pays farmers for their milk.
Farmers normally have to hold Fonterra shares in proportion to the volume of their supply - one share gives the right to supply one unit of milk.
So the price a farmer pays for the right to supply a unit of milk is the market value of a Fonterra share, which is the present value of expected future dividends. The problem is that this price bears no relation to relevant costs or services.
The correct price for a right to supply is the marginal cost to Fonterra of processing an extra unit of milk, which has nothing to do with the value of a Fonterra share.
Alternatively, Fonterra could ditch the requirement for farmers to buy the right to supply and, instead, buy milk from farmers at prices that reflect the market value of their milk, minus the marginal cost of processing each unit of it. Precedents from other industries may be instructive.
The current one-size-fits-all approach puts all farmers in the same risk box, rather than allocating risk to the party best placed to manage it.
At present, all milk at all times from all farmer-shareholders in all locations receives the same single nationally averaged price, which is calculated at the end of the season when costs and market returns are known. Farmers have a guaranteed buyer, and nothing signals the relative value to Fonterra of different quantities of milk from different locations at different times of the year.
The result is poor alignment between milk flows, processing assets and Fonterra's value-add strategy.
Allocation of risk
Risk allocation is another domain where Fonterra's current tools are limited.
As an example, it may make more sense for farmers with a lower risk appetite to lock in their milk price in advance of a season and let the processor carry market risks within the season, recognising that farmers can make major changes to their production levels across seasons but have much less room to adjust within a season.
The current one-size-fits-all approach puts all farmers in the same risk box, rather than allocating risk to the party best placed to manage it.
Fonterra's capacity is also constrained by its limited access to additional share capital, which currently comes from two sources: farmer-shareholders buying more shares to match any growth in their milk supply, and retained earnings (just $200 million has been retained over last two and a half years despite significant growth).
Apart from providing financial horse-power needed to implement its growth plan, accessing equity from other sources would expose Fonterra's strategy to stronger scrutiny and accountability, and help to better align its business with critical value drivers.
Constrained by old beliefs
These and other more market-driven tools are not available to Fonterra for two reasons.
First, while Fonterra remains so dominant in New Zealand, some tools could be used in an anti-competitive manner.
Second, most of the tools cut across deeply ingrained industry beliefs. These precepts flow from a basic fear, forged in the industry's early days, that farmers will lose to outsiders if outsiders are allowed in. However, nothing from industry historians like Arthur Ward, H G Philpott, David Yerex, Gordon McLauchlan or Tony Nightingale indicates that this fear has any empirical foundation - on the contrary.
The industry is also shaped by another potent fear - the idea that New Zealand dairy exporters competing against each other in foreign markets will drive the price down. A corollary belief is that a near-monopoly New Zealand dairy exporter can get higher prices than its overseas competitors. Fonterra was created on this premise - as its first chairman trumpeted: "Fonterra gives us market power".
With rare exceptions, this is false. Anyone trading commodity-related dairy products in competitive markets is a price-taker, not a price-maker. About this our Commerce Commission and other authorities are unequivocal.
Fonterra's plans are further challenged by the real prospect of more milk from larger countries spilling into export markets.
Yet the myths still have a strong hold, as we see in DairyNZ's 2010-2020 industry strategy assertion that "competition between processors may erode export returns".
Waikato University's professor Jacqueline Rowarth even recycles the hoary old chestnut that "New Zealand farmers would be peasants" if they don't have a near-monopoly dairy exporter.
Fonterra's root problem is that its approach to the future is still heavily constrained by misplaced beliefs from the past. The result is a self-shackled industry that holds the country back.
Where to from here?
At a big-picture level, Fonterra is relying on three things for its future: more people around the world becoming middle class and choosing to buy milk; countries that want more milk importing it more than increasing local production; and Fonterra winning a sizeable chunk of this expected growth in demand.
The problem is that Fonterra needs to produce a lot more milk more cheaply than its competitors. However, New Zealand is no longer the lowest cost producer - several countries are now cheaper. Our kind climate, plentiful grass, cheap water and good cow genetics - the core elements of our competitive edge - are no longer enough.
Nor is it clear that countries with growing milk demand will rely on imports. Many are quickly building relatively low cost home-grown milk supplies.
Fonterra's plans are further challenged by the real prospect of more milk from larger countries spilling into export markets. While milk production in most countries has been mainly a domestic business, Europe, Canada and the USA have the capacity to readily increase their dairy exports to a degree that could dwarf any increase from New Zealand.
Fonterra's best option - for it, the industry and the country as a whole - would be to voluntarily give up a large enough chunk of its farm gate market to create workable competition.
In short, Fonterra's competitive position in global dairy markets has serious vulnerabilities. Yet as a country we continue to concentrate our approach in a single entity, with farmers relying heavily on land prices rising to achieve a reasonable rate of return.
Fortress Fonterra is not the answer. We need the process of diversity - a cornerstone of growing wealth for the country. This matters much more than Fonterra's size and the illusion of scale.
How do we get there? Following the current pathway, the short answer is - very slowly, if ever.
Several leading columnists have renewed calls for Fonterra's foodservices and consumer business to be parcelled into a separate company controlled by Fonterra with access to non-farmer equity. Others see the co-operative form as inherently unsuitable for driving innovation.
Fonterra's best option - for it, the industry and the country as a whole - would be to voluntarily give up a large enough chunk of its farm gate market to create workable competition. It should also actively back the creation of a wholesale milk market. Many regulatory restraints on Fonterra could then lapse.
Such a step-change would establish more definitively and quickly the diversity dynamic so essential for resources to move and adapt in response to changing risks and opportunities.
It would also create a catalyst for Fonterra's leaders to use the tools it needs to fix the root problems impairing its strategy for generating higher returns for farmers.
The industry has persevered with the experiment of a near-monopoly dairy organisation for a very long time when it is clearly not achieving its goals and is not in the best interests of economy. It's time to put preconceptions aside.
Stubbornness can be a good attribute, but it makes no sense if it means marching the country to a dead-end.