More detail on what exactly caused NYSE's trading glitch on Tuesday

More detail on what exactly caused NYSE’s trading glitch on Tuesday

A trader works on the floor of the New York Stock Exchange. 

Peter Kramer | CNBC

NYSE President Lynn Martin and other exchange officials confirmed to CNBC that the root cause of the Big Board’s trading glitch at the Tuesday open was due to a manual error involving the Exchange’s Disaster Recovery configuration. 

After the 9/11 disaster, the NYSE was obligated to maintain a primary trading site (at the NYSE) and a back-up site (which is in Chicago). 

related investing news

Pro Picks: Watch all of Tuesday's big stock calls on CNBC

CNBC Pro

On Monday evening, routine maintenance was being performed on the software for the Chicago back-up site. 

On Tuesday morning, the back-up system (Chicago) was mistakenly still running when the primary system (NYSE) came online. 

Because the back-up was still running, when the primary site started up some stocks behaved as if trading had already started.  

As a result, Designated Market Makers (DMMs) who would normally publish an opening auction print for each stock were prevented from doing so because the system operated as if an opening had already occurred.  This caused significant price dislocations and trading halts. 

Martin said the NYSE was looking into implementing stronger testing protocols. 

NYSE says Tuesday's trading glitch due to 'manual error'

NYSE says Tuesday’s trading glitch due to ‘manual error’

The bell at the NYSE

Source: NYSE

The day after a major trading glitch at the New York Stock Exchange open, the NYSE has issued a statement on what happened: “The root cause was determined to be a manual error involving the Exchange’s Disaster Recovery configuration at system start of day.”

That’s all they are saying. In plain words, it appears they tested a “disaster recovery configuration” that did not involve using the floor, and it did not reset.

That fits the facts as we know them.

Traders noted that both Designated Market Makers (DMMs) and floor brokers appear to have been frozen out of the order book that is used to build the opening print. No opening print was provided in dozens of big-name companies. “It was almost like trading opened without the participation of the floor,” one observer who asked to remain anonymous told me.

What we know is that dozens of stocks opened at prices well above or below their prior day closing prices. Most were halted shortly after the open under rules designed to damp down excessive volatility, and most reopened five to 10 minutes after the open at prices much closer to Monday’s closing prices. Many orders to buy and sell stocks did not make it into the order book that determines the opening price, and the opening auction print did not happen in those affected stocks.

The announcement by the NYSE late Tuesday that some trades that occurred right at the open would be busted, but others would not, added to the confusion. Many are trying to figure out how much money they may have lost yesterday.

NYSE's Tuesday trading glitch explained — Why some of the trades may be busted

NYSE’s Tuesday trading glitch explained — Why some of the trades may be busted

A trader works on the floor of the New York Stock Exchange. 

Peter Kramer | CNBC

The New York Stock Exchange experienced technical issues at the open Tuesday.  Dozens of stocks opened at prices well above or below their prior day closing prices. Most were halted shortly after the open under rules designed to damp down excessive volatility, and most reopened five to 10 minutes after the open at prices much closer to yesterday’s closing prices.

Stocks affected included big names like Altria, Mastercard, McDonalds, Uber, Wells Fargo, Verizon, Rio Tino, Shell, AT&T, Lilly, Mosaic, Wells Fargo, Nike, Nucor, Transocean, Prudential,3M, Newmont Mining, Southern, United Pacific, Sony, United Parcel Service, Altria, Valero Energy, Occidental Petroleum, Royal Dutch Shell, MetLife, Visa, Walmart, and Exxon Mobil.

The Big Board, owned and operated by the Intercontinental Exchange, later issued a statement saying “All NYSE systems are currently operational.”

Just prior to 11:00 a.m. ET, the NYSE issued a second statement: “The exchange continues to investigate issues with today’s opening auction. In a subset of symbols, opening auctions did not occur. The exchange is working to clarify the list of symbols. Impacted member firms may consider filing for Clearly Erroneous or Rule 18 Claims.”

“Clearly Erroneous” means the NYSE would determine that the initial prices in the stocks affected were not valid trades and the NYSE would determine that a later price would be the “correct” opening price. 

What happened?

Every day, stocks open at the NYSE at or near 9:30 a.m. ET.  There is only a single opening price, which is determined by thousands of orders to buy and sell individual stocks. These orders are aggregated into a single “book” for each stock that gauges overall supply and demand.  A single price is then quoted at the open and all orders are aggregated into a single opening “auction print.”

For whatever reason, it appears that many orders to buy and sell stocks did not make it into the order book that determines the opening price, and that the opening auction print did not happen in those stocks affected.

The effect was that many stocks opened on very low volume and due to a supply-demand imbalance opened at prices far away from their closing price Monday.

To give two examples:  Mosaic closed Monday at $48.35, but opened at $40.29, a drop of about 16%.  It was halted almost immediately, but reopened at 9:43 a.m. at $48.00.

Walmart closed Monday at $142.64 but opened at $159.88, a jump of 12%. It, too, was almost immediately halted and reopened at 9:40 a.m. ET at $141.51.

What will the NYSE do?

The NYSE has already hinted it may bust all the initial trades of companies affected when it said, “Impacted member firms may consider filing for Clearly Erroneous or Rule 18 Claims.”

This is the most likely path because many investors who, for example, put in a market order to sell Mosaic at the open this morning were clearly burned (it opened down 16%) and would likely threaten lawsuits if not made whole, since the price drop had nothing to do with the company or external events.

Most likely, the “correct” opening price will be the price when the stocks reopened.

So what happened?

The NYSE has not provided an explanation.  However, in the past these types of outages have been associated with software or security upgrades that caused snafus in the system.

On July 8, 2015, trading was halted for nearly four hours after the NYSE experienced what it called  an “internal technical issue.”   The NYSE later said, “The root cause was determined to be a configuration issue.”

In a separate event, faulty software brought down Knight Trading in August, 2012, in an incident which sent enormous amounts of erroneous orders onto the trading floor.  The incident forced to Knight to sell out to a group of trading firms.

“Nine times out of ten, these problems happen because of a software change in the system,” one market participant who asked to remain anonymous, told me.

Burton Malkiel on why his classic investment book, 'A Random Walk Down Wall Street,' is relevant 50 years later

Burton Malkiel on why his classic investment book, ‘A Random Walk Down Wall Street,’ is relevant 50 years later

This week, Princeton professor Burton Malkiel has published the updated, 50th anniversary edition of A Random Walk Down Wall Street: the Best Investment Guide that Money Can Buy.  More than any other book, it popularized the idea of indexing as an investment strategy and why you can’t beat the market. Malkiel was a close friend of Vanguard founder Jack Bogle and spent 28 years on the board of Vanguard. 

Malkiel will be on CNBC’s “The Exchange” today at 1:30 PM ET. Below are excerpts from a series of email interviews.  They have been lightly edited for style and clarity. 

You recommended index funds 50 years ago even before index funds existed. Do you still believe that and what is the evidence?

I believe even more strongly than ever that index investing is an optimal strategy and that index funds should constitute the core of everyone’s portfolio. Standard & Poor’s publishes annual reports showing how actively managed funds compare with index funds. Each year about two-thirds of active managers underperform an index fund. And the one third who outperform in one year tend not to be the same as the one-third who outperform in the next. When you measure performance over a decade or more, 90 percent of active managers are outperformed by a broad-based index fund. On average, active managers underperform the market by about one percent per year. 

You famously said stocks tend to follow a “Random Walk.”  What is a Random Walk? 

Random Walk means that the history of past stock market prices cannot be used to predict the future. Sometimes there is some momentum in the market, but momentum strategies do not work reliably, and prices change randomly. Sometimes value stocks and small-cap stocks outperform, but they can underperform for years. So-called “factor funds” have underperformed the market over the past 15 years. 

Despite the evidence, it seems most people don’t want to believe the evidence. Why can’t we accept that stocks follow a Random Walk?

People fool themselves when they see apparent patterns because streaks are more memorable than randomness. Even sports figures fool themselves into believing that the streak exists and that they have a “hot hand.” For example, behavioral psychologists have examined the basketball free throw records of college and pro players who believe that if they make a number of free throws in a row, they are more likely to have success in the next shot. The evidence is quite the opposite. A 50% free throw shooter has a 50% chance of success on the next shot no matter how many previous shots went in. 

You are a big backer of the Efficient Market Hypothesis (EMH), which says that asset prices reflect all available information. But you have pointed out this does not mean that prices are always accurate. Define EMH, what it says, and what it does not say.

Efficient markets do not imply that prices are always correct. Even if everyone valued stocks as the present value of future cash flows, the future can only be estimated. So prices are often “wrong.” EMH says that no one knows for sure if they are too high or too low. EMH admits that bubbles can exist but no one knows for sure how much they inflate before popping. Alan Greenspan suggested that the market was “irrationally exuberant” in 1996.  The bubble popped in 2000. Meme stocks like Gamestop sold at bubble levels but hedge fund Melvin Capital went bankrupt shorting them. The market may not be perfectly efficient and it may make egregious errors.  BUT IT IS EXTREMELY HARD TO BEAT. 

Exchange Traded Funds (ETFs), most of which are tied to index funds, are continuing to rake in money.  How do you feel about ETFs? 

I like ETFs if they are broad-based index funds. I do not favor the very specialized ones that really represent active management. I believe that the leveraged ones (such as three times the up or down movement of the market) are really gambling contracts, not investment products. 

Last time we spoke you said the next revolution after index funds is in investment advice. Most advisers charge 1% or more. When it’s done online it is 0.25%.  Are there any signs that the “advice revolution” is happening?

I think a revolution in investment advising services is underway, just as indexing itself was a revolution.  Investment advisors charge between 1 and 3 percent per year to manage portfolios for individuals. Software companies can do it effectively for 25 basis points for those who accept fully electronic management and 50 basis points for a hybrid model where you can also talk to a human being from time to time. I work with Wealthfront (a fully electronic) and Rebalance (a hybrid) investment manager. The electronic or computer managers can also provide direct indexing where the fund can efficiently harvest tax losses while maintaining a pre-tax exposure to the broad market. 

A couple years ago you wrote an op-ed in the Wall Street Journal very critical of ESG (Environmental, Social and Governance) funds, saying they were a “self-defeating strategy.” Since then, they have come under even more scrutiny. Do you still feel that way?  Why?

ESG funds promise that you can do good for society with your investments and do well financially at the same time. But there is no agreement over what is a “good” investment. Is a natural gas company good because it is the cleanest burning carbon and the bridge we will need for decades on the road to a carbon-free world? Or is it bad because it is a pollutant? Is a munitions maker good because it is providing Ukraine with defensive weapons or bad because its products kill people? Are Meta and Visa good because they are not major polluters, or are they bad because they can cause extreme anguish for teenagers and because they charge exorbitant interest rates to poor people? ESG funds also have high fees and they have been underperforming standard index funds. 

You have always preached of the benefits of a diversified portfolio.  What should a diversified portfolio look like?

Portfolios should be broadly diversified but will be different for people in different circumstances.Young people should dollar cost average by investing regularly and almost exclusively in equity index funds (60/40 stocks to bond for young people, is not appropriate). An investor in his/her 70s and 80s taking required minimum distributions needs a larger proportion of limited duration fixed income. 

The S&P was down almost 20% last year. In years when it has been down 20%, it typically bounces back the following year. What can we expect for stocks in 2023?

I do not make short-term stock market predictions. No one can do this correctly with any consistency. But cyclically-adjusted price-earnings multiples (CAPEs) give the best forecasts for long-run equity returns.  Today CAPEs are well above average. This suggests that returns over the next decade are likely to be below the 9%-10% long-run historical stock market returns. Investors then need to be modest in their expectations and consider that returns could be only half the historical average.

Two classic books on long-term investing are out in new editions

Two classic books on long-term investing are out in new editions

Burton Malkiel, a Princeton University economics professor, in New York, Jan. 26, 2010.

Julie Glassberg | The New York Times

Want to learn how to be a better investor in 2023?  Two classic books on long-term investing are out in new editions. If your New Year’s resolution is to learn more about the stock and bond markets, you cannot do better than to read these books. 

Here’s why every investor should read them. 

related investing news

Is the bottom in for the bear? Market valuation measures suggest not

CNBC Pro

This week, Princeton professor Burton Malkiel has published the 50th Anniversary Edition of A Random Walk Down Wall Street: the Best Investment Guide that Money Can Buy. It’s hard to underestimate the impact this book has had on the investment community. It was first published in 1973, and by the time I met Burton Malkiel in the late 1990s it had been in print for 25 years and was already an investment classic. More than any other book, it popularized the idea of indexing as an investment strategy and why you can’t beat the market. Malkiel was a close friend of Jack Bogle and spent 28 years on the board of Vanguard. This is all-new updated edition. 

In December, the Wharton School’s Jeremy Siegel published a new (6th) edition of his classic, Stocks for the Long Run: The Definitive Guide to Financial Market Returns & Long-Term Investment Strategies. First published in 1994, Siegel examined stock and bond returns going back 200 years and concluded that, on average, stocks produced inflation-adjusted returns of 6.5%-7% a year, far outperforming bonds. This was a pivotal study that helped convince many that a simple buy and hold strategy was the best long-term investment. 

Two other investment classics round out my list of “must-reads” for long term investors. 

Common Sense on Mutual Funds by Jack Bogle (10th Anniversary Edition, 2009). Meeting Jack Bogle in the mid-1990s changed my life. It was a time of superstar investors like Bill Miller at Legg Mason. Bogle convinced me that: 1) for most investors low-cost index funds were the way to go, 2) the outperformance of the small (very small) group of active managers who did outperform was negated by the high fees they —charged, and 3) once you got the mix of assets right for your risk tolerance, the key was to stick with the plan and not freak out when markets dropped. More than anyone — including Warren Buffett — Bogle changed how an entire generation looks at investing. 

Winning the Loser’s Game by Charles Ellis (8th edition, 2021). Ellis founded Greenwich Associates, an international consultancy where he advised large institutional investors, in 1972 and, like Burton Malkiel, was on the board of Vanguard for many years. In 1975, he published an essay, “The Loser’s Game,” in a financial journal, in which he laid down his central thesis: “The investment management business (it should be a profession but is not) is built upon a simple and basic belief: Professional money managers can beat the market. That premise appears to be false.” In 1985 he expanded the article into a book, now called Winning the Loser’s Game: Timeless Strategies for Successful Investing. This book came to distill much of the wisdom that Malkiel, Bogle and Siegel had separately published on beating the markets, the efficient market hypothesis, market timing and asset allocation. 

Like Malkiel, Ellis urged investors to diversify into low-cost index fund investing, which was a radical idea because there were no low-cost index funds at the time! 

The market eventually caught up with Malkiel, Siegel, Ellis and Bogle. Not only did Jack Bogle launch the first successful index mutual fund (based on the S&P 500) in 1975 but, 18 years later, the first Exchange Traded Fund (ETF), also based on the S&P 500, was launched. Investors now had not just an index fund, they had a low-cost, tax-efficient wrapper they could buy it in. 

Since then, “The Index Revolution”, as Charley Ellis called it, has only grown. ETFs are now a nearly $7 trillion business, and closing in on the shrinking mutual fund business. For investors confused by the constant noise and the need to “do something,” these books provide a calming antidote. 

Note:  Burton Malkiel will appear on CNBC’s “The Exchange” this Friday at 1:30 PM ET.

SEC pushes ahead with slate of proposals to boost corporate reporting, disclosure requirements

SEC pushes ahead with slate of proposals to boost corporate reporting, disclosure requirements

A flag outside the U.S. Securities and Exchange Commission headquarters in Washington, D.C., U.S., on Wednesday, Feb. 23, 2022.

Al Drago | Bloomberg | Getty Images

While Washington is consumed with the elections next week, the Securities and Exchange Commission is continuing to consider numerous rulemaking proposals that could significantly increase the reporting and disclosure requirements of corporate America.

The SEC is considering two proposals Wednesday. One would adopt additional rules around how companies report proxy votes. Among other measures, the proposal could require funds to categorize the votes so they are easier to find and provide the vote outcome in a machine readable format.

A second would address the redemption of mutual fund shares and address the liquidity needs for funds when there are significant redemptions.

These are fairly technical proposals. However, SEC Chair Gary Gensler has more than two dozen other proposals that are in final rule stage, meaning the agency is still considering public comments, but they could be voted on and adopted sometime in the near future. These include pay vs. performance, climate change disclosure, cybersecurity risk governance, proxy voting advice, share repurchase disclosures, money market fund reforms, short sale disclosure reforms, and the loan or borrowing of securities.

These proposals are moving slowly from the “proposed” stage to the “adopted” stage; all or part of his agenda could be adopted in 2023.

If there is a single theme emerging from the SEC under Gary Gensler, it would be that he is seeking many new rules that would provide increased disclosures for everything from climate risks to private equity. Gensler has said this would increase market transparency.

Republicans disagree. There is likely to be vociferous pushback on more controversial proposals such as climate disclosure.

The elections next week are unlikely to change that. While the Republicans could harass the SEC by calling for more hearings next year, or could also refuse to authorize the higher SEC budget Gensler has pushed for, he has a 3-2 majority and will continue to maintain that edge into 2023.

However, Republicans could turn to the courts. It’s key to remember that last June, in West Virginia v. Environmental Protection Agency, the Supreme Court ruled that the “major questions doctrine” limited the scope of powers that Congress granted to the Environmental Protection Agency under the Clean Air Act. The Supreme Court said that Congress must provide clear direction to the EPA, not just a broad delegation of power, to regulate greenhouse gas emissions.