Insider trading is deeply woven into the fabric of Wall Street. Recent revelations about the illegalities of information flows at expert networks and hedge funds like SAC Capital signal, I hope, a long overdue decline in amoral activities that Wall Street insiders have exploited for decades to score outsized riches while the average investor is often left holding the bag.

This article explains how the average investor can and should even the playing field with Wall Street insiders.

Wall Street’s Advantage

It’s important for investors to understand that buying and selling on news is a good way to hand your cash over to Wall Street. Large investment companies have spent millions of dollars on high speed trading systems that allow them to process incoming news and make trades in milliseconds. Goldman Sachs criminally charged a former programmer for taking some code that may have shaved a few milliseconds off the time it takes to make a trade.

When investors trade on short-term news stories, Wall Street gets to front-run them and make enormous profits. Check out this article from MarketWatch, which describes the University of Michigan/Thomson Reuters consumer report being sold to high frequency traders two seconds before the general public. Once people realized this was happening, they stopped trading on the news, and high-frequency trading dried up as there was nobody to front-run.

Unfortunately, there are still plenty of data points that investors trade on, perhaps not knowing that Wall Street knows about it well before they do. The grand jury indictment against SAC Capital includes an email from a portfolio manager to Cohen, which reads: “I am very comfortable that this qtr is going to be solid vs current consensus and guidance. I am getting coffee on [T]ues[day] afternoon with the guy who runs north American generics business.” In another email that was forwarded to Cohen, a manager recommended selling Dell stock based on a “2nd hand read from someone at the company.”

The indictment states that Cohen began selling Dell (DELL) 10 minutes after receiving the email. SAC is by no means the only firm engaged in these sort of practices — just the only one that has been caught.

Level the Playing Field – With Proper Diligence

The long history of insider trading makes it hard not to be cynical about the stock market. The Washington Post’s Dylan Matthews goes so far as to suggest that insider trading should be completely legalized — just to make it clear to individual investors how high the odds are stacked against them. Unless a more effective enforcement mechanism can be put in place, it’s possible that Matthew’s idea might just be the best solution. Investors need to realize the obstacles in place and the advantages that large institutional investors have. Most investors cannot just grab a coffee with a department head in the company they’re considering buying stock in.

So how should investors react? Is the solution, as Matthews suggests for all investors to simply avoid trying to pick stocks by investing only in passive index funds? I don’t think so. Much of Wall Street has been earning easy money for decades, leveraging their massive resources and inside contacts to beat the market. As a result, these firms have gotten progressively worse at doing high quality research by studying corporate filings and analyzing accounting data.

As government restrictions continue to tighten, fewer firms have the contacts and resources to beat the market without high quality research. More investment professionals — and individual investors — are realizing the importance of performing due diligence on corporate filings. The problem is that due diligence is extremely hard work. If it were easy, everyone would be doing it. Form 10-Ks can be up to a thousand pages long. Many investors just look at the income statement or the balance sheet. They hardly have time for the financial footnotes, which contain a great deal of significant information.

It is almost impossible to properly analyze these filings by hand, which is why New Constructs developed our patented research system that allows our small team of analysts to go through over 3,000 Form 10-Ks in a year with the help of automated computer systems. We identify and reverse accounting distortions to convert GAAP data into economic earnings to give investors a better sense of the value companies are creating for shareholders. With due diligence and a proper long-term mindset, investors can level the playing field and give themselves a chance to compete with Wall Street.

Our research and models do not make investing easy. Hard work is still necessary to properly understand and analyze and a company. What New Constructs does is make diligence practical. Investors cannot compete with the Steve Cohens of the world based on resources, contacts, or sources of information. They can, however, compete on diligence.

Appreciate Your Competition: A Brief History of Insider Trading

Diligence is rarer than one might expect on Wall Street, mainly due to the fact that it hasn’t been necessary for a long time. Until the creation of Regulation Fair Disclosure (RegFD) by the SEC in 2000, insider trading was a core business on Wall Street. It was common practice for companies to inform Wall Street analysts (usually in order of favor) whenever they expected to beat or miss consensus EPS estimates. In turn, Wall Street firms set up trades to exploit this insider info while the analysts shared it with their institutional clients (usually in order of who paid them the most). That is how things were for about as long as there was a “consensus” estimate.

The idea that suddenly Wall Street would stop engaging in insider-trading-like activities simply because of Reg FD is naïve. We are talking about habits, very profitable ones, that are decades old. You may ask: How did long-standing insider trading laws not cover the rampant illicit behavior described above? Good question. And you could ask them same question about the nefarious trading in mortgage-backed securities during the latest financial meltdown. You could ask the same question about the analysts who wrote glowing research reports on technology company IPOs in the late 1990s, while in the same breadth emailing friends message like “I cannot believe anyone is stupid enough to invest in this piece of …”

Usually, the reason for no prosecution is that no laws were broken. Too often, there has been no specific law broken because the SEC and other regulators are not able to create laws fast enough to keep up with how quickly things change on Wall Street. For example, per this May 11, 2010, report, big Wall Street banks and large hedge funds exploit their access to more information to front-run unsuspecting investors. The takeaway is that certain exchanges (NYSE and Nasdaq by admission) are selling information about investors without their knowledge. Buyers of this information use it to anticipate trades — i.e., front run. The trading intentions of thousands of investors (which is what the exchanges sell) can be a quite reliable indicator of short-term price direction, which means the buyers of the info are legally engaging in insider trading, in my opinion.

Is this practice technically illegal? No, there is no law on the books that says the practice is not allowed. And no law is being written at this point because powerful Wall Street lobbyists argue that the information in question is public and available to all investors. From the report: “Technically, this may be true, however, realistically, not many retail or institutional investors have the capital to invest in the type of computer systems needed to access this information and most are not even aware that it exists at all. They also are not aware that the information is being provided to HFT’s [traders] revealing critical information about trading intentions.”

This situation is a perfect example of how effectively Wall Street is able to walk the line between what is actually illegal and what should be illegal and earn outsized profits all along the way. So far regulators have not been able to keep up with the innovative methods Wall Street has develop to obtain an inside edge. Reg FD more strictly defined insider trading and made it illegal to trade on information that was not available to the general public. Institutional investors could no longer learn about quarterly earnings ahead of retail investors. This change did not, however, stop insider trading. It simply meant that Wall Street had to get a little more creative.

“Expert networks” for a while were a favored way for Wall Street traders to get inside information on companies while distancing themselves from legal liability. While many of these firms operated legitimately, others were simply cover for firms to provide specialized information to hedge funds and connect fund managers to industry experts. Often, these “experts” are actually individuals with inside information on a company.

For instance, $276 million in profits or avoided losses were made by an affiliate of SAC in 2008 after a paid consultant for an expert network firm-who just so happened to be overseeing the clinical trial of a major Alzheimer’s drug-provided confidential information on the trial to a hedge fund portfolio manager. The grand jury indictment described SAC’s insider trading scheme as “on a scale without known precedent in the hedge fund industry.” That statement is true only in the unfortunately naïve eyes of the grand jury.

Sam McBride contributed to this article

Disclosure: David Trainer and Sam McBride receive no compensation to write about any specific stock, sector, or theme.