Pentagon funds our robot masters
To get our regular email updates, simply click here.
In dystopian science fiction movies, it always starts this way: The Pentagon funds a contest of machines built to play war games using artificially intelligent software. End of civilization ensues.
This time, in real life, teams of enthusiastic young hackers met in Las Vegas at Defcon. Taking its name from the numerical naming sequence for defense alerts, it’s really a computer game of Catch the Flag. The idea is to find, plug and defend security holes in your own server while seeking and exploiting the vulnerabilities of other teams.
Players get bragging rights, a shot at winning some cash and a showcase for their talents. This year’s DARPA Cyber Challenge at the Paris Hotel on the Vegas strip reimagines that concept with code writing, artificially intelligent software robots running on supercomputers.
AI as a science is gaining momentum because researchers understand the potential of thinking software. In theory, it could recognize context and build solutions on the fly to take advantage, saving humans the time and trouble. While that’s all well and good, there is another way to think about artificial intelligence. And it’s not good.
Although he suffers from ALS and needs AI software developed by Intel (INTC) to communicate, renowned physicist Stephen Hawking worries such software would lead to a constant state of reinvention for machines.
The growth of intelligence would be exponential, only constrained by irrational, slow-witted human overlords. Machines might seek human eradication in an effort to build efficient machines/solutions. As Hawking puts it: "A super intelligent AI will be extremely good at accomplishing its goals, and if those goals aren't aligned with ours, we're in trouble."
The organizers of the Cyber Challenge see things differently. The Pentagon-funded event spent $3.75 million in prize money and another $55 million organizing. That entailed recruiting seven teams of security researchers from industry to academia, designing and building supercomputers for each, andsoftware that could be used to watch over their software bots perform in real time. It was all very impressive and, according to organizers, well worth the money spent.
One software bot even managed to find and inoculate the Crackaddr bug, a piece of decade old malware that had never been cracked before by a non-human.
A bot programmed by engineers from Raytheon (RTN) played an especially aggressive game. After it patched holes on its own server it pounced on exposed vulnerabilities on multiple fronts, scoring points. However, its own deficiencies ultimately slowed processing power.
In the end the contest winner came from Carnegie Mellon University. Its bot, named Mayhem, took the opposite approach. It ran up an impressive lead early on offense before effectively shutting down halfway through the competition.
The commentators reasoned the bot chose this course because it determined there was no further benefit to be derived by patching and exploiting new holes. The most prudent course of action was to simply wait as other teams exposed vulnerabilities and could no longer effectively respond.
While it is impressive that a piece of software can reason at all, it’s also disconcerting. I don’t want to seem like an alarmist or pessimistic given the very bright investment outlook for new technologies.
Yet I hope the next wave of artificially intelligent software doesn’t simply wait for humans to reach the point we can no longer respond to exposed vulnerabilities. That would not lead to a Hollywood ending.
##
About Jon Markman: A pioneer in the development of stock-rating systems and screening software, Jon Markman is co-inventor on two Microsoft patents and author of the bestselling books The New Day Trader Advantage, Swing Trading and Online Investing, as well as the annotated edition of Reminiscences of a Stock Operator. He was portfolio manager and senior investment strategist at a multi-strategy hedge fund from 2002 to 2005; managing editor and columnist at CNBC on MSN Money from 1997 to 2002; and an editor, investments columnist and investigative reporter at the Los Angeles Times from 1984 to 1997.
To get our regular email updates, simply fill in the form below:
