Our Final Invention

I received this book as part of the First Reads giveaways and was very excited to read it. The book wasn't exactly what I expected, and I'm not sure I agreed with everything Barrat proposes, but it did make me think about AI in a way I never had before, and that was, I believe, at least half his purpose. The book is an easy read for a layperson, and a brilliant foundation for a science fiction writer. Mostly, the book made me want to write about a superintelligent machine race that has been steadily taking over the universe, but that could just be me. The book itself is more concerned with the machine race evolving possibly right under our noses.

Barrat starts by immediately addressing Asimov's Three Laws. Thankfully, there are scientists out there who remember that A) Asimov wrote fiction and B) he invented the Three Laws as plot contrivances and C) Most of his plots revolve around robots finding ways around those three laws. From there, Barrat talks about his Busy Child Scenario of an artificial superintelligence that quickly outgrows humanity and finds better uses for our molecules.

The main points I agree with are that if a Singularity does occur, it's going to occur in the military where the research is happening behind closed doors. Artificial General Intelligence is not going to be created by a group of ragtag scientists looking to build a better world, but by corporate military looking for new ways to kill people. With that foundation, the idea of an intelligence explosion is pretty terrifying.

I'm still a bit lost on the idea of an ASI's drives. Some of them seem to be anthropomorphizing robots in exactly the way Barrat cautions against. That said, I could see a lot of those drives being programmed in with the best of intentions, and then running away with the future. I have no trouble believing that humans could accidentally create a learning machine without failsafes that would keep it from disrupting the power grid or financial markets. These are the scariest parts of the book, and the ideas that need to get to the people at the center of the action.

I'm not at the center of that action, so I am not entirely sure what to do upon finishing the book. Siri may be the most advanced piece of AI ever in a suburban home, but I still have to repeat myself to her six times to be understood, so from my vantage, ASI is a long way away. That said, humans have a history of messing with forces beyond our ken and thus causing horrific accidents. While I'm not convinced an accident on the extinction level Barrat proposes is the most likely outcome, I do think that letting the machines literally do all the thinking could have grave and immediate consequences. I hope that this book and its sources are required reading behind those closed doors where the Singularity is a real, achievable goal.