Image Credit: iStock.com/Henrik5000
If you have used your cell phone’s personal assistant to help you find your way or taken advantage of its translation or speech-to-text programs, then you have benefitted from a deep learning neural network architecture. Inspired by the human brain’s ability to learn, deep learning neural networks are based on a class of machine algorithms that can learn to find patterns and closely represent those patterns at many levels. As additional information is received, the network refines those patterns, gains experience, and improves its probabilities, essentially learning from its mistakes. This is called “deep learning” because the networks that are involved have a depth of more than just a few layers.
Basic deep learning concepts were developed many years ago; with today’s availability of high performance computing environments and massive datasets, there has been a resurgence of deep learning neural network research throughout the science community. Scalable tools are being developed to train these networks, and brain-inspired computing algorithms are achieving state-of-the-art results on tasks such as visual object classification, speech and image recognition, bioinfomatics, neuroscience, language modeling, and natural language understanding.
Improvements in computational energy efficiency and throughput are being realized in neurosynaptic or cognitive neural network architectures. A great example of this is the Lawrence Livermore National Laboratory (LLNL) and IBM Research collaboration to build a new brain-inspired supercomputer. The hardware and software ecosystem is based on IBM’s breakthrough neurosynaptic computer chip called TrueNorth. LLNL has been testing the scalable platform and has noted that TrueNorth processes the equivalent of 16 million neurons and 4 billion synapses and consumes the energy equivalent of a hearing-aid battery. The TrueNorth ecosystem encompasses multiple-chip machines that seamlessly integrate with a software language, a programming environment, an algorithm and application library, and tools for neural network composition. TrueNorth is currently in use at over 30 universities and government/corporate labs.
Additional information about DOE-funded deep learning research and technical details are provided in Dr. William Watson’s latest white paper, “In the OSTI Collections: Deep Learning.” DOE’s March 2017 Science Showcase also features deep learning research results available in DOE databases and related links of interest.