Intuition Machine - Issue #31

Welcome to the weekly newsletter of Deep Learning. We will be making a new change and pivot slightly from the original focus of just covering Deep Learning. We will now cover both Deep Learning as well as its convergence with Blockchain technology.

We’ve come up away to organize the topics to appeal to the broadest of audiences. The more general topics always are at the top, while more specialized ones are towards the bottom. We hope that this newsletter we appeal to all those interested in Deep Learning developments and its relationship to decentralized consensus architectures.

Scientists are using artificial intelligence (AI) to help predict which embryos will result in IVF success. In a new study, AI was found to be more accurate than embryologists at pinpointing which embryos had the potential to result in the birth of a healthy baby.

Pointer networks are a variation of the sequence-to-sequence model with attention. Instead of translating one sequence into another, they yield a succession of pointers to the elements of the input series.

Today I would like to post a more technical and pure engineering topic. The heart of the matter in Artificial Intelligence(AI) is more practical/empirical based than theoretical. Even though the conceptual framework is undoubtedly important.

The default approach to initializing the state of an RNN is to use a zero state. This often works well, particularly for sequence-to-sequence tasks like language modeling where the proportion of outputs that are significantly impacted by the initial state is small. In some cases, however, it makes sense to (1) train the initial state as a model parameter, (2) use a noisy initial state, or (3) both. This post examines the rationale behind trained and noisy intial states briefly, and presents drop-in Tensorflow implementations.

It suggests that even abstract knowledge is stored in relation to some form of “location” and that much of what we consider to be “thought” is implemented by inference and behavior generating mechanisms originally evolved to move and infer with fingers and eyes.