Scalable External Memories for Neural Networks
Time:
Abstract
Neural Virtual Machines, by emulating Harvard architecture, have integrated the traditional computer architecture into modern neurocomputational systems, allowing non-local representations and efficient local learning. However, their memory update operations face significant scalability challenges. This work explores two research directions motivated by these limitations. The first investigates the feasibility of single-pass full-capacity learning in neural systems and demonstrates, under span-rule-constrained linear threshold models, the nonexistence of such learning rules. The second explores alternative memory architectures based on vector-symbolic representations and introduces an efficient clean-up method for vector-symbolic key-value memory with linearithmic complexity. Together, these directions highlight both theoretical limitations of scalable neural memory systems and potential approaches toward more efficient and interpretable memory-based learning architectures.
