Silicon Brains, Photonics, etc.

The semester is officially over, which is exciting: I finally get to get back into the swing of research (with the occasional GRE study break, of course). As such, blogging will tend to be lighter; but before I lock myself in the lab, here are a few things that came to pass while I was busy finishing up the semester…

Building Brains in Silicon
Among other things, I wrote a paper for my computational neuroscience class on – you guessed it – some really cool work coming out of Kwabena Boahen’s group (formerly here at Penn, now at Stanford) on silicon-based artifical neural systems. This is sometimes classed as ‘neuromorphic engineering’, a term (coined by Carver Mead in the 1980’s) which has come to refer to a relatively recent interdisciplinary paradigm dealing with the development and study of artificial neural systems, drawing on principles from such fields as physics, biology, and computer/electrical engineering to design electronic-based analogues of biological systems. A number of people are using this to try to design new VLSI-based systems based on biological systems.

Some others are trying to reverse this scenario: while ‘real’ neural systems are experimentally studied by neurobiologists while grossly simplified ones are modeled by computational neuroscientists, groups like Boahen’s are trying to bridge these modes of inquiry by exploiting similarities between electronic and neural circuits. Mahowald and Douglas wrote a seminal paper in 1991 describing the first ‘silicon neuron’, and a good deal of work has gone on since then. For example, a number of ‘thermodynamic’ models of ion channels have been developed, building on concepts like Hodgkin/Huxley-type models. Anyway, by exploiting the beautiful similarity between ion channels and metal-oxide-semiconductor (MOS) transistors as two-state systems (simplistically, ion channels are either open or closed, with the energy barrier – and hence the transition rate – between the two states being modulated via, for example, a voltage; on the other hand, a voltage applied across the source and the drain of a MOSFET causes charges to diffuse through the ‘conduction channel’, with the effective barrier to this diffusion being modulated by a gate voltage), Boahen and his graduate student Kai Hynna have recently taken an important step toward ‘building a brain in silicon’. Using an approach that combines the advantages of experiment and artificial modeling, they have developed a simple electronic circuit that replicates the nonlinear dynamics of the gating particles of voltage-dependent ion channels.

References:
- Hynna and Boahen’s recent paper: K. M. Hynna and K. Boahen, Neural Computation 19, 327 (2007).
- 1991 silicon neuron paper: M. Mahowald and R. Douglas, Nature 354, 515 (1991).
- Thermodynamic models of ion channels: A. Destexhe and J. R. Huguenard, J. Comput. Neurosci. 9, 259 (2000).

Update: I guess Tech Review thought this stuff is cool, too: the latest issue has an article on Boahen’s work. It takes a broader view of his work than I have above - I just focused on one particular aspect.

Quasicrystals and Complex Materials as 3D Photonic Structures
I wrote another paper for my modern optics class, based on this recent experimental paper by Man, Megens, Steinhardt and Chaikin on three-dimensional quasicrystals as complete photonic bandgap materials. Here’s the deal: since Schrödinger’s wave equation and the electromagnetic wave equation are formally similar (neglecting spin statistics), it isn’t all that surprising that a number of analogies exist between electronic waves and light. In particular, electromagnetic waves can propagate in structures of periodic dielectric constant, and interference due to multiple Bragg reflections from these interfaces leads to directional-dependent energy band gaps. A major goal is to try to develop artificial structures to act as complete, omnidirectional photonic bandgap (PBG) crystals with bandgaps in the visible regime (wavelength ~ 400-700nm), and a lot of effort has gone into this. Interestingly, recent innovations in materials science and the study of complex materials – such as quasicrystals (QC), liquid crystals (LCs), and colloidal self-assembly – have breathed new life into this quest.

References:
- Experimental confirmation of the almost-spherical effective Brillouin zone (and hence the potential of developing a 3D PBG structure) of a macroscopic 3D icosahedral photonic QC: W. Man, M. Megens, P. J. Steinhardt and P. M. Chaikin, Nature 436, 993 (2005).
- Experimental approach towards assembling 3D analogues of the QC structures studied by Man et al. on a smaller scale using holographic optical trapping: Y. Roichman and D. G. Grier, Opt. Exp. 13, 5434 (2005).
- Another experimental approach, using a novel 7-beam optical interference holography technique: W. Y. Tam, Appl. Phys. Lett. 89, 251111 (2006).
- Using nematic liquid crystals in ‘inverse opal’ structures as PBG materials (tuned by parameters such as an external electric field) – for example, since liquid crystals are birefringent, modulating their orientational order using a field can influence their optical properties (a principle on which liquid crystal displays are based): K. Busch and S. John, Phys. Rev. Lett. 83, 967 (1998).
- Recent computational work has indicated a feasible method of fabricating 3D visible PBG crystals with two different types of lattice structure using self-assembly of a mixture of colloidal spheres of two different sizes: A. P. Hynninen, J. H. Thijssen, E. C. Vermolen, M. Dijkstra, and A. van Blaaderen, Nature Mater. 6, 202 (2007).

Neural Networks III

Last week I wrote the second post in my ongoing series of posts on Computational Neuroscience, based on a class I’m taking on the subject. I take notes on my computer in class anyway, so what I post here is a modified version of those and hence a reasonably good course chronicle.

While a lot of the development can be very technical, I’m trying to make these posts accessible to most people with a general interest in biophysics, neural networks or computational neuroscience. This means that I have to gloss over a lot of interesting details, including a lot of fun derivations, but that’s ok - for one thing, I don’t know how to easily incorporate equations into these posts (nor do I have the time to). However, I do provide references to more detailed papers when I feel the need.

Last week’s subject was Mikhail Bongard’s problems of pattern recognition, Herrnstein’s discovery that pigeons were particularly adept at solving them, more perceptrons and the credit assignment problem, and how backpropagation provided a means of solving it. The week before, I introduced McCulloch-Pitts neurons and perceptrons, particularly the perceptron convergence theorem and linearly separable problems. This week builds on the past two weeks: we’ll be looking at NETtalk, the Ising Model and Hopfield Networks, and Attractor Dynamics.

Neural Networks II

Last week’s subject was McCulloch-Pitts neurons and perceptrons, particularly the Perceptron Convergence Theorem and linearly separable problems. This week builds on that, and this post’s subject is Pigeons, the Credit Assignment Problem, and Backpropagation.

Neural Networks I

Yesterday was my first computational neuroscience class, a field that is totally foreign (and interesting) to me. We started off talking about neural networks, a field which has its roots in systems theory and the work of Nicholas Rashevsky, “a Russian émigré theoretical physicist who developed a program in ‘mathematical biophysics’ at the University of Chicago during the 1930s”. For example, he was one of the first scientists to analyze traffic mathematically (which confuses me a bit, since I’m not sure how much of an issue traffic was in the ’20’s or ’30’s).

At any rate, work building on that of Rashevsky and others plodded along for a while, until the seminal work of Warren McCulloch and Walter Pitts at MIT (A Logical Calculus of Ideas Immanent in Nervous Activity, 1943, Bulletin of Mathematical Biophysics 5:115-133) while considering the question: what kind of things can be computed mathematically? Today the subject of their paper is often referred to as a McCulloch-Pitts neuron.

A McCulloch-Pitts neuron is quite simple: essentially, it is a simplified tunable linear threshold unit (the Wikipedia article I’ve linked to does a particularly good job at explaining it), and what McCulloch and Pitts showed was that with a network of these, one can compute any logical expression, or any combination of logical predicates. The device is tunable in the sense that each input can be weighted (and a tunable ‘bias’ is often applied, too), and by varying the different weights with respect to a given threshold, the operation being computed can either be AND, OR, NOT, etc. Clearly this is incredibly simplified - no probabilities, no further considerations in the mechanism, nothing but the bare bones. The fact that a network of these can be so powerful was a big deal in the ’40’s, and people began wondering whether such ‘neural networks’ could be made to act like a brain. Clearly the jump from simple logical operations using a network of McCulloch-Pitts neuron to advanced concepts (like pattern recognition) is huge.

Twenty years later, Frank Rosenblatt - an engineer at Cornell - came up with what he called the Perceptron. Essentially, Rosenblatt was looking for a way to come up with an image recognition machine, equipped with some programmatic behavior to make it better as time went on. This resembles neural networks to a large extent - here’s a picture from Wolfram research:

As in McCulloch-Pitts neural networks, the weighted inputs (and bias, which in the image is 1) are summed, and assessed against a given threshold. Each input comes from a set of ‘feelers’ on the screen where the image is displayed e.g. photoreceptors. Things get a teensy bit complicated, and I’m trying to keep this equation-free, so I’ll jump to the punchline: Rosenblatt was able to show that for a linearly separable problem (that is, trying to distinguish between two different things that form different ‘clumps’ in an n-dimensional parameter space that can be separated by an n-1 dimensional hyperplane), the Perceptron only takes a finite number of examples for it to learn successfully. This Perceptron Convergence Theorem was again a Very Big Deal, and led to a good deal of research in the field for a little under 10 years…

Until it all stopped. Why? Because of a monograph (Perceptrons) - published by Minsky and Papert (at MIT as well) - that put an end to it all, and gave rise to the New and Improved (i.e. ‘hot’) field of Artificial Intelligence. For many years after that, the field of neural networks was completely and utterly dead.

Until next time…